A method, device and system for automatic monitoring on-site of fully mechanized mining

Abstract

The present application relates to the field of coal mine system monitoring technology, and in particular to a method, device and system for automatic monitoring of a comprehensive mining site, the method for automatic monitoring of a comprehensive mining site comprising: obtaining server configuration information; the server configuration information comprises: video service configuration information, camera configuration information and network load configuration information; based on the server configuration information, obtaining monitoring data, the monitoring data comprises: video service performance data, video stream and load data; executing monitoring strategies corresponding to the monitoring data; the monitoring strategies comprise: video service performance control strategies, camera control strategies and load control strategies. By obtaining different monitoring data and executing corresponding monitoring strategies, the video monitoring function can be monitored in real time on the well, and video service performance anomalies, camera anomalies and load anomalies can be processed, so as to ensure the normal operation and stability of the video monitoring function.

Description

Technical Field

[0001] The present application relates to the technical field of coal mine system monitoring, and in particular to a method, device and system for automatic monitoring on-site of fully mechanized mining. Background Art

[0002] With the rapid development of smart mining business, the fully mechanized mining automation system has been widely recognized in the industry, and the order volume has increased significantly. During the on-site maintenance of the fully mechanized mining automation system, through video monitoring, the operation of various equipment and systems in the fully mechanized mining face can be observed in real time, such as the working status of coal mining machines, supports, pump stations, scraper conveyors, crushers, transfer machines, belt conveyors and other equipment, providing intuitive image information for the safe production of coal mines.

[0003] However, there is currently no tool to monitor the video monitoring function of the comprehensive mining automation monitoring system, so it is impossible to monitor the video monitoring function in real time on the well, and it is impossible to ensure the normal operation and stability of the video monitoring function. Summary of the invention

[0004] The present application aims to at least solve one of the technical problems that there is currently no tool to monitor the video monitoring function of the comprehensive mining automation monitoring system, the video monitoring function cannot be monitored in real time on the well, and the normal operation and stability of the video monitoring function cannot be guaranteed.

[0005] To this end, the present application provides a method for automatic monitoring of comprehensive mining sites. By acquiring different monitoring data and executing corresponding monitoring strategies, it can monitor the video monitoring function in real time on the well, and handle situations such as abnormal video service performance, camera abnormalities, and load abnormalities, thereby ensuring the normal operation and stability of the video monitoring function.

[0006] The present application also provides an automatic monitoring device and system for comprehensive mining site including the above-mentioned automatic monitoring method for comprehensive mining site.

[0007] According to an embodiment of the first aspect of the present application, a method for automatic monitoring of a comprehensive mining site is applied to a monitoring server, including: obtaining server configuration information; the server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information; based on the server configuration information, monitoring data is obtained, and the monitoring data includes: one or more of video service performance data, video stream and load data; executing a monitoring strategy corresponding to the monitoring data; the monitoring strategy includes: one or more of a video service performance control strategy, a camera control strategy and a load control strategy.

[0008] Optionally, when the monitoring data to be acquired is video service performance data, the monitoring data is acquired based on the server configuration information, including: acquiring the video service performance data based on the video service configuration information; the video service configuration information is the configuration information of the video service on each execution server; the video service includes one or more of the WebAPI service, the Nginx service, the camera PTZ service, and the WebRTC service; the video service performance data is the system resource data occupied by the video service when each execution server is running; the system resource data includes one or more of the CPU usage, the memory usage, and the hard disk I / O usage;

[0009] The monitoring strategy corresponding to the execution monitoring data includes: determining whether the system resource data occupied by the video service performance data exceeds a first threshold set on each execution server; if so, restarting the video service on the corresponding execution server.

[0010] Optionally, when the monitoring data obtained is network load configuration information, the monitoring data is obtained based on the server configuration information, including: obtaining load data based on the network load configuration information; the network load configuration information includes the total number of cameras and one or more execution servers configured with the WebRTC service; the load data includes the number of connections of cameras on the WebRTC service in the execution server;

[0011] The monitoring strategy corresponding to the execution monitoring data includes: determining whether the load data exceeds a second threshold set on each execution server; if so, restarting the WebRTC service on the corresponding execution server.

[0012] Optionally, when the monitoring data is obtained as a video stream, the monitoring data is obtained based on the server configuration information, including: obtaining the video stream based on the camera configuration information; the camera configuration information includes one or more of a camera identifier, a network address and a resolution; the video stream includes image information captured by the camera;

[0013] Execute the monitoring strategy corresponding to the monitoring data, including: sampling from the video stream to obtain a sampled image; calculating the gradient variance value and grayscale value of the sampled image; comparing the gradient variance value and grayscale value of two sampled images obtained in a preset period to obtain a contrast value; if the contrast value exceeds a preset threshold, send an automatic cleaning instruction, which is used to control the camera to perform self-cleaning.

[0014] Optionally, obtaining a video stream based on camera configuration information includes: determining an execution server to which the camera is connected based on the camera configuration information; and reading the video stream in the cache when it is detected that the execution server stores the video stream sent by the camera in a cache.

[0015] Optionally, when it is determined that the execution server is connected to a camera and the waiting time for not receiving a video stream is greater than a preset time threshold, a camera failure alarm is sent.

[0016] Optionally, the gradient variance values ​​and grayscale values ​​of two sampling images obtained in a preset period are compared to obtain a contrast value. If the contrast value exceeds a preset threshold, an automatic cleaning instruction is sent, including: determining the percentage of the difference between the gradient variance value of the first sampling image and the gradient variance value of the second sampling image as the first contrast value; determining the percentage of the difference between the grayscale value of the first sampling image and the grayscale value of the second sampling image as the second contrast value; if the first contrast value is greater than a third threshold and the second contrast value is less than a fourth threshold, an automatic cleaning instruction is sent.

[0017] Optionally, the method for automatic monitoring at comprehensive mining site also includes: generating a monitoring result report.

[0018] According to the second aspect of the embodiment of the present application, an automatic monitoring device for comprehensive mining site includes: a configuration module, used to obtain server configuration information; the server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information; an acquisition module, used to obtain monitoring data based on the server configuration information, the monitoring data includes: one or more of video service performance data, video stream and load data; an execution module, used to execute corresponding monitoring strategies according to different monitoring data; the monitoring strategies include: one or more of video service performance control strategies, camera control strategies and load control strategies.

[0019] According to an embodiment of the third aspect of the present application, an automatic monitoring system for a comprehensive mining site includes: a camera configured to collect image information; a cleaning mechanism configured to clean the camera; an execution server configured to run a video service; the video service includes one or more of a WebAPI service, an Nginx service, a camera pan / tilt service, and a WebRTC service; a monitoring server configured to execute an automatic monitoring method for a comprehensive mining site according to an embodiment of the first aspect of the present application.

[0020] One of the above technical solutions has at least the following advantages or beneficial effects:

[0021] An automatic monitoring method for comprehensive mining site in an embodiment of the present application is applied to a monitoring server, including: obtaining server configuration information; the server configuration information includes: one or more of video service configuration information, camera configuration information, and network load configuration information; based on the server configuration information, monitoring data is obtained, and the monitoring data includes: one or more of video service performance data, video stream, and load data; the monitoring strategy corresponding to the monitoring data is executed; the monitoring strategy includes: one or more of video service performance control strategy, camera control strategy, and load control strategy. By obtaining different monitoring data and executing corresponding monitoring strategies, the video monitoring function can be monitored in real time on the well, and abnormal video service performance, camera abnormality, and load abnormality can be handled, so as to ensure the normal operation and stability of the video monitoring function.

[0022] An automatic monitoring device and system for comprehensive mining site provided in an embodiment of the present application is provided with the above-mentioned automatic monitoring method for comprehensive mining site. Since the automatic monitoring method for comprehensive mining site has the above-mentioned technical effect, an automatic monitoring device and system for comprehensive mining site provided with the automatic monitoring method for comprehensive mining site should also have corresponding technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A structural block diagram of an automatic monitoring system for fully mechanized mining site provided by an embodiment of the present application is shown;

[0024] Figure 2 A schematic diagram of a process of applying a method for automatic monitoring of a fully mechanized mining site provided by an embodiment of the present application to a monitoring server is shown;

[0025] Figure 3 A block diagram of an automatic monitoring device for comprehensive mining site provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] As mentioned above, the comprehensive mining automation system, whose full name is comprehensive mechanized coal mining automation system, is an important technology used in coal mining in the mining industry. The comprehensive mining automation system realizes comprehensive automated operation and monitoring of the comprehensive mining working face of coal mines by integrating advanced automation equipment and intelligent control systems.

[0028] The comprehensive mining automation system generally includes multiple execution servers for running video monitoring functions, providing real-time video images of the working face, and helping operators and maintenance personnel to remotely observe the equipment operating status and working environment.

[0029] When the video surveillance system is running, multiple video services will be started, such as WebAPI (Web Application Programming Interface) service, Nginx service, camera PTZ service and WebRTC (Web Real-Time Communication, network real-time communication) service, etc. WebAPI service is a network application program interface based on HTTP protocol. It uses JSON or XML format to transmit data. It is a server-side application that allows client applications to access data on the server through HTTP requests; Nginx service is used to solve the problem of high concurrent connections. It can not only act as a Web server to quickly respond to data requests, but also act as a reverse proxy to integrate with slower upstream servers to achieve load balancing, traffic distribution, streaming media processing, dynamic image adjustment, content caching and other functions; Camera PTZ service is a remote monitoring, video transmission, data storage and intelligent analysis service provided to users based on camera PTZ technology. Camera PTZ service not only improves the flexibility and comprehensiveness of monitoring, but also realizes intelligent processing and analysis of monitoring data through advanced technologies such as cloud computing and big data; WebRTC is a real-time communication technology that allows network applications or sites to establish peer-to-peer connections between browsers without the help of an intermediary, to achieve the transmission of video streams, audio streams or other arbitrary data, and the use of WebRTC service greatly simplifies the development and deployment of real-time communication.

[0030] However, after running on the execution server for a period of time, these video services will occupy too many system resources, causing the execution server to freeze, and in serious cases, it will cause the execution server to crash; and because of the harsh environment underground in the mine, it is easy for the camera to be covered with dust, resulting in a decrease in the image quality of the video, thus affecting the normal operation of the video monitoring function. Currently, there is no tool to monitor the video monitoring function of the comprehensive mining automation monitoring system, and it is impossible to monitor the video monitoring function in real time on the mine surface, and it is impossible to ensure the normal operation and stability of the video monitoring function.

[0031] In order to solve at least one of the technical problems existing in the prior art or related technology, the present application provides a method, device and system for automatic monitoring of the comprehensive mining site, which is applied to the monitoring server and includes: obtaining server configuration information; the server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information; based on the server configuration information, obtaining monitoring data, the monitoring data includes: one or more of video service performance data, video stream and load data; executing the monitoring strategy corresponding to the monitoring data; the monitoring strategy includes: one or more of video service performance control strategy, camera control strategy and load control strategy. By obtaining different monitoring data and executing the corresponding monitoring strategy, the video monitoring function can be monitored in real time on the well, and the video service performance abnormality, camera abnormality and load abnormality can be handled, so as to ensure the normal operation and stability of the video monitoring function.

[0032] The following describes, with reference to the accompanying drawings, methods, devices and systems for automatic monitoring of comprehensive mining sites according to some embodiments provided by the present application.

[0033] See also Figures 1 to 3 The first aspect of the present application provides an automatic monitoring method for fully mechanized mining site, which is applied to a monitoring server and includes:

[0034] S210: Obtain server configuration information; the server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information.

[0035] After the program starts running, it first detects the operating environment of the video monitoring function of the comprehensive mining automation system and obtains one or more of the video service configuration information, camera configuration information and network load configuration information.

[0036] The video service configuration information is the configuration information of the video service on each execution server; the video service includes one or more of the WebAPI service, Nginx service, camera PTZ service and WebRTC service. Since the video services running on different execution servers may be different, for example, the WebAPI service and Nginx service are running on execution server A, and the camera PTZ service and WebRTC service are running on execution server B, then the video service configuration information is that the WebAPI service is configured on execution server A, the Nginx service is configured on execution server A, the camera PTZ service is configured on execution server B, and the WebRTC service is configured on execution server B; or, the WebAPI service and WebRTC service are running on execution server A, and the Nginx service, camera PTZ service and WebRTC service are running on execution server B, then the video service configuration information is The WebAPI service is configured on execution server A, the Nginx service is configured on execution server B, the camera PTZ service is configured on execution server B, and the WebRTC service is configured on execution server A and execution server B; or, the WebRTC service runs on execution server A, and the WebAPI service, Nginx service, camera PTZ service and WebRTC service run on execution server B, then the video service configuration information is that the WebAPI service is configured on execution server B, the Nginx service is configured on execution server B, the camera PTZ service is configured on execution server B, the WebRTC service is configured on execution server A and execution server B, etc.

[0037] The camera configuration information includes one or more of the camera ID, network address and resolution. The camera configuration information can be obtained through the camera PTZ service. The camera ID is the unique identification code or name of each camera, which is used to distinguish different cameras in video surveillance. It can be numbers, letters, or a combination of numbers and letters. The network address is the information required for the camera to connect to the network. It can be an IP address (Internet Protocol Address). The IP address is the unique identifier of each camera in the network. The IP address enables the camera to communicate on the network, including the camera sending video streams to the execution server. The resolution refers to the number of pixels that the camera can achieve when capturing video images. It is usually expressed in the form of horizontal pixels multiplied by vertical pixels, such as 1920x1080. It determines the clarity and detail of the video. A higher resolution means a clearer image and can capture more details, but it also requires more storage space and network bandwidth to transmit and process this data.

[0038] The network load configuration information includes the total number of cameras and one or more execution servers configured with the WebRTC service. The total number of cameras is equivalent to the total load number; the camera transmits the video stream through the WebRTC service, and the transmission of the video stream is the most important factor occupying the network. Therefore, monitoring the network load of the video surveillance function mainly monitors the status of the cameras connected to the WebRTC service. For example, if the total number of cameras is 12 and the WebRTC service is configured on execution servers A, B, and C, it is necessary to monitor the number of cameras connected to execution servers A, B, and C.

[0039] S220: Based on the server configuration information, monitoring data is obtained, where the monitoring data includes: one or more of video service performance data, video stream and load data.

[0040] Video service performance data refers to the system resource data occupied by video services when running on each execution server; system resource data includes one or more of CPU usage, memory usage, and hard disk I / O usage. Video service performance data can be used to monitor the impact of video services on execution servers to prevent video services from occupying too many system resources, causing execution servers to freeze.

[0041] The video stream includes the image information captured by the camera. By monitoring the video stream, the dust status of the camera can be determined. Dust on the camera will cause the video image quality to deteriorate, affecting the normal operation of the video surveillance function.

[0042] The load data includes the number of camera connections on the WebRTC service in the execution server. By monitoring the load data, we can prevent the network load from being unbalanced, which may cause some execution servers or network links to be overloaded while others are relatively idle, affecting the performance, stability and user experience of the network.

[0043] S230: Execute the monitoring strategy corresponding to the monitoring data; the monitoring strategy includes: one or more of a video service performance control strategy, a camera control strategy, and a load control strategy.

[0044] By acquiring different monitoring data and executing corresponding monitoring strategies, the video surveillance function can be monitored in real time on the well, and abnormal video service performance, camera abnormalities, and load abnormalities can be handled to ensure the normal operation and stability of the video surveillance function.

[0045] In an illustrative embodiment, Figure 2 As shown, when the monitoring data to be acquired is video service performance data, the monitoring data is acquired based on the server configuration information, including:

[0046] S2310: Obtain video service performance data based on the video service configuration information.

[0047] According to the video service configuration information, use the psutil (process and system utilities, Python system and process utility) tool to obtain the process and system utilization rate of each execution server. psutil is a cross-platform Python library that can obtain information about the process and system utilization rate of the system, including CPU, memory, disk, network, etc. For example, if the video service configuration information is that the WebAPI service is configured on execution server A, the Nginx service is configured on execution server A, the camera PTZ service is configured on execution server B, and the WebRTC service is configured on execution server B, the video service performance data obtained by using the psutil tool is to obtain one or more of the CPU utilization rate, memory utilization rate, and hard disk I / O utilization rate when running the WebAPI service and Nginx service on execution server A, and one or more of the CPU utilization rate, memory utilization rate, and hard disk I / O utilization rate when running the camera PTZ service and WebRTC service on execution server B.

[0048] Execute the monitoring strategy corresponding to the monitoring data, including:

[0049] S2311: Determine whether the system resource data occupied by the video service performance data exceeds a first threshold set on each execution server.

[0050] In this embodiment, since the video services running on each execution server are different, the first threshold value set on each execution server may be different. It is understandable that the setting of the first threshold value is to prevent the operation of the video service from having a serious impact on the execution server, so it can be set according to the system resources occupied by the video service itself and the hardware configuration of the execution server. For example, the WebAPI service and the Nginx service are running on the execution server A, and the camera PTZ service and the WebRTC service are running on the execution server B. The system resources occupied by the WebAPI service and the Nginx service themselves are less than the system resources occupied by the camera PTZ service and the WebRTC service. Therefore, the first threshold value set on the execution server A can be one or more of the CPU usage rate less than 40%, the memory usage rate less than 40%, and the hard disk I / O usage rate less than 40%, and the first threshold value set on the execution server B can be one or more of the CPU usage rate less than 50%, the memory usage rate less than 50%, and the hard disk I / O usage rate less than 50%.

[0051] S2312: If yes, restart the video service on the corresponding execution server.

[0052] In this embodiment, when the system resource data occupied by the video service performance data exceeds the first threshold value set on each execution server, it means that the operation of the video service will cause the execution server to freeze, and in severe cases, it may cause the execution server to crash. For example, the CPU usage rate occupied by the WebAPI service and the Nginx service running on the A execution server reaches 50%, and the A execution server does not have enough processing power to handle the burst request. At the same time, other system processes do not have enough CPU resources to process. At this time, the WebAPI service and the Nginx service running on the A execution server can be restarted; or, the memory usage rate occupied by the camera pan / tilt service and the WebRTC service running on the B execution server reaches 60%. When the memory usage approaches or exceeds the threshold, the B execution server may start to use the swap space (swap), resulting in a sharp drop in performance. At this time, the camera pan / tilt service and the WebRTC service running on the B execution server can be restarted; or, the hard disk I / O usage rate occupied by the camera pan / tilt service and the WebRTC service running on the B execution server reaches 55%. When the hard disk I / O usage approaches or exceeds the threshold, it will affect the system performance and stability, resulting in system instability. At this time, the camera pan / tilt service and the WebRTC service running on the B execution server can be restarted.

[0053] To restart the video service, you can use PowerShell technology, which is a powerful command line tool and scripting language developed by Microsoft. It is built on the .NET Framework and can help developers automate various system tasks. When the program detects that the first threshold is exceeded, it will automatically restart the corresponding service. This setting can complete the video service performance control without human perception, will not affect the user's use of the comprehensive mining automation monitoring system, and ensure the normal operation and stability of the video monitoring function.

[0054] In an illustrative embodiment, Figure 2 As shown, when the monitoring data to be acquired is network load configuration information, the monitoring data is acquired based on the server configuration information, including:

[0055] S2320: Obtain load data based on the network load configuration information;

[0056] The WebRTC service will write the connected camera information into the database table. The pymysql technology can be used to obtain the corresponding values ​​in the database table, thereby obtaining the number of camera connections on the WebRTC service in the execution server. For example, the total number of cameras is 12. By obtaining the corresponding values ​​in the database table, it can be obtained that the number of camera connections on the WebRTC service in the A execution server is 3, the number of camera connections on the WebRTC service in the B execution server is 7, and the number of camera connections on the WebRTC service in the C execution server is 2.

[0057] pymysql is a MySQL client library written in pure Python for connecting to and operating MySQL databases. pymysql does not depend on C language or other external libraries, which makes it very convenient to install and use on different operating systems.

[0058] Execute the monitoring strategy corresponding to the monitoring data, including:

[0059] S2321: Determine whether the load data exceeds a second threshold set on each execution server.

[0060] In this embodiment, since the hardware configurations of each execution server are different, the second threshold value can be set according to the carrying capacity of each execution server. For example, the load average is calculated first. The total number of cameras is 12, and there are 3 execution servers configured with WebRTC services, then the load average is 4. Then, adjustments are made according to the carrying capacity of the execution server. The hardware configuration of execution server A is relatively low, and the second threshold value of execution server A is set to 4; the hardware configuration of execution server B is medium, and the second threshold value of execution server B is set to 6; the hardware configuration of execution server C is relatively high, and the second threshold value of execution server C is set to 7. Then, it is determined whether the load data exceeds the second threshold value set on each execution server.

[0061] S2322: If yes, restart the WebRTC service on the corresponding execution server.

[0062] In this embodiment, when the load data exceeds the second threshold set on the execution server, it means that the network load exceeds the upper limit of the execution server, which will cause the current execution server to be overloaded, while other execution servers are relatively idle, affecting network performance. For example, the number of camera connections on the WebRTC service in the B execution server in the load data is 7, and the second threshold of the B execution server is set to 6, then the WebRTC service on the B execution server is restarted.

[0063] By restarting the WebRTC service on the overloaded execution server, the locked camera can be released, allowing the Nginx service to reallocate resources, quickly restoring low-latency video transmission and improving user experience.

[0064] In an illustrative embodiment, Figure 2 As shown, when the monitoring data is a video stream, the monitoring data is obtained based on the server configuration information, including:

[0065] S2330: Obtain a video stream based on the camera configuration information.

[0066] In an illustrative embodiment, obtaining a video stream based on camera configuration information includes:

[0067] S2330-1: Based on the camera configuration information, determine the execution server to which the camera is connected.

[0068] The WebRTC service will write the connected camera information into the database table. The execution server to which the camera is connected can be queried through the camera ID. For example, for camera A with camera ID a, it can be determined that it is connected to execution server A by querying the database table.

[0069] S2330-2: When it is detected that the execution server stores the video stream sent by the camera in the buffer area, read the video stream in the buffer area.

[0070] In this embodiment, the A execution server receives the video stream sent by camera a through the WebRTC service, and the A execution server stores the video stream in the cache area. When the monitoring server detects that there is a video stream in the A execution server cache area, it reads the video stream in the A execution server cache area.

[0071] In an illustrative implementation, when it is determined that the execution server is connected to a camera and the waiting time for not receiving a video stream is greater than a preset time threshold, a camera failure alarm is sent.

[0072] In this embodiment, after camera a is connected to execution server A, the waiting time is greater than a preset time threshold, for example, the preset time threshold is 5 seconds, and the time camera a is connected to execution server A is greater than 5 seconds and execution server A does not receive the video stream, then it means that camera a has a fault and a camera fault alarm needs to be sent to prompt the operator to handle it.

[0073] Execute the monitoring strategy corresponding to the monitoring data, including:

[0074] S2331: Sampling is performed from the video stream to obtain a sampled image.

[0075] In this embodiment, a counter is first set to control the sampling frequency, for example, a 5-second counter is set, and sampling is performed every 5 seconds; the read video stream is decoded, and the decoding process is to convert the video data into a processable frame sequence; according to the determined sampling frequency, the frames to be sampled are selected from the decoded video frames, for example, the timestamp of each sampling point is calculated, and the frames closest to these timestamps are found from the video frames; the selected frames are extracted as pictures and saved to a local file or database, and the sampled pictures can be saved in picture formats such as JPEG and PNG.

[0076] S2332: Calculate the gradient variance value and grayscale value of the sampled image;

[0077] In this embodiment, the gray value refers to the brightness value of each pixel in the image. In a gray image, this value is usually an integer from 0 (black) to 255 (white). Calculating the gray value of the sampled image refers to calculating the average gray value of the entire image, that is, the average gray value of all pixels. The gradient variance value reflects the degree of change of the pixel value in the image. The gradient is a measure of the rate of change of the pixel value in the image, and the gradient variance is the square of the standard deviation of these gradient values, that is, the variance, which can measure the degree of local change in the image.

[0078] Image processing libraries such as OpenCV or PIL are provided in Python, which can realize the calculation of gradient variance values ​​and grayscale values. For example, first use the OpenCV library to convert the sampled image into a grayscale image and calculate the grayscale value; then use the gradient operator, such as the Sobel operator, to calculate the gradient of the grayscale image to obtain two gradient maps, which correspond to the gradients in the x direction and the y direction respectively; finally, calculate the variance of the two gradient maps to obtain the gradient variance value of the sampled image.

[0079] S2333: Compare the gradient variance value and grayscale value of the two sampled images obtained in a preset period to obtain a contrast value; if the contrast value exceeds a preset threshold, send an automatic cleaning instruction, which is used to control the camera to perform self-cleaning.

[0080] In an exemplary embodiment, the gradient variance value and the grayscale value of two sampled images acquired in a preset period are compared to obtain a contrast value, and if the contrast value exceeds a preset threshold, an automatic cleaning instruction is sent, including:

[0081] S2333-1: Determine a percentage of a difference between a gradient variance value of the first sampling picture and a gradient variance value of the second sampling picture as a first contrast value.

[0082] In this embodiment, the difference between the gradient variance value of the first sampling picture and the gradient variance value of the second sampling picture is first calculated, and the percentage of the difference relative to the gradient variance value of the first sampling picture is used as the first contrast value. For example, if the gradient variance value of the first sampling picture is 100 and the gradient variance value of the second sampling picture is 70, then the first contrast value is 30%.

[0083] S2333-2: Determine the percentage of the difference between the grayscale value of the first sampling picture and the grayscale value of the second sampling picture as the second contrast value.

[0084] In this embodiment, the difference between the grayscale value of the first sampling image and the grayscale value of the second sampling image is first calculated, and the percentage of this difference relative to the grayscale value of the first sampling image is used as the second contrast value. For example, if the grayscale value of the first sampling image is 100 and the grayscale value of the second sampling image is 90, then the second contrast value is 10%.

[0085] S2333-3: If the first comparison value is greater than the third threshold value and the second comparison value is less than the fourth threshold value, an automatic cleaning instruction is sent.

[0086] In this embodiment, if the scene lighting is normal, that is, the light will not change much, and the camera is not dusty, then the first contrast value will be relatively small, that is, the gradient variance value of the sampled image changes very little; and the larger the first contrast value, that is, the larger the gradient variance value of the sampled image changes, it means that there is a lot of dust on the camera, which affects the quality of the captured image. If the scene lighting is abnormal, that is, the light changes greatly, then the second contrast value will be relatively large, that is, the gray value of the sampled image changes greatly. Therefore, to determine whether the camera is affected by dust, it is necessary to exclude the situation where the second contrast value changes greatly. In other words, when the first contrast value is greater than the third threshold value and the second contrast value is less than the fourth threshold value, it means that the change in the image quality captured by the camera is caused by dust, so an automatic cleaning instruction is sent, and the automatic cleaning instruction is sent to the cleaning mechanism of the corresponding camera through the WebAPI service.

[0087] Furthermore, the third threshold and the fourth threshold can be configured based on the resolution of the camera. The resolution of the camera directly determines the clarity and detail of the image captured by the camera. The higher the resolution, the higher the clarity of the image, and the more obvious the change in the gradient variance value of the sampled image. Therefore, the higher the resolution, the smaller the third threshold and the fourth threshold can be set. For example, if the resolution of the camera is 1920x1080, the third threshold is set to 20%, and the fourth threshold is set to 20%; if the resolution of the camera is 800x600, the third threshold is set to 30%, and the fourth threshold is set to 30%.

[0088] S240: Generate a monitoring result report.

[0089] The monitoring result report can be generated by video service performance control results, camera self-cleaning times, load control results, etc., which can summarize and analyze the monitoring results of video service performance control, camera control and system load control within a specified period. By collecting and analyzing relevant data, the stability and efficiency of the system can be evaluated, providing a reference for future optimization and maintenance work.

[0090] like Figure 3 As shown, the second embodiment of the present application provides an automatic monitoring device for fully mechanized mining site, comprising:

[0091] The configuration module 31 is used to obtain server configuration information; the server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information.

[0092] The acquisition module 32 is used to acquire monitoring data based on the server configuration information, where the monitoring data includes: one or more of video service performance data, video stream and load data.

[0093] The execution module 33 is used to execute corresponding monitoring strategies according to different monitoring data; the monitoring strategies include: one or more of video service performance control strategies, camera control strategies and load control strategies.

[0094] In some embodiments of the present application, when the monitoring data obtained is video service performance data, the acquisition module 32 can be used to obtain video service performance data based on video service configuration information; the video service configuration information is the configuration information of the video service on each execution server; the video service includes one or more of the WebAPI service, Nginx service, camera gimbal service and WebRTC service; the video service performance data is the system resource data occupied by the video service when the video service is running on each execution server; the system resource data includes one or more of the CPU usage, memory usage and hard disk I / O usage.

[0095] The execution module 33 may be used to determine whether the system resource data occupied by the video service performance data exceeds a first threshold set on each execution server; if so, restart the video service on the corresponding execution server.

[0096] In some embodiments of the present application, when the monitoring data obtained is network load configuration information, the acquisition module 32 can be used to obtain load data based on the network load configuration information; the network load configuration information includes the total number of cameras and one or more execution servers configured with the WebRTC service; the load data includes the number of camera connections on the WebRTC service in the execution server.

[0097] The execution module 33 can be used to determine whether the load data exceeds the second threshold set on each execution server; if so, restart the WebRTC service on the corresponding execution server.

[0098] In some embodiments of the present application, when the monitoring data is acquired as a video stream, the acquisition module 32 can be used to acquire the video stream based on the camera configuration information; the camera configuration information includes one or more of the camera identification, network address and resolution; the video stream includes image information captured by the camera.

[0099] The execution module 33 can be used to sample from the video stream to obtain a sampled image; calculate the gradient variance value and grayscale value of the sampled image; compare the gradient variance value and grayscale value of two sampled images obtained in a preset period to obtain a contrast value; if the contrast value exceeds a preset threshold, send an automatic cleaning instruction, which is used to control the camera to perform self-cleaning.

[0100] In some embodiments of the present application, the acquisition module 32 can be used to determine the execution server to which the camera is connected based on the camera configuration information; when it is detected that the execution server stores the video stream sent by the camera in the cache area, the video stream in the cache area is read.

[0101] In some embodiments of the present application, the acquisition module 32 can be used to send a camera failure alarm when it is determined that the execution server has not received a video stream for a waiting time greater than a preset time threshold when the server is connected to the camera.

[0102] In some embodiments of the present application, the execution module 33 can be used to determine the percentage of the difference between the gradient variance value of the first sampling image and the gradient variance value of the second sampling image as the first contrast value; determine the percentage of the difference between the grayscale value of the first sampling image and the grayscale value of the second sampling image as the second contrast value; if the first contrast value is greater than the third threshold and the second contrast value is less than the fourth threshold, send an automatic cleaning instruction.

[0103] In some embodiments of the present application, the automatic monitoring device for fully mechanized mining site further includes:

[0104] The generating module 34 is used to generate a monitoring result report.

[0105] In the above text, the automatic monitoring device for comprehensive mining site of the embodiment of the present application is described from the perspective of functional modules in combination with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, each step of an embodiment of a method for automatic monitoring of comprehensive mining site in the embodiment of the present application can be completed by hardware integrated logic circuits and / or software instructions in the processor, and the steps of an automatic monitoring method for comprehensive mining site applied in combination with the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or a combination of hardware and software modules in the decoding processor to execute. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above-mentioned embodiment of the method for automatic monitoring of comprehensive mining site in combination with its hardware.

[0106] like Figure 1 As shown, the third aspect of the present application provides an automatic monitoring system for fully mechanized mining site, including:

[0107] The camera 130 is configured to collect image information.

[0108] The cleaning mechanism 140 is configured to clean the camera.

[0109] The execution server 120 is configured to run a video service; the video service includes one or more of a WebAPI service, an Nginx service, a camera PTZ service, and a WebRTC service.

[0110] The monitoring server 110 is configured to execute an automatic monitoring method for comprehensive mining site provided in the embodiment of the first aspect of the present application.

[0111] The monitoring server 110 can be an industrial computer, a desktop computer, a laptop computer, a tablet computer, a single-chip microcomputer, a PLC (Programmable Logic Controller), a DSP (digital signal processor), an FPGA (Field Programmable Gate Array), an ASIC (Application-specific integrated circuit), or other device with storage and computing functions, but is not limited thereto.

[0112] Since the automatic monitoring system for comprehensive mining site provided in this embodiment has the automatic monitoring method for comprehensive mining site provided by any of the above embodiments, the dual-processor control system has all the beneficial effects of the automatic monitoring method for comprehensive mining site provided by any of the above embodiments, which will not be repeated here.

[0113] It should be noted that the terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0114] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "exemplary embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0115] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in the field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for automatic monitoring of fully mechanized mining sites, applied to a monitoring server, characterized in that: include: Get server configuration information; The server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information; Based on the server configuration information, acquiring monitoring data, the monitoring data including: one or more of video service performance data, video stream and load data; Executing a monitoring strategy corresponding to the monitoring data; the monitoring strategy includes: one or more of a video service performance control strategy, a camera control strategy, and a load control strategy; When the monitoring data obtained is a video stream, the obtaining of the monitoring data based on the server configuration information includes: Based on the camera configuration information, the video stream is obtained; the camera configuration information includes one or more of a camera identifier, a network address and a resolution; the video stream includes image information captured by the camera; The executing the monitoring strategy corresponding to the monitoring data includes: Sampling from the video stream to obtain a sampled picture; Calculating the gradient variance value and grayscale value of the sampled image; Determine a first contrast value as a percentage of a difference between a gradient variance value of the first sampling picture and a gradient variance value of the second sampling picture; Determine a percentage of a difference between a grayscale value of the first sampling picture and a grayscale value of the second sampling picture as a second contrast value; If the first comparison value is greater than a third threshold value, and the second comparison value is less than a fourth threshold value, an automatic cleaning instruction is sent.

2. The automatic monitoring method for fully mechanized mining site according to claim 1, characterized in that: When the monitoring data to be acquired is the video service performance data, acquiring the monitoring data based on the server configuration information includes: Based on the video service configuration information, the video service performance data is obtained; the video service configuration information is the configuration information of the video service on each execution server; the video service includes one or more of the WebAPI service, the Nginx service, the camera PTZ service and the WebRTC service; the video service performance data is the system resource data occupied by the video service when each of the execution servers is running; the system resource data includes one or more of the CPU usage, the memory usage and the hard disk I / O usage; The executing the monitoring strategy corresponding to the monitoring data includes: Determining whether the system resource data occupied by the video service performance data exceeds a first threshold set on each of the execution servers; If yes, restart the video service on the corresponding execution server.

3. The automatic monitoring method for fully mechanized mining site according to claim 1, characterized in that: When the monitoring data to be acquired is the network load configuration information, acquiring the monitoring data based on the server configuration information includes: Based on the network load configuration information, the load data is obtained; the network load configuration information includes the total number of cameras and one or more execution servers configured with the WebRTC service; the load data includes the number of connections of cameras on the WebRTC service in the execution server; The executing the monitoring strategy corresponding to the monitoring data includes: Determine whether the load data exceeds a second threshold set on each of the execution servers; If yes, restart the WebRTC service on the corresponding execution server.

4. The automatic monitoring method for fully mechanized mining site according to claim 1, characterized in that: The acquiring of the video stream based on the camera configuration information includes: Based on the camera configuration information, determining an execution server to which the camera is connected; When it is detected that the execution server stores the video stream sent by the camera in a buffer area, the video stream in the buffer area is read.

5. The automatic monitoring method for fully mechanized mining site according to claim 4, characterized in that: When it is determined that the execution server, in a state of being connected to a camera, has not received the video stream for a waiting time greater than a preset time threshold, a camera failure alarm is sent.

6. The automatic monitoring method for fully mechanized mining site according to claim 1, characterized in that: Also includes: Generate monitoring results report.

7. An automatic monitoring device for fully mechanized mining site, characterized in that: include: Configuration module, used to obtain server configuration information; The server configuration information includes: one or more of video service configuration information, camera configuration information and network load configuration information; An acquisition module, configured to acquire monitoring data based on the server configuration information, wherein the monitoring data includes one or more of video service performance data, video stream and load data; When the monitoring data obtained is a video stream, the obtaining of the monitoring data based on the server configuration information includes: Based on the camera configuration information, the video stream is obtained; the camera configuration information includes one or more of a camera identifier, a network address and a resolution; the video stream includes image information captured by the camera; An execution module, used to execute corresponding monitoring strategies according to different monitoring data; the monitoring strategies include: one or more of video service performance control strategies, camera control strategies and load control strategies; The executing the monitoring strategy corresponding to the monitoring data includes: Sampling from the video stream to obtain a sampled picture; Calculating the gradient variance value and grayscale value of the sampled image; Determine a first contrast value as a percentage of a difference between a gradient variance value of the first sampling picture and a gradient variance value of the second sampling picture; Determine a percentage of a difference between a grayscale value of the first sampling picture and a grayscale value of the second sampling picture as a second contrast value; If the first comparison value is greater than a third threshold value, and the second comparison value is less than a fourth threshold value, an automatic cleaning instruction is sent.

8. An automatic monitoring system for fully mechanized mining site, characterized in that: include: A camera is configured to collect image information; a cleaning mechanism configured to clean the camera; an execution server configured to run a video service; The video service includes one or more of a WebAPI service, an Nginx service, a camera PTZ service, and a WebRTC service; The monitoring server is configured to execute an automatic monitoring method for comprehensive mining site as described in any one of claims 1-6.

Images