Mining area production safety intelligent monitoring method

Abstract

The invention discloses a mining area production safety intelligent monitoring method. The method comprises the following steps: constructing a judgment rule base; obtaining mining area video data and performing data transmission based on a transmission protocol; mining area video data are received and preprocessed, and enhanced video data are obtained; performing target identification and tracking on the enhanced video data to obtain a positioning result and a tracking result of the personnel and the equipment; performing feature extraction on the positioning result and the tracking result of the personnel and the equipment, and constructing a corresponding personnel behavior feature model and an equipment state feature model; matching the personnel behavior feature model with a personnel behavior judgment rule, and performing identification and early warning on the abnormal behavior of the personnel; and matching the equipment state feature model with the equipment state judgment rule, and performing identification and early warning on the abnormal working condition of the equipment. According to the invention, by fusing a plurality of advanced technologies, the accuracy and real-time performance of mining area safety monitoring are effectively improved, and a powerful guarantee is provided for mining area safety production.

Description

Technical Field

[0001] The invention belongs to the technical field of mine production safety, and in particular relates to an intelligent monitoring method for mine production safety. Background Art

[0002] In the process of intelligent monitoring of production safety in mining areas, how to achieve real-time analysis of video data and accurate identification of abnormal situations is a key technical issue. First of all, the amount of video data collected by high-definition digital cameras is huge, and it is necessary to establish an efficient data transmission channel to ensure that the video data can be quickly and completely uploaded to the data analysis unit. Secondly, the video contains complex and changeable scene information, and the judgment of personnel behavior and equipment status requires the use of advanced image recognition algorithms. However, the mining environment is harsh, the lighting conditions are poor, and there may be large noise interference in the video screen, which brings challenges to image recognition. How to accurately locate and track personnel and equipment in a complex environment, and extract key features for analysis, is a technical problem that needs to be overcome.

[0003] In addition, to determine whether personnel behavior and equipment status are abnormal, it is necessary to establish a reasonable rule base and threshold standards. However, the production process in mining areas is diverse and the safety regulations are complicated. How to customize personalized judgment rules according to different types of work and different scenarios, and continuously optimize and improve them in practice, is also a complex technical issue.

[0004] In short, intelligent video surveillance in mining areas involves multiple links such as data collection, transmission, storage, and analysis. How to connect various links, achieve end-to-end technical collaboration, and ultimately safeguard safe production in mining areas is a comprehensive technical issue that urgently needs to be studied in depth and overcome. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method for intelligent monitoring of mine production safety to solve the problems existing in the above prior art.

[0006] To achieve the above object, the present invention provides a method for intelligent monitoring of production safety in a mining area, comprising the following steps:

[0007] Constructing a judgment rule library, wherein the judgment rule library includes personnel behavior judgment rules and equipment status judgment rules;

[0008] Acquire mining area video data and transmit data based on transmission protocol;

[0009] Receiving the mining area video data and performing preprocessing to obtain enhanced video data;

[0010] Performing target recognition and tracking on the enhanced video data to obtain positioning results and tracking results of personnel and equipment;

[0011] Extract features from the positioning results and tracking results of personnel and equipment respectively, and build corresponding personnel behavior feature models and equipment status feature models;

[0012] Matching the personnel behavior feature model with personnel behavior judgment rules to identify and warn of abnormal personnel behavior;

[0013] The equipment status feature model is matched with the equipment status judgment rule to identify and warn of abnormal equipment conditions.

[0014] Optionally, the process of building a judgment rule base includes:

[0015] Obtain the process flow and safety procedures for mine production, obtain relevant data on personnel behavior and equipment status based on the process flow and safety procedures, use machine learning algorithms to analyze the relevant data on personnel behavior and equipment status, and construct personnel behavior judgment rules and equipment status judgment rules respectively.

[0016] Optionally, the transmission protocol adopts an RTP transmission protocol based on a UDP protocol, and a retransmission request and a forward error correction mechanism are added to the RTP transmission protocol.

[0017] Optionally, the process of receiving the mining area video data includes:

[0018] The mining area video data is received by adopting multi-threading technology, and the received mining area video data is cached by adopting a data buffer mechanism; it is determined whether there is packet loss or disorder in the mining area video data, and if so, a retransmission request is triggered until the complete mining area video data is received; the received complete mining area video data is decoded and format converted to obtain the converted mining area video data.

[0019] Optionally, the process of preprocessing the mining area video data to obtain enhanced video data includes:

[0020] The mining area video data is segmented by using an adaptive threshold segmentation algorithm, the segmented mining area video data is subjected to brightness and contrast enhancement processing, and then a denoising algorithm in the wavelet transform domain is used to perform noise reduction processing on the mining area video data after brightness and contrast enhancement, and an adaptive unsharp masking algorithm is used to perform edge enhancement processing on the denoised mining area video data to obtain an enhanced denoising result in the spatial domain;

[0021] Through the video frame adaptive motion estimation and compensation technology, the continuous video frames of the mining area video data are subjected to temporal denoising. The motion vector of the pixel is estimated by using the temporal redundancy between adjacent frames, and motion compensation is performed in combination with adaptive weights to obtain enhanced denoising results in the temporal domain.

[0022] Combining the enhanced denoising results in spatial and temporal domains, the final enhanced video data is generated through a weighted fusion strategy.

[0023] Optionally, the process of performing target recognition and tracking on the enhanced video data to obtain positioning results and tracking results of personnel and equipment includes:

[0024] A target detection algorithm is used to identify the enhanced video data to obtain the location areas of personnel and equipment; a semantic segmentation algorithm is used to perform pixel segmentation on the detected personnel area and equipment area to obtain the target contour information of personnel and equipment; a tracking algorithm is used to track the target contour information of personnel and equipment in continuous video frames to obtain the spatial position change information of personnel and equipment respectively.

[0025] Optionally, the process of matching the personnel behavior feature model with personnel behavior judgment rules to identify and warn of abnormal personnel behavior includes:

[0026] The posture features and motion features of the human target are extracted, and the features are fused to construct a human behavior feature model; the constructed human behavior feature model is compared with the human behavior judgment rules in the judgment rule library to calculate the similarity; if the similarity is lower than the preset threshold, it is judged as abnormal behavior and an early warning is triggered; if the similarity is higher than the threshold, it is judged as normal behavior and no early warning is triggered.

[0027] Optionally, the process of matching the equipment state feature model with the equipment state judgment rule to identify and warn of abnormal equipment conditions includes:

[0028] Extract the key information of the equipment operating parameters and status indicator lights of the equipment target, and build an equipment status feature model based on the equipment operating parameters and status indicator light key information; match the equipment status feature model with the equipment status judgment rules in the judgment rule library. If the equipment status feature successfully matches the abnormal status pattern, it is judged that the equipment is in an abnormal working condition; if it successfully matches the normal status pattern, it is judged that the equipment is in a normal working condition.

[0029] The present invention also provides an electronic device, comprising: a memory and a processor; the memory is used to store a program; the processor is used to execute the program to implement each step of the intelligent monitoring method for mine production safety.

[0030] The present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each step of the intelligent monitoring method for production safety in a mining area is implemented.

[0031] Compared with the prior art, the present invention has the following advantages and technical effects:

[0032] The present invention discloses a mine safety monitoring method. Aiming at the real-time monitoring problem of personnel behavior and equipment status in the harsh environment of the mine, the method first establishes a personalized judgment rule library, personnel behavior judgment rules and equipment status judgment rules; then obtains high-quality enhanced video data through efficient video transmission and preprocessing technology; then uses a deep learning algorithm to accurately locate and track personnel and equipment in the video, and extracts key features; finally, the extracted features are compared with the patterns in the rule library to achieve intelligent identification and early warning of abnormal personnel behavior and abnormal equipment conditions. By integrating multiple advanced technologies, the present invention effectively improves the accuracy and real-time performance of mine safety monitoring, and provides a strong guarantee for safe production in mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0034] Figure 1 It is a schematic diagram of the process flow of the intelligent monitoring method for mine production safety according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Embodiment 1

[0038] like Figure 1 As shown, this embodiment provides a method for intelligent monitoring of mine production safety, including the following steps:

[0039] Constructing a judgment rule library, wherein the judgment rule library includes personnel behavior judgment rules and equipment status judgment rules;

[0040] Acquire mining area video data and transmit data based on transmission protocol;

[0041] Receiving the mining area video data and performing preprocessing to obtain enhanced video data;

[0042] Performing target recognition and tracking on the enhanced video data to obtain positioning results and tracking results of personnel and equipment;

[0043] Extract features from the positioning results and tracking results of personnel and equipment respectively, and build corresponding personnel behavior feature models and equipment status feature models;

[0044] Matching the personnel behavior feature model with personnel behavior judgment rules to identify and warn of abnormal personnel behavior;

[0045] The equipment status feature model is matched with the equipment status judgment rule to identify and warn of abnormal equipment conditions.

[0046] As a specific embodiment, the process of constructing a judgment rule library including personnel behavior judgment rules and equipment status judgment rules includes:

[0047] According to the production process and safety regulations of the mining area, combined with the judgment rules of personnel behavior and equipment status, a personalized judgment rule library and threshold standards are established. The judgment rule library contains the feature extraction and matching rules required for video data analysis, providing a basis for subsequent video analysis.

[0048] As an implementable method, obtain the production process and safety regulations of the mining area. For example, the production process of a mining area includes rock drilling, blasting, loading, transportation and other links. The safety regulations stipulate the operating specifications, safety distances, allowable errors, etc. of each link. Obtain relevant data on personnel behavior and equipment status, such as collecting personnel's position, posture, operation actions, and equipment's operating status, temperature, pressure and other data through cameras, sensors and other equipment. These data are the basis for subsequent analysis. Through data analysis and machine learning algorithms, establish a personalized judgment rule library and threshold standards. For example, by analyzing historical data, it can be found that under normal circumstances, the vibration frequency of the rock drill fluctuates within a certain range. If the vibration frequency exceeds the threshold, there may be equipment failure or operating error. Store these rules and thresholds in the rule library for subsequent abnormal event judgment. The threshold standard can be adjusted according to the specific production environment and equipment conditions of the mining area. For example, the normal vibration frequency range of different models of rock drills may be different.

[0049] As a specific embodiment, an efficient video data compression algorithm and transmission protocol are used to build a stable and reliable video data transmission channel to quickly and completely transmit video data from a high-definition digital camera, and the receiving process includes:

[0050] The original video data collected by a high-definition camera is obtained, and the original video data is compressed by using an H.265 encoding algorithm to obtain compressed video data; wherein, the H.265 encoding algorithm is provided with quantization parameters and a bit rate control strategy; the compressed video data is transmitted over the network by using an RTP transmission protocol based on a UDP protocol; wherein, a retransmission request and a forward error correction mechanism are added to the RTP transmission protocol; during the network transmission process, the network status is monitored in real time, and the sending rate of the video data is dynamically adjusted according to the network status; at the receiving end, the video data is received by using a multi-threading technology, and the received video data is cached by using a data buffering mechanism; it is determined whether the received video data has packet loss or disorder; if so, a retransmission request is triggered until a complete video frame is received; the received complete video frame is decoded and format converted to obtain a converted video frame.

[0051] As an implementable method, the raw video data collected by the high-definition camera, such as a video stream with a resolution of 1920x1080, a frame rate of 30fps, and a color gamut of BT.2020. In order to reduce storage and transmission pressure, the raw video data needs to be compressed. The H.265 encoding algorithm can save about 30% to 50% of the bit rate compared to the H.264 encoding algorithm, and the file size is smaller at the same image quality. For example, setting the quantization parameter QP to 30, the bit rate control strategy to constant bit rate CBR, and the target bit rate to 4Mbps can effectively control the quality and size of the compressed video. The compressed video data needs to be transmitted over the network. The RTP transmission protocol based on the UDP protocol can achieve real-time transmission with lower latency, which is more suitable for mining area monitoring scenarios. In order to improve the reliability of transmission, retransmission requests and forward error correction mechanisms are added to the RTP protocol. For example, it can be set to send an FEC redundancy packet every 10 RTP data packets to repair lost data packets. At the same time, the NACK mechanism can be enabled at the sending end to request the receiving end to retransmit the lost data packets. During network transmission, it is necessary to monitor network conditions in real time, such as bandwidth, delay, packet loss rate, etc. Dynamically adjusting the video data transmission rate according to network conditions can avoid network congestion and ensure transmission stability. For example, when the network bandwidth is reduced, the video transmission bit rate can be reduced, or the frame rate can be reduced. When the network condition returns to normal, it will be restored to the original transmission rate. At the receiving end, multi-threading technology can improve reception efficiency. For example, multiple threads can be opened to receive different RTP data packets at the same time. The data buffering mechanism can smooth network jitter and ensure the smoothness of video playback. For example, setting the buffer size to 1 second of video data can effectively cope with short-term network fluctuations. The receiving end needs to determine whether the received video data has packet loss or disorder. For example, by checking the sequence number of the RTP data packet, it can be determined whether there is packet loss or disorder. If there is packet loss, a retransmission request is triggered until a complete video frame is received. For example, the receiving end can send a NACK message to the sending end to request retransmission of the lost data packet. The received complete video frame needs to be decoded and format converted. For example, H.265 encoded video frames are decoded into YUV format and then converted into RGB format for subsequent image processing and analysis.

[0052] As a specific embodiment, in view of the noise interference introduced by the harsh environment and poor light conditions in the mining area, the received video data is preprocessed using an adaptive image enhancement and denoising algorithm to improve the image quality and obtain enhanced video data after preprocessing.

[0053] As an implementable method, an adaptive threshold segmentation algorithm is used for preprocessing the video data, and the adaptive separation of the image background and the target is achieved by dynamically adjusting the segmentation threshold; the brightness and contrast of the segmented video image data are adaptively enhanced, and the pixel grayscale value is dynamically adjusted according to the brightness distribution characteristics of the local area of ​​the image; an adaptive denoising algorithm in the wavelet transform domain is used to reduce the noise of the video image after brightness enhancement, and the image signal is converted to the wavelet domain through wavelet decomposition, and the denoising threshold is adaptively set according to the statistical characteristics of each sub-band coefficient; the denoised video image is edge enhanced, and an adaptive unsharp masking algorithm is used to dynamically adjust the enhancement coefficient according to the local gradient characteristics of the image; the continuous video frames are subjected to time domain denoising through inter-frame adaptive motion estimation and compensation technology, and the motion vector of the pixel point is accurately estimated by using the time redundancy between adjacent frames, and motion compensation is performed in combination with adaptive weights; the adaptive enhancement denoising results of the spatial domain and the temporal domain are integrated, and the final preprocessed video data is generated through a weighted fusion strategy.

[0054] As an implementable method, the adaptive threshold segmentation algorithm can separate the image background and target. Traditional image segmentation methods usually use global thresholds, but in the mining monitoring scene, due to the drastic changes in lighting conditions, the global threshold is often difficult to adapt to all situations. For example, when the sun is strong at noon, the global threshold may cause the target area to be mistakenly segmented as the background; and when the light is dim at night, the global threshold may cause the background noise to be mistakenly segmented as the target. The adaptive threshold segmentation algorithm can dynamically adjust the segmentation threshold according to the brightness distribution characteristics of the local area of ​​the image. For example, the image can be divided into several small blocks, and the optimal threshold for each small block is calculated separately. In this way, even in scenes with drastic changes in lighting conditions, a good segmentation effect can be achieved, and targets such as ores, vehicles, and personnel can be effectively distinguished from the background. Brightness and contrast adaptive enhancement processing can improve the visual quality of the image. In the mining monitoring scene, due to changes in weather and lighting conditions, the brightness and contrast of the collected video images are often uneven. For example, in the backlit or shadowed area, the image brightness is low and the detail information is seriously lost; while in the strong light area, the image brightness is too high and overexposure is prone to occur. Adaptive enhancement processing can dynamically adjust the pixel grayscale value according to the brightness distribution characteristics of the local area of ​​the image. For example, for areas with low brightness, its brightness can be increased to restore detail information; for areas with high brightness, its brightness can be reduced to avoid overexposure. In this way, the visual quality of the image can be effectively improved, making the target more clearly visible. The adaptive denoising algorithm in the wavelet transform domain can effectively remove image noise. In the mining area monitoring video, due to the equipment's own noise and environmental interference, there is often a lot of noise in the image, which will affect the subsequent target detection and recognition. Wavelet transform is a time-frequency analysis method that can decompose the image signal into sub-bands of different scales and directions. Noise is usually concentrated in the high-frequency sub-band, while the main information of the image is concentrated in the low-frequency sub-band. The adaptive denoising algorithm can adaptively set the denoising threshold according to the statistical characteristics of each sub-band coefficient. For example, the denoising threshold can be dynamically adjusted according to the noise level of the high-frequency sub-band to remove noise while retaining the image's detail information as much as possible. For example, setting the threshold to 5 times the local variance can effectively remove noise. Edge enhancement processing can make the image edge clearer. In the mining area monitoring scene, the edge information of the target is crucial for the recognition and positioning of the target. The adaptive unsharp masking algorithm can dynamically adjust the enhancement coefficient according to the local gradient characteristics of the image. For example, in the area with large edge gradient of the image, the enhancement coefficient can be increased to make the edge clearer; while in the flat area of ​​the image, the enhancement coefficient can be reduced to avoid noise amplification. For example, setting the gradient threshold to 50, the enhancement coefficient of the area greater than the threshold is 2, and the enhancement coefficient of the area less than the threshold is 1, can achieve adaptive edge enhancement. Inter-frame adaptive motion estimation and compensation technology can reduce the flicker and jitter of video images.The monitoring videos of mining areas are usually shot continuously, and there is a lot of temporal redundant information between adjacent frames. Motion estimation technology can use the temporal redundancy between adjacent frames to accurately estimate the motion vector of the pixel. For example, the block matching algorithm can be used to find the displacement of the corresponding pixel blocks between adjacent frames. Motion compensation technology can use the motion vector to predict the current frame, thereby reducing the noise in the time domain. For example, when performing motion compensation, setting the search range to 15 frames before and after and the block size to 8x8 pixels can effectively remove the flicker and jitter of the video image. Combining the adaptive enhancement and denoising results of the spatial and temporal domains, the final pre-processed video data can be obtained. Spatial domain processing is mainly aimed at the enhancement and denoising of single-frame images, such as the adaptive threshold segmentation, brightness enhancement, denoising and edge enhancement mentioned above. Temporal domain processing mainly uses the temporal correlation between adjacent frames for denoising, such as inter-frame adaptive motion estimation and compensation. Since spatial domain and temporal domain processing have their own advantages and disadvantages, the results of the two need to be fused. For example, in areas with complex image textures, the spatial domain processing results can be given a greater weight; while in areas with relatively smooth images, the temporal domain processing results can be given a greater weight. For example, the texture complexity threshold is set to 8, the spatial weight of the area greater than the threshold is 7, and the temporal weight is 3; the spatial weight of the area less than the threshold is 3, and the temporal weight is 7. The spatial and temporal processing results can be adaptively fused according to the image content. The preprocessed enhanced video data is transmitted to the data analysis unit to improve the accuracy and efficiency of subsequent target detection, recognition and tracking. After preprocessing, the quality of the video image has been significantly improved, the target is clearer, and the noise is less, which lays a good foundation for subsequent data analysis. For example, in the preprocessed video, targets such as people, vehicles and equipment are easier to be recognized by the detection algorithm, thereby improving the accuracy of recognition. At the same time, due to the reduction of noise, the probability of false detection is also reduced.

[0055] Furthermore, the process of using the adaptive denoising algorithm in the wavelet transform domain to perform denoising on the video image after brightness enhancement includes:

[0056] The video image after brightness enhancement is obtained, and wavelet transform is performed on it to convert the image from the spatial domain to the wavelet transform domain. In the wavelet transform domain, thresholds at different scales and directions are adaptively determined according to the statistical characteristics of the wavelet coefficients. The wavelet coefficients are compared with the thresholds. If the absolute value of the wavelet coefficients is less than the threshold, they are set to zero to achieve threshold processing of the wavelet coefficients. Through the threshold processing of the wavelet coefficients, the noise in the image is effectively suppressed, while the important details and texture information of the image are retained. The inverse wavelet transform is performed on the wavelet coefficients after threshold processing, and the image is converted from the wavelet transform domain back to the spatial domain to obtain the denoised video image. The quality of the denoised video image is evaluated, and objective evaluation indicators such as peak signal-to-noise ratio are used to determine whether the denoising effect meets the preset quality requirements. If the quality of the denoised video image meets the expected requirements, the denoised video image is output; otherwise, the parameter settings of the wavelet transform are adjusted, and steps 1 to 6 are repeated until the quality requirements are met.

[0057] Wavelet transform is a mathematical tool that decomposes image signals into different frequency subbands. It is particularly useful in image processing because it can separate noise (usually high-frequency components) from important image details (usually low-frequency components) in the image. Wavelet transform of video images can convert images from the spatial domain to the wavelet transform domain for processing on different frequency subbands. For example, a 640x480 pixel video frame can be decomposed into three levels of wavelet to obtain low-frequency subband LL3 and high-frequency subbands LH3, HL3, HH3, and lower resolution subbands LL2, LH2, HL2, HH2, etc. In the wavelet transform domain, noise usually appears as random fluctuations of wavelet coefficients. By analyzing the statistical properties of wavelet coefficients, such as variance or standard deviation, thresholds at different scales and directions can be adaptively determined. Different scales correspond to image details of different resolutions, and different directions correspond to edges in different directions in the image. For example, for areas with rich details, the threshold can be set higher to retain more detail information; while for smooth areas, the threshold can be set lower to better remove noise. Assuming that the standard deviation of the wavelet coefficients in the high-frequency subband LH3 in the horizontal direction is 5, the threshold can be set to 10, that is, 2 times the standard deviation. Thresholding of wavelet coefficients refers to comparing the wavelet coefficients with the threshold and setting the wavelet coefficients whose absolute values ​​are less than the threshold to zero. This method can effectively suppress noise in the image, because noise usually corresponds to smaller coefficients. At the same time, important details and texture information of the image are retained, because this information usually corresponds to larger coefficients. For example, if a wavelet coefficient is 3 and the threshold is 5, the coefficient will be set to zero; if a wavelet coefficient is -8 and the threshold is 5, the coefficient will be retained. Inverse wavelet transform is the process of converting the coefficients in the wavelet transform domain back to the spatial domain image. By performing inverse wavelet transform on the wavelet coefficients after thresholding, a denoised video image can be obtained. For example, by performing inverse wavelet transform on the thresholded subband coefficients such as LL3, LH3, HL3, and HH3, a video frame with the original resolution can be reconstructed. Peak signal-to-noise ratio (PSNR) is a commonly used image quality evaluation indicator. It measures the degree of image distortion by comparing the mean square error between the original image and the processed image. The higher the PSNR value, the better the image quality. For example, if the PSNR of the denoised video image is 30dB, it indicates that the denoising effect is good. If the quality of the denoised video image does not meet the expected requirements, you need to adjust the parameter settings of the wavelet transform and repeat the steps of wavelet transform, thresholding, and inverse wavelet transform until the quality requirements are met. For example, you can try to change parameters such as the wavelet basis function, the number of decomposition layers, or the threshold calculation method. Assuming that the Haar wavelet basis was originally used, you can try to use the Daubechies wavelet basis to see if you can get a better denoising effect. If the number of decomposition layers is too small, it may not be possible to effectively separate the noise and image details. You can try to increase the number of decomposition layers.Through the above steps, the adaptive denoising algorithm in the wavelet transform domain can be used to reduce the noise of the video image, improve the image quality, and lay the foundation for subsequent image analysis and processing.

[0058] As a specific embodiment, the enhanced video data is comprehensively applied with a deep learning-based target detection and semantic segmentation algorithm to accurately locate and track people and equipment in the image. By analyzing the spatial position changes of people and equipment in continuous image frames, the process of determining whether the behavior of people and the status of equipment are abnormal includes:

[0059] Perform image enhancement processing on the acquired raw video data to improve the video quality and clarity, and provide high-quality input data for subsequent target detection and semantic segmentation. Use deep learning-based target detection algorithms, such as YOLO or FasterR-CNN, to process the enhanced video frames, identify the personnel and equipment targets in the image, and obtain their position coordinate information. Use semantic segmentation algorithms, such as FCN or DeepLab, to perform pixel-level segmentation on the personnel and equipment areas obtained by target detection to obtain more detailed target contours and position information. Track personnel and equipment targets in continuous video frames through tracking algorithms, such as Kalman filtering or particle filtering, to obtain their spatial position change information in continuous frames. According to the personnel position information obtained by target detection and tracking, analyze their motion trajectory and position changes in continuous frames to determine whether the personnel's behavior is abnormal, such as whether there is abnormal behavior such as running or falling. According to the position change and status information of the device, determine whether the device is abnormal, such as whether the device is moving, tilting, and other abnormal states. The abnormal judgment results of personnel behavior and equipment status are integrated to generate the final abnormal alarm information and notify relevant personnel to handle it to ensure the safe operation of personnel and equipment.

[0060] As an implementable method, in harsh environments such as mining areas, video data is often affected by insufficient light and dust interference, resulting in poor image quality. In order to improve video quality and clarity, image enhancement processing technology is first used, such as dynamic range compression technology to enhance dark details in the image, so that the personnel and equipment in the video are more clearly visible. For example, in a dark mine video, after enhancement processing, the color and logo of the worker's helmet that were originally unclear become clear, which is helpful for subsequent safety monitoring. Next, the enhanced video frame is processed using a deep learning target detection algorithm, such as YOLO. The YOLO algorithm can quickly identify the location of personnel and equipment in the video frame and output the bounding box of the target. For example, when processing a video of a loading area, YOLO can accurately mark the location of each loading machine and operator, which is crucial for monitoring the safety of loading operations. In order to obtain more detailed target information, semantic segmentation algorithms, such as DeepLab, are then used to perform pixel-level segmentation on the personnel and equipment areas detected by YOLO. This segmentation helps obtain, for example, the precise outline of the tool in the worker's hand, so that in safety monitoring, it can better judge whether the worker is using the tool correctly. In addition, by tracking the target in the continuous video frames through tracking algorithms, such as Kalman filtering, the spatial position change information of the target in the continuous frames can be obtained. In a moving conveyor belt scene, the tracking algorithm can monitor whether the materials on the equipment are moving in the normal position, or whether there are materials falling due to abnormal conditions. Finally, based on the information obtained from target detection and tracking, the behavior of personnel and the status of equipment are analyzed. For example, if a worker is detected to move frequently from one location to another in a short period of time, it may indicate that the worker is urgently dealing with certain problems or there are safety hazards. Similarly, if the position of the equipment changes suddenly, it may indicate that the equipment has tilted or shifted, and needs to be checked and handled immediately. Through this series of image processing and analysis technologies, the safety monitoring capabilities in harsh environments such as mining areas can be effectively improved, potential safety issues can be discovered and handled in a timely manner, and the safe operation of personnel and equipment can be guaranteed.

[0061] Furthermore, based on the position information of the person obtained by target detection and tracking, the movement trajectory and position change of the person in the continuous frames are analyzed to determine whether the person's behavior is abnormal, such as whether there is abnormal behavior such as running or falling. The process includes:

[0062] Through the pre-established target detection model, the human target is detected in the input video frame and its position coordinate information is obtained. For each detected human target, the target tracking algorithm such as Kalman filter is used to track the target in continuous video frames to obtain its motion trajectory. For the tracked human motion trajectory, the trajectory jitter caused by detection and tracking errors is removed through trajectory smoothing to obtain the smoothed human motion trajectory. According to the displacement change of the human motion trajectory between continuous frames, the instantaneous speed and acceleration of the person are calculated and compared with the preset running speed and acceleration thresholds. If the threshold is exceeded, it is judged as running behavior. By analyzing the height change of the human motion trajectory, it is judged whether the person has a sudden drop in height from standing to lying down. If so, it is further combined with the speed change to judge whether the person has fallen. Classification models such as support vector machines are used to classify the characteristics of the human motion trajectory and position change, and comprehensively identify abnormal behaviors such as running and falling. According to the results of video analysis, it is judged whether the person has abnormal behavior. If an abnormality is detected, a timely warning is issued to realize intelligent human behavior monitoring.

[0063] As an implementable method, using the YOLO or FasterR-CNN algorithm of deep learning can effectively identify the human target in the input video frame and obtain its position coordinate information. Next, using the target tracking algorithm, such as Kalman filtering, the identified human target can be tracked in continuous video frames. This algorithm can handle the movement of the target in the picture through the process of prediction and updating, and can continue to track even if the target is partially blocked or temporarily disappears. In order to improve the accuracy of tracking, it is usually necessary to smooth the tracked movement trajectory of the person, which can be achieved by algorithms such as moving average or Gaussian filtering. Smoothing can effectively remove the trajectory jitter caused by detection and tracking errors, making the movement trajectory more consistent with the actual movement situation. By analyzing the movement trajectory of the person, the instantaneous speed and acceleration can be calculated. This step usually involves calculating the time derivative of the position data, which can be used to determine whether the movement state of the person is abnormal. In addition, by analyzing the height change of the person's movement trajectory, it can be determined whether the person has a sudden drop in height from standing to lying, which is usually an important signal of falling. Combined with the speed change, the system can more accurately determine the fall event. Finally, machine learning models, such as support vector machines, can be used to classify the movement trajectory and position change characteristics of people, thereby comprehensively identifying abnormal behaviors such as running and falling. This classification model can be trained and verified in multiple scenarios to improve its accuracy and robustness.

[0064] Furthermore, the key features of the posture and action of the person in the video image are extracted from the target detection and tracking results, a person behavior feature model is established, and the model is compared with the behavior pattern in the personalized judgment rule library to achieve real-time recognition and early warning of abnormal behavior of the person, including:

[0065] Perform target detection on the acquired video images, identify the human targets in the video, and obtain the location, size and other information of the human targets; track the human targets according to the target detection results, and obtain the motion trajectory of the human targets in continuous video frames; extract the posture features and motion features of the tracked human targets, the posture features include the coordinates of the key points of the human skeleton, and the motion features include the movement speed and acceleration of the key points; fuse the extracted human posture features and motion features to construct a human behavior feature model, which can characterize the behavior patterns of the personnel; establish a personalized judgment rule library in advance, which contains various normal behavior patterns and abnormal behavior patterns; compare the constructed human behavior feature model with the behavior patterns in the personalized judgment rule library, and calculate the similarity; if the similarity is lower than the preset threshold, it is judged as abnormal behavior and triggers an early warning; if the similarity is higher than the threshold, it is judged as normal behavior and no early warning is triggered.

[0066] As an implementable method, target detection is performed on the acquired video images to identify the human targets in the video and obtain the position, size and other information of the human targets. This information provides basic data for subsequent tracking and behavior analysis. According to the target detection results, the human targets are tracked to obtain the motion trajectory of the human targets in continuous video frames. For the tracked human targets, their posture features and motion features are extracted. Posture features include the coordinates of key points of the human skeleton, such as using the OpenPose algorithm to extract the coordinates of key points such as the customer's head, shoulders, and hips. Assume that in a certain frame, the customer's head coordinates are (120, 220), the left shoulder coordinates are (110, 230), and the right shoulder coordinates are (130, 230). Motion features include the movement speed and acceleration of key points, such as calculating the displacement difference and time difference of the head in continuous frames to obtain the speed and acceleration. Assuming that the displacement of the head between two frames is 10 pixels and the time is 1 second, the speed is 100 pixels / second. The extracted human posture features and motion features are fused to construct a human behavior feature model. For example, the coordinates of the head, shoulders, and hips and their speed and acceleration information are integrated into a multidimensional vector, such as (120, 220, 110, 230, 130, 230, 100, 5, 3), where the first six values ​​are the key point coordinates and the last three values ​​are the speed and acceleration. This vector can characterize the customer's behavior pattern in the current frame. A personalized judgment rule library is pre-established, which contains various normal behavior patterns and abnormal behavior patterns. For example, the rule library can contain a "normal walking" mode, whose feature vector range is (100-150, 200-250, ..., 50-150, 1-5, 1-3), and a "falling" mode, whose feature vector range is (50-100, 100-150, ..., 200-300, 1-2, 5-1). These rules are established based on a large number of statistical data of normal and abnormal behaviors. The constructed personnel behavior feature model is compared with the behavior pattern in the personalized judgment rule library to calculate the similarity. For example, the cosine similarity is used to calculate the similarity between the current customer's behavior feature vector and each mode in the rule base. Assume that the calculated similarity with the "normal walking" mode is 9, and the similarity with the "falling" mode is 2. If the similarity is lower than the preset threshold, it is judged as abnormal behavior and an early warning is triggered. For example, if the threshold is set to 7, the similarity between the current customer's behavior feature vector and the "falling" mode is only 2, which is lower than the threshold. The system determines that the customer may fall, immediately triggers an early warning, and notifies the security personnel to go to handle it. If the similarity is higher than the threshold, it is judged as normal behavior and no early warning is triggered. Target detection and tracking ensure continuous monitoring of personnel, the extraction of posture and motion features provides a detailed basis for behavior analysis, the construction of feature models and the comparison of rule bases realize intelligent judgment of behavior, and ultimately the early warning mechanism ensures the safe operation of personnel and equipment.

[0067] As a specific embodiment, the operation parameters and key information of the status indicator of the equipment in the video image are obtained from the target detection and tracking results, the equipment status features are extracted, and the equipment status features are matched with the equipment status patterns in the personalized judgment rule library by using a rule-based expert system algorithm. The process of judging whether the equipment is in an abnormal working condition includes:

[0068] The target detection algorithm is used to analyze the video image, identify the equipment target in the image, and obtain the location and area information of the equipment. According to the location and area information of the equipment target, the tracking algorithm is used to track the equipment target to obtain the motion trajectory of the equipment target in the video image. For the tracked equipment target area, the key information of the equipment operation parameters and status indicator lights in the area are extracted to obtain the status characteristics of the equipment. The extracted equipment status characteristic data is input into the rule-based expert system, and the equipment status characteristics are analyzed and judged by the expert system algorithm. The expert system matches the equipment status characteristics with the equipment status patterns in the judgment rule library based on the pre-established personalized judgment rule library. If the equipment status characteristics match the abnormal status pattern in the judgment rule library, it is judged that the equipment is in an abnormal working condition; if it matches the normal status pattern, it is judged that the equipment is in a normal working condition. According to the judgment result of whether the equipment is in an abnormal working condition, an equipment operation status report is generated and sent to relevant personnel so that corresponding maintenance or processing measures can be taken in time.

[0069] As an implementable method, the target detection algorithm is used to analyze the video image, identify the device target in the image, and obtain the location and area information of the device. According to the location and area information of the device target, the tracking algorithm is used to track the device target to obtain the motion trajectory of the device target in the video image. For the tracked device target area, the key information of the device operating parameters and status indicator lights in the area are extracted to obtain the status characteristics of the device. The current operating status of the device can be judged through these status indicator light information. The extracted device status feature data is input into the rule-based expert system, and the device status features are analyzed and judged by the expert system algorithm. The expert system matches the device status features with the device status patterns in the judgment rule library based on the pre-established personalized judgment rule library. For example, the rule library contains a variety of device status modes, such as "normal mode", "warning mode", "serious fault mode", etc. Each mode corresponds to a series of feature combinations, such as "normal mode" corresponds to "green indicator light, 1Hz flashing frequency", and "serious fault mode" corresponds to "red indicator light, 2Hz flashing frequency". By matching these modes, the system can accurately judge the current status of the device. If the equipment status feature matches the abnormal status pattern in the judgment rule base, the equipment is judged to be in abnormal working condition; if it matches the normal status pattern, the equipment is judged to be in normal working condition. Based on the judgment result of whether the equipment is in abnormal working condition, the equipment operation status report is generated and sent to relevant personnel so that corresponding maintenance or treatment measures can be taken in time. For example, the system generates a detailed report containing equipment ID, current status, abnormal description, and recommended treatment measures, and sends it to the maintenance engineer via email or SMS. The report content may include: "Equipment ID: 001, current status: serious failure, abnormal description: LED indicator red flashing frequency 2Hz, recommended treatment measures: stop the machine immediately for inspection". In this way, maintenance personnel can respond quickly and reduce production losses caused by equipment failure. In this way, a series of steps such as target detection, tracking, state feature extraction, expert system analysis and report generation form a complete equipment status monitoring and early warning system. Each step is linked together to ensure real-time monitoring of equipment status and timely handling of abnormal situations, thereby improving the stability and safety of the production line. This system can not only reduce downtime caused by equipment failure, but also extend the service life of equipment, reduce maintenance costs, and improve overall production efficiency. Through accurate status judgment and timely early warning mechanisms, enterprises can achieve refined management of equipment and improve the intelligence level of the production process.

[0070] As an additional implementation method, this embodiment also includes real-time monitoring of the slope stability of the mining area. Intelligent monitoring of the slope stability of the mining area can achieve early identification, early warning and disaster prevention of slope deformation, thereby ensuring the safety and sustainable development of the mining area. The specific process includes:

[0071] Utilize a variety of sensors and monitoring technologies, such as GNSS, InSAR, drones, 3D laser scanners, etc., to achieve all-round monitoring of slopes. Through multi-source data monitoring, support the convergence and management of multi-source heterogeneous data such as satellite remote sensing images, drone oblique photography, ground 3D laser scanning, GNSS receivers, crack meters, inclinometers, etc. Integrate multi-source displacement monitoring data such as InSAR, ground-based radar, GNSS, etc., and quantitatively analyze slope deformation in multiple dimensions. Input the collected slope monitoring data into the big data analysis model to conduct real-time evaluation of the slope status, issue a warning signal when an abnormal situation is detected, and automatically notify relevant management personnel.

[0072] As a specific embodiment, the core of slope monitoring lies in multi-mode collaborative observation of sky-air-ground multi-platforms, including time coordination, space coordination, parameter coordination and intelligent analysis. Time coordination refers to the use of different monitoring methods at different stages to achieve continuous monitoring covering the entire process of landslide incubation and development. Spatial coordination combines point-type and surface monitoring, ground-ground and underground monitoring, and sky-air-ground multi-platforms to achieve full coverage, multi-level and multi-precision monitoring of the entire mine pit. Multi-parameter coordination realizes the coordinated monitoring of surface displacement, geometric deformation, surface temperature, rock and soil moisture, rainfall, etc., and supports spatiotemporal correlation and intelligent analysis.

[0073] Embodiment 2

[0074] The present invention also provides an electronic device, comprising: a memory and a processor; the memory is used to store a program; the processor is used to execute the program to implement each step of the intelligent monitoring method for mine production safety.

[0075] Embodiment 3

[0076] The present invention also provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, each step of the intelligent monitoring method for production safety in a mining area is implemented.

[0077] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for intelligent monitoring of production safety in mining areas, characterized in that: The following steps are involved: Constructing a judgment rule library, wherein the judgment rule library includes personnel behavior judgment rules and equipment status judgment rules; Acquire mining area video data and transmit data based on transmission protocol; Receiving the mining area video data and performing preprocessing to obtain enhanced video data; Performing target recognition and tracking on the enhanced video data to obtain positioning results and tracking results of personnel and equipment; Extract features from the positioning results and tracking results of personnel and equipment respectively, and build corresponding personnel behavior feature models and equipment status feature models; Matching the personnel behavior feature model with personnel behavior judgment rules to identify and warn of abnormal personnel behavior; The equipment status feature model is matched with the equipment status judgment rule to identify and warn of abnormal equipment conditions.

2. The method according to claim 1, characterized in that The process of building a judgment rule base includes: Obtain the process flow and safety procedures for mine production, obtain relevant data on personnel behavior and equipment status based on the process flow and safety procedures, use machine learning algorithms to analyze the relevant data on personnel behavior and equipment status, and construct personnel behavior judgment rules and equipment status judgment rules respectively.

3. The method according to claim 1, characterized in that The transmission protocol adopts the RTP transmission protocol based on the UDP protocol, and the RTP transmission protocol is added with a retransmission request and a forward error correction mechanism.

4. The method according to claim 3, characterized in that The process of receiving the mining area video data includes: The mining area video data is received by adopting multi-threading technology, and the received mining area video data is cached by adopting a data buffer mechanism; it is determined whether there is packet loss or disorder in the mining area video data, and if so, a retransmission request is triggered until the complete mining area video data is received; the received complete mining area video data is decoded and format converted to obtain the converted mining area video data.

5. The method according to claim 1, characterized in that The process of preprocessing the mining area video data to obtain enhanced video data includes: The mining area video data is segmented by using an adaptive threshold segmentation algorithm, the segmented mining area video data is subjected to brightness and contrast enhancement processing, and then a denoising algorithm in the wavelet transform domain is used to perform noise reduction processing on the mining area video data after brightness and contrast enhancement, and an adaptive unsharp masking algorithm is used to perform edge enhancement processing on the denoised mining area video data to obtain an enhanced denoising result in the spatial domain; Through the video frame adaptive motion estimation and compensation technology, the continuous video frames of the mining area video data are subjected to temporal denoising. The motion vector of the pixel is estimated by using the temporal redundancy between adjacent frames, and motion compensation is performed in combination with adaptive weights to obtain enhanced denoising results in the temporal domain. Combining the enhanced denoising results in spatial and temporal domains, the final enhanced video data is generated through a weighted fusion strategy.

6. The method according to claim 1, characterized in that The process of performing target recognition and tracking on the enhanced video data to obtain positioning results and tracking results of personnel and equipment includes: A target detection algorithm is used to identify the enhanced video data to obtain the location areas of personnel and equipment; a semantic segmentation algorithm is used to perform pixel segmentation on the detected personnel area and equipment area to obtain the target contour information of personnel and equipment; a tracking algorithm is used to track the target contour information of personnel and equipment in continuous video frames to obtain the spatial position change information of personnel and equipment respectively.

7. The method according to claim 6, characterized in that The process of matching the personnel behavior feature model with the personnel behavior judgment rules and identifying and warning abnormal personnel behavior includes: The posture features and motion features of the human target are extracted, and the features are fused to construct a human behavior feature model; the constructed human behavior feature model is compared with the human behavior judgment rules in the judgment rule library to calculate the similarity; if the similarity is lower than the preset threshold, it is judged as abnormal behavior and an early warning is triggered; if the similarity is higher than the threshold, it is judged as normal behavior and no early warning is triggered.

8. The method according to claim 6, characterized in that The process of matching the equipment status feature model with the equipment status judgment rule to identify and warn of abnormal equipment conditions includes: Extract the key information of the equipment operating parameters and status indicator lights of the equipment target, and build an equipment status feature model based on the equipment operating parameters and status indicator light key information; match the equipment status feature model with the equipment status judgment rules in the judgment rule library. If the equipment status feature successfully matches the abnormal status pattern, it is judged that the equipment is in an abnormal working condition; if it successfully matches the normal status pattern, it is judged that the equipment is in a normal working condition.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the method for intelligent monitoring of mine production safety as described in any one of claims 1-8.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, each step of the method for intelligent monitoring of mine production safety as described in any one of claims 1 to 8 is implemented.

Images