Coal mine production safety violation identification method based on video analysis

Abstract

The present invention belongs to the field of coal mine video surveillance and relates to a method for identifying coal mine production safety violations based on video analysis. The method collects on-site surveillance videos of target coal mine production zones, identifies and analyzes the operating support structures and cover support structures of the target coal mine production zones in the on-site surveillance videos, and on the one hand evaluates the abnormal stability violation risks of the operating support structures and cover support structures of the target coal mine production zones, thereby judging whether there are support abnormal stability violations in the target coal mine production zones, and if so, an alarm is issued; on the other hand, by tracking the activities of operating personnel located above the operating support structure of the target coal mine production zone in the on-site surveillance video, it is judged whether there are abnormal operation violations by personnel in the target coal mine production zones, and if so, an alarm is issued, thereby achieving effective identification and analysis of the most indispensable support structure-related violations in coal mine production scenarios.

Description

Technical Field

[0001] The invention belongs to the field of coal mine video monitoring and relates to a method for identifying illegal behaviors in coal mine production safety based on video analysis. Background Art

[0002] With the continued importance of coal resources in the energy structure and increasingly stringent safety production requirements, the manual inspection safety monitoring method with limited monitoring range and lack of real-time performance is difficult to effectively respond to the safety challenges brought by the complex coal mine operating environment. The use of video monitoring to improve the safety management level of coal mine operations has gradually become an urgent need for the development of the industry.

[0003] In the prior art, there are also some related solutions involving the identification of coal mine production safety violations through video surveillance. For example, the invention patent application for a coal mine tunnel monitoring method, device and computer storage medium with Chinese Patent Number CN2020101218377 obtains a tunnel monitoring image sequence in the coal mine tunnel, identifies and classifies the targets therein, and generates a first alarm instruction if the target includes personnel; if the target is a mine car, the mine car is tracked and counted, thereby realizing the monitoring of violations in coal mine tunnel transportation operations.

[0004] Another invention patent application with Chinese patent number CN2021104524415 is for a method for warning of violations and dangerous behaviors of workers in a coal mine comprehensive mining face. The method first uses a camera to obtain images of workers in a coal mine comprehensive mining face environment, performs image preprocessing, and trains a worker border detection model. It determines whether dangerous behavior occurs by judging whether the worker's border enters or approaches the scraper ROI area, and whether the worker's border and the cutting head ROI area are close to each other. Abnormal behaviors of workers falling, climbing over the scraper, and approaching the cutting head are identified and warned in a timely manner.

[0005] Although the above scheme can effectively use video surveillance analysis to identify safety violations in coal mines, it still faces the limitations commonly found in existing technologies. Specifically, existing technologies mainly focus on operating personnel and operating equipment for safety violations in coal mines, and ignore the effective identification and analysis of violations of support structures, which are the most essential in coal mine production scenarios.

[0006] On the one hand, the support structure is the basic infrastructure to ensure the stability and safety of the working space. It not only serves the roof control, effectively blocks the collapse of the roof and prevents major safety hazards such as coal wall collapse, and realizes reliable coverage and reinforcement of the underground mining face, but also serves as a necessary working surface for the operators to perform their tasks, ensuring the safety and convenience of the operators when performing various operations. However, the existing technology lacks the analysis of abnormal stability violation risks and the identification of violation behaviors for the support structure in the coal mine production area, resulting in the inability to assess the bearing state and potential failure risks of the support structure in real time and accurately, resulting in blind spots in the monitoring of key safety indicators such as roof pressure changes, deformation or damage of the support structure, and shaking of the support working platform.

[0007] On the other hand, existing technologies for analyzing violations of workers performing tasks above support structures often focus on checking whether the workers are wearing basic personal protective equipment, such as safety helmets, safety belts, etc., and over-focus on the physical wearing of personal protective equipment, and fail to pay sufficient attention to the status and efficiency of mining lamps as key lighting equipment. In addition, they ignore whether there is a risk of crossing the boundary in the movement trajectory of the workers and whether there is a safety risk in the working posture, resulting in the inability to achieve comprehensive and accurate identification and effective prevention and control of violations of workers. Summary of the invention

[0008] In view of this, in order to solve the problems raised in the above background technology, a method for identifying coal mine production safety violations based on video analysis is proposed.

[0009] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides a method for identifying coal mine production safety violations based on video analysis, including: S1. Real-time collection of on-site monitoring videos of target coal mine production areas, and identification of the operating support structures and cover support structures of the target coal mine production areas in the on-site monitoring videos.

[0010] S2. Analyze the relevant features of the operating support structure and cover support structure of the target coal mine production zone in the on-site monitoring video, evaluate the abnormal stability violation risk of the operating support structure and cover support structure of the target coal mine production zone, and judge whether there is any abnormal stability violation of support in the target coal mine production zone. If it is judged to exist, execute S4.

[0011] S3. Track the activities of the operating personnel located above the operating support structure of the target coal mine production area in the on-site monitoring video, and determine whether there are any abnormal operating violations by personnel in the target coal mine production area. If so, execute S5.

[0012] S4. Alarm for abnormal violations of support stability in the production area of ​​the target coal mine.

[0013] S5. Alarm any abnormal or illegal operation by personnel in the production area of ​​the target coal mine.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention combines the annotation information of the target coal mine production zone design drawings to conduct a detailed analysis of the edge contour data in the video frame image, thereby realizing high-speed and high-precision identification of the working support structure and cover support structure of the target coal mine production zone in the on-site monitoring video, providing favorable support for the subsequent identification of violations in the target coal mine production zone.

[0015] (2) The present invention captures the subtle shaking phenomenon of the working support structure of the target coal mine production zone in the on-site monitoring video, and scientifically evaluates the abnormal stability violation risk of the working support structure of the target coal mine production zone, which not only significantly improves the coal mine production safety management level, but also provides a data basis for realizing comprehensive intelligent monitoring and preventive maintenance strategies for the coal mine operating environment.

[0016] (3) The present invention captures the deformation degree of each closed-form structural surface of the cover support structure in the target coal mine production zone in the on-site monitoring video, and combines the two dimensions of overall deformation coefficient and continuous deformation coefficient to greatly enhance the ability to accurately assess the abnormal stability violation risk of the cover support structure in the target coal mine production zone, help promote the intelligent process of coal mine safety management, and provide strong scientific and technological support for ensuring the safety of coal mine operations.

[0017] (4) The present invention achieves comprehensive and detailed safety management by comprehensively evaluating the three major violation risks of equipment lighting, operating distance and operating posture of the operators located above the operating support structure of the target coal mine production area in the on-site monitoring video. It not only strengthens the safety awareness of the operators and encourages the operating behavior to be more in line with the safety operating procedures, but also greatly enhances the safety assurance capability of coal mine production and promotes the modernization and intelligent development of production safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 The present invention is a flowchart of the steps for implementing the method. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] See also Figure 1 As shown, the present invention provides a method for identifying coal mine production safety violations based on video analysis, including: S1. real-time acquisition of on-site monitoring videos of target coal mine production zones, and identification of operating support structures and cover support structures of target coal mine production zones in the on-site monitoring videos.

[0022] Exemplarily, the above-mentioned target coal mine production zone may specifically refer to the excavation working face, tunnel intersections and turns, coal mining working face, etc., which can simultaneously include the working support structure that serves as the necessary working face for the workers to perform their tasks and the cover support structure that serves the roof control.

[0023] Specifically, the identification of the working support structure and the cover support structure of the target coal mine production zone in the on-site monitoring video includes: dividing the on-site monitoring video into a number of video frame images in sequence according to the unit monitoring time, and marking them as video frame images after image preprocessing.

[0024] The edge contour data in each video frame is detected, and based on the working support structure design contour and the cover support structure design contour marked on the target coal mine production partition design drawings stored in the WEB cloud, each key feature point in the working support structure design contour is mapped to the video frame edge contour, so as to retrieve the best edge pixel points matching each key feature point in the video frame edge contour, and the best edge pixel points matching each key feature point in the video frame edge contour are connected through the connected domain search to form each continuous edge contour segment, and the preset morphological operation is used to fill the defects of each continuous edge contour segment, so as to identify the working support structure contour of the target coal mine production partition in each video frame.

[0025] It should be noted that the key feature points within the design outline of the above-mentioned working support structure include but are not limited to the foundation point, support point, turning point, anchor point, etc. of the support structure, among which the foundation point of the support structure refers to the point where the support structure is connected to the ground or the underlying support structure, the support point of the support structure refers to the connection point where the support structure uses the internal support material, such as the connection point of the steel structure support and the reinforced concrete support, the support turning point refers to the position where the shape of the support structure changes, such as the junction of a curved segment and a straight segment, and the anchor point refers to the fixed position of an anchor rod or anchor cable.

[0026] Similarly, the outline of the cover support structure of the target coal mine production area in each video frame is identified.

[0027] The video frames are integrated in chronological order to realize the identification of the working support structure and cover support structure of the target coal mine production zone in the on-site monitoring video.

[0028] Specifically, the retrieval of the best edge pixel points matching each key feature point within the edge contour of the video frame includes: constructing a neighborhood corresponding to each key feature point within the design contour of the working support structure according to a predefined principle, calculating the gradient direction and gradient amplitude of each pixel point in the neighborhood, dividing a number of gradient direction intervals according to preset direction angles, allocating corresponding gradient direction intervals to each pixel point in the neighborhood corresponding to the key feature point, weightedly accumulating the gradient direction and gradient amplitude of each pixel point within the same gradient direction interval to generate a directional histogram, and then concatenating the directional histograms within the neighborhood into a vector after normalization, using the vector as a feature descriptor of the key feature point, thereby obtaining the feature descriptors corresponding to each key feature point in the design contour template of the working support structure.

[0029] It should be noted that the above-mentioned predefined principle refers to taking a circular area constructed with the key feature point as the center and a preset value as the radius as the neighborhood corresponding to the key feature point.

[0030] It should also be noted that the gradient direction and gradient amplitude of each pixel point in the above neighborhood are calculated by applying the convolution kernel analysis of the Sobel operator in the horizontal and vertical directions. The specific calculation formula is an existing mature technology and will not be repeated here.

[0031] The position coordinates of each key feature point in the design outline of the working support structure marked on the design drawing of the target coal mine production partition are obtained, and a potential matching label with the key feature point is assigned to each pixel point in the edge outline of the video frame image whose distance with a key feature point is less than or equal to a preset distance value. The feature descriptor of each pixel point in the edge outline of the video frame image with a potential matching label with the key feature point is obtained, and the similarity is calculated with the feature descriptor of the key feature point, and the pixel point corresponding to the maximum similarity is selected as the best edge pixel point matching the key feature point in the edge outline of the video frame image, thereby retrieving the best edge pixel point matching each key feature point in the edge outline of the video frame image.

[0032] It should be noted that the above-mentioned feature descriptor is a sequence vector of preset dimension, and the feature descriptor of each pixel point in the edge contour of the video frame image having a potential matching label with the key feature point, and its similarity with the feature descriptor of the key feature point is obtained by using the cosine similarity calculation formula.

[0033] The embodiment of the present invention combines the annotation information of the design drawings of the target coal mine production partitions to conduct a detailed analysis of the edge contour data in the video frame image, so as to realize high-speed and high-precision identification of the working support structure and the cover support structure of the target coal mine production partition in the on-site monitoring video, thereby providing favorable support for the subsequent identification of violations in the target coal mine production partitions.

[0034] S2. Analyze the relevant features of the operating support structure and cover support structure of the target coal mine production zone in the on-site monitoring video, evaluate the abnormal stability violation risk of the operating support structure and cover support structure of the target coal mine production zone, and judge whether there is any abnormal stability violation of support in the target coal mine production zone. If it is judged to exist, execute S4.

[0035] Specifically, the method of evaluating the abnormal stability violation risk of the working support structure of the target coal mine production zone includes: converting each video frame into a grayscale image and performing horizontal projection and vertical projection respectively, and obtaining the horizontal grayscale projection curve and vertical grayscale projection curve of the working support structure of the target coal mine production zone in each video frame.

[0036] It should be noted that the process of obtaining the horizontal grayscale projection curve and the vertical grayscale projection curve of the working support structure of the target coal mine production zone in each of the above-mentioned video frames is as follows: the grayscale cumulative value of each column of pixels in the working support structure area of ​​the target coal mine production zone in each video frame after horizontal projection is obtained, and substituted into a rectangular coordinate system with the column number as the horizontal axis and the pixel grayscale value as the vertical axis, and the horizontal grayscale projection curve of the working support structure of the target coal mine production zone in each video frame is drawn. It should be noted that the horizontal direction in the horizontal grayscale projection curve refers to the representation of the projection curve, that is, the curve is placed horizontally on the image, rather than the direction of the projected pixel column itself.

[0037] The grayscale cumulative value of each row of pixels in the working support structure area of ​​the target coal mine production zone in each video frame after vertical projection is obtained, and substituted into a rectangular coordinate system with the row number as the horizontal axis and the pixel grayscale value as the vertical axis to draw the vertical grayscale projection curve of the working support structure of the target coal mine production zone in each video frame.

[0038] Obtain the monitoring period corresponding to the on-site monitoring video and capture the monitoring time point corresponding to each video frame. By comparing the offset of the horizontal and vertical grayscale projection curves of the working support structure of the target coal mine production zone with the previous video frame, obtain the motion vector of the working support structure of the target coal mine production zone at each monitoring time point. If the motion vector at a certain monitoring time point is greater than the preset reasonable motion vector, it is judged that there is a jitter phenomenon in the working support structure of the target coal mine production zone at the monitoring time point. The motion vector deviation ratio is used as the jitter amplitude at the monitoring time point, and then the continuous jitter duration of the working support structure of the target coal mine production zone in the monitoring period corresponding to the on-site monitoring video is obtained. , cumulative number of jitters And the single maximum jitter amplitude .

[0039] It should be noted that the offsets of the horizontal and vertical grayscale projection curves of the working support structure of the target coal mine production zone between the above-mentioned video frames and their previous video frames are obtained by subtracting the horizontal grayscale projection curve of the working support structure of the target coal mine production zone in each video frame from the corresponding horizontal grayscale projection curve of the previous video frame and performing absolute value conversion to obtain an absolute deviation sequence of the horizontal grayscale projection curves of the working support structure of the target coal mine production zone between each video frame and its previous video frame, and selecting the maximum element value in the absolute deviation sequence as the offset to obtain the offset of the horizontal grayscale projection curve of the working support structure of the target coal mine production zone between each video frame and its previous video frame.

[0040] Similarly, the offset of the vertical grayscale projection curve of each video frame and its previous video frame in the working support structure of the target coal mine production zone is obtained.

[0041] It should also be noted that the process of obtaining the motion vector of the working support structure of the above-mentioned target coal mine production zone at each monitoring time point is: by performing square root operations on the offset of the horizontal and vertical grayscale projection curves of the working support structure of the target coal mine production zone between each video frame and its previous video frame, and using the calculation result as the motion vector of the working support structure of the target coal mine production zone in each video frame, and corresponding each video frame to each monitoring time point, so as to obtain the motion vector of the working support structure of the target coal mine production zone at each monitoring time point.

[0042] The above-mentioned motion vector deviation ratio refers to the difference between the motion vector at the monitoring time point when the working support structure of the target coal mine production area vibrates and the preset reasonable motion vector, and the ratio of the preset reasonable motion vector.

[0043] By formula The abnormal stability violation risk assessment index of the operating support structure of the target coal mine production zone is obtained, among which The duration of the monitoring period corresponding to the on-site monitoring video. is the number of video frames corresponding to the on-site monitoring video, is the preset reference jitter amplitude, is a natural constant.

[0044] The embodiment of the present invention captures the subtle shaking phenomenon of the working support structure of the target coal mine production zone in the on-site monitoring video, and scientifically evaluates the abnormal stability violation risk of the working support structure of the target coal mine production zone, which not only significantly improves the level of coal mine safety production management, but also provides a data basis for realizing comprehensive intelligent monitoring and preventive maintenance strategies for the coal mine operating environment.

[0045] Specifically, the abnormal stability violation risk of the cover support structure of the target coal mine production zone is evaluated, including: arbitrarily selecting a preset number of pixel points in the cover support structure of the target coal mine production zone in the first video frame of the on-site monitoring video for marking, recorded as marked pixel points, and randomly connecting and combining the marked pixel points to obtain closed morphological structural surfaces.

[0046] In the subsequent video frames of the on-site monitoring video, each marked pixel point in the cover support structure of the target coal mine production area is continuously located, and each closed morphological structural surface is obtained according to the same connection combination method.

[0047] According to the deformation degree of each closed morphological structural surface in each subsequent video frame of the on-site monitoring video and the corresponding closed morphological structural surface in the first video frame, the overall deformation coefficient and continuous deformation coefficient of the cover support structure of the target coal mine production zone in the on-site monitoring video are calculated respectively, and the product of the two is used as the abnormal stability violation risk assessment index of the cover support structure of the target coal mine production zone.

[0048] It should be noted that the process of obtaining the deformation degree of each closed morphological structure surface of each subsequent video frame image of the above-mentioned on-site monitoring video and the closed morphological structure surface corresponding to the first video frame image is: obtaining the coordinate deviation value of each marked pixel point corresponding to each closed morphological structure surface of each subsequent video frame image of the on-site monitoring video and the marked pixel point corresponding to the closed morphological structure surface of the first video frame image, adding them up and comparing them with the preset coordinate offset value, to obtain the point offset degree of each closed morphological structure surface of each subsequent video frame image of the on-site monitoring video and the closed morphological structure surface corresponding to the first video frame image.

[0049] The non-overlapping lengths of the corresponding line segments of each closed morphological structure surface of each subsequent video frame of the on-site monitoring video and the corresponding line segments of the closed morphological structure of the first video frame are obtained, and the non-overlapping lengths are accumulated and compared with the total line segment length of the closed morphological structure corresponding to the first video frame to obtain the degree of line deviation between the corresponding closed morphological structure surface of each subsequent video frame of the on-site monitoring video and the corresponding closed morphological structure surface of the first video frame.

[0050] Obtain the deviation area between each closed morphological structure surface of each subsequent video frame of the on-site monitoring video and the closed morphological structure corresponding to the first video frame, and compare it with the area of ​​the closed morphological structure corresponding to the first video frame to obtain the degree of surface offset between each closed morphological structure surface of each subsequent video frame of the on-site monitoring video and the closed morphological structure surface corresponding to the first video frame.

[0051] The degree of point, line and surface deviation is multiplied and accumulated with the corresponding preset weight proportions to obtain the degree of deformation of each closed morphological structure surface of each subsequent video frame of the on-site monitoring video and the closed morphological structure surface corresponding to the first video frame.

[0052] It should also be noted that the overall deformation coefficient of the cover support structure of the target coal mine production area in the above-mentioned on-site monitoring video refers to the cumulative value of the deformation degree of each closed-form structural surface of the end video frame of the on-site monitoring video and the corresponding closed-form structural surface of the first video frame.

[0053] The calculation process of the continuous deformation coefficient of the cover support structure of the target coal mine production zone in the above-mentioned on-site monitoring video is as follows: the deformation degree of each closed morphological structure surface of each subsequent video frame of the on-site monitoring video and the corresponding closed morphological structure surface of the first video frame is recorded as ,in It is the number of each subsequent video frame of the on-site monitoring video. , is the number of each closed morphological structure surface, , according to the formula The continuous deformation coefficient of the cover support structure of the target coal mine production zone in the on-site monitoring video is obtained, where For the follow-up of the on-site surveillance video Video frame The deformation degree of the closed morphological structure surface and the closed morphological structure surface corresponding to the first video frame, It is the number of subsequent video frames of the on-site monitoring video.

[0054] The embodiment of the present invention captures the deformation degree of each closed-form structural surface of the cover support structure in the target coal mine production zone in the on-site monitoring video, and combines the two dimensions of overall deformation coefficient and continuous deformation coefficient to greatly enhance the ability to accurately assess the abnormal stability violation risk of the cover support structure in the target coal mine production zone, help promote the intelligent process of coal mine safety management, and provide strong scientific and technological support for ensuring the safety of coal mine operations.

[0055] Specifically, the determination of whether there is any support abnormal stability violation in the target coal mine production zone includes: if the abnormal stability violation risk assessment index of the operating support structure or the cover support structure of the target coal mine production zone is greater than the preset support structure abnormal stability violation risk assessment index tolerance threshold, then it is determined that there is a support abnormal stability violation in the target coal mine production zone; otherwise, it is determined that there is no support abnormal stability violation in the target coal mine production zone.

[0056] S3. Track the activities of the operating personnel located above the operating support structure of the target coal mine production area in the on-site monitoring video, and determine whether there are any abnormal operating violations by personnel in the target coal mine production area. If so, execute S5.

[0057] Specifically, the determination of whether there is any abnormal operation violation by personnel in the target coal mine production zone includes: evaluating the equipment lighting violation risk, operation distance violation risk and operation posture violation risk of the operating personnel located above the operating support structure of the target coal mine production zone in the on-site monitoring video, and recording the corresponding evaluation indicators and their preset tolerance thresholds as , and .

[0058] If satisfied If the condition is met, it is judged that there is no abnormal operation violation by personnel in the production area of ​​the target coal mine; otherwise, it is judged that there is abnormal operation violation by personnel in the production area of ​​the target coal mine.

[0059] Specifically, the The analysis process includes: determining the position and irradiation direction of the mining lamp of the operator located above the operating support structure of the target coal mine production area in each video frame of the on-site monitoring video; outlining the mining lamp irradiation area of ​​the operator in each video frame of the on-site monitoring video according to the scope of the mining lamp irradiation area stored in the WEB cloud; taking the average pixel grayscale value of the mining lamp irradiation area as the reference light brightness; and calculating the average light brightness of the mining lamp irradiation area of ​​the operator in the on-site monitoring video. , according to the formula Assess the risk of equipment lighting violations for workers located above the support structure of the target coal mine production area in the on-site monitoring video, It is the standard brightness of light for safe operation in the preset coal mine production area.

[0060] It should be noted that the process of determining the position and illumination direction of the miner's lamp of the operator located above the operating support structure of the production zone of the target coal mine in each video frame in the above-mentioned on-site monitoring video is as follows: based on the miner's lamp image features stored in the established database, the target detection technology is used to identify and outline the miner's lamp contour area of ​​the operator located above the operating support structure of the production zone of the target coal mine in each video frame in the on-site monitoring video, the coordinates of the center point of the miner's lamp contour area are used as the miner's lamp position, the line connecting the two endpoints of the miner's lamp contour area through the center point position is used as the miner's lamp illumination line, and the endpoint with a relatively large pixel grayscale value is used as the illumination direction end, thereby obtaining the miner's lamp illumination direction.

[0061] It should also be noted that the reason why the risk of equipment lighting violations can be used as a basis for judging whether there are abnormal operation violations by personnel in the production area of ​​the target coal mine is that before entering the underground mine or the work area with insufficient light, the operator must ensure that the mining lamp he wears fully complies with the safety standards, that is, it can provide sufficient brightness to ensure clear vision and safe operation. If during the operation, it is found or inspected that the light intensity of the mining lamp is insufficient to meet the visual needs, and the operator fails to take timely measures such as replacing the backup light source or reporting the problem to obtain the necessary lighting remedy, this behavior will be regarded as a violation of the safety operating procedures and recorded as an abnormal violation of improper use of personal protective equipment. Therefore, it can be used as a basis for judging whether there are abnormal operation violations by personnel in the production area of ​​the target coal mine.

[0062] Specifically, the The analysis process includes: tracking the activity position of the operating personnel located above the operating support structure of the target coal mine production zone in each video frame in the on-site monitoring video and calculating the relative distance between the operating personnel and the boundary of the operating support structure of the target coal mine production zone, comparing the relative distance with the preset operation boundary safety distance, obtaining the proportion of the operating personnel's out-of-bounds distance in each video frame in the on-site monitoring video, and accumulating them to evaluate the operating distance violation risk of the operating personnel located above the operating support structure of the target coal mine production zone in the on-site monitoring video.

[0063] It should be noted that the process of obtaining the proportion of the operating personnel's out-of-bounds distance in each video frame in the above-mentioned on-site monitoring video is: if the relative distance is less than the preset operating boundary safety distance, the absolute difference between the preset operating boundary safety distance and the relative distance is obtained, and the absolute difference and the preset operating boundary safety distance are used as the proportion of the out-of-bounds distance; conversely, if the relative distance is greater than or equal to the preset operating boundary safety distance, the proportion of the out-of-bounds distance is set to 0, thereby obtaining the proportion of the operating personnel's out-of-bounds distance in each video frame in the on-site monitoring video.

[0064] Specifically, the The analysis process includes: identifying the posture of the workers located above the operating support structure of the target coal mine production zone in each video frame in the on-site monitoring video, matching them with the posture sets of dangerous working postures of coal mine workers stored in the WEB cloud, and selecting the maximum matching degree to evaluate the risk of violation of the operating posture of the workers located above the operating support structure of the target coal mine production zone in the on-site monitoring video.

[0065] It should be noted that the posture and morphology recognition process of the workers located above the working support structure of the target coal mine production area in each video frame in the above-mentioned on-site monitoring video is as follows: based on the image feature set of key parts of the human body stored in the established database, including but not limited to features such as hands, head and feet, target detection technology is used to identify the key parts of the workers located above the working support structure of the target coal mine production area in each video frame in the on-site monitoring video, and the posture and morphology of the workers are outlined.

[0066] It should also be noted that the above-mentioned posture morphology matching process is to place the posture morphology of the workers located above the working support structure of the target coal mine production area in each video frame in the on-site monitoring video on the second layer, and place each posture morphology in the dangerous working posture morphology set of the coal mine workers on the first layer, overlap and compare the first and second layers, and use the proportion of the length of the posture overlap curve as the posture morphology matching degree.

[0067] The embodiment of the present invention achieves comprehensive and detailed safety management by comprehensively evaluating the three major violation risks of equipment lighting, operating distance and operating posture of operators located above the operating support structure of the target coal mine production area in the on-site monitoring video. It not only strengthens the safety awareness of operators and encourages operating behaviors to be more in line with safe operating procedures, but also greatly enhances the safety assurance capabilities of coal mine production and promotes the modernization and intelligent development of production safety management.

[0068] S4. Alarm for abnormal violations of support stability in the production area of ​​the target coal mine.

[0069] S5. Alarm any abnormal or illegal operation by personnel in the production area of ​​the target coal mine.

[0070] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. Method for identifying illegal behaviors in coal mine production safety based on video analysis, It is characterized in that The method includes: S1. Collect the on-site monitoring video of the target coal mine production zone in real time, and identify the operating support structure and cover support structure of the target coal mine production zone in the on-site monitoring video; S2. Analyze the relevant features of the working support structure and the cover support structure of the target coal mine production zone in the on-site monitoring video, evaluate the abnormal stability violation risk of the working support structure and the cover support structure of the target coal mine production zone, and judge whether there is any support abnormal stability violation in the target coal mine production zone. If it is judged that there is, execute S4; S3. Track the activities of the operators located above the operating support structure of the target coal mine production zone in the on-site monitoring video, and determine whether there are any abnormal operation violations by the operators in the target coal mine production zone. If so, execute S5; S4. Alarm for abnormal and illegal support stability in the production area of ​​the target coal mine; S5. Alarm the abnormal and illegal operation of personnel in the production area of ​​the target coal mine; The identifying of the working support structure and the cover support structure of the target coal mine production zone in the on-site monitoring video includes: dividing the on-site monitoring video into a plurality of video frame images in sequence according to the unit monitoring time, and marking them into respective video frame images after image preprocessing; Detect edge contour data in each video frame, and map each key feature point in the design contour of the working support structure to the edge contour of the video frame based on the design contour of the target coal mine production partition design drawings and the cover support structure design contour stored in the WEB cloud, so as to retrieve the best edge pixel points matching each key feature point in the edge contour of the video frame, connect the best edge pixel points matching each key feature point in the edge contour of the video frame through connected domain search, so as to form each continuous edge contour line segment, and use preset morphological operations to fill each continuous edge contour line segment defect, thereby identifying the working support structure contour of the target coal mine production partition in each video frame; Similarly, the outline of the cover support structure of the target coal mine production zone in each video frame is identified; Integrate the video frames in time sequence to identify the working support structure and cover support structure of the target coal mine production zone in the on-site monitoring video; The method of assessing the abnormal stability violation risk of the working support structure of the target coal mine production zone includes: converting each video frame into a grayscale image and performing horizontal projection and vertical projection respectively, and obtaining a horizontal grayscale projection curve and a vertical grayscale projection curve of the working support structure of the target coal mine production zone in each video frame; Obtain the monitoring period corresponding to the on-site monitoring video and capture the monitoring time point corresponding to each video frame. By comparing the offset of the horizontal and vertical grayscale projection curves of the working support structure of the target coal mine production zone with the previous video frame, obtain the motion vector of the working support structure of the target coal mine production zone at each monitoring time point. If the motion vector at a certain monitoring time point is greater than the preset reasonable motion vector, it is judged that there is a jitter phenomenon in the working support structure of the target coal mine production zone at the monitoring time point. The motion vector deviation ratio is used as the jitter amplitude at the monitoring time point, and then the continuous jitter duration of the working support structure of the target coal mine production zone in the monitoring period corresponding to the on-site monitoring video is obtained. , cumulative number of jitters And the single maximum jitter amplitude ; By formula The abnormal stability violation risk assessment index of the operating support structure of the target coal mine production zone is obtained, among which The duration of the monitoring period corresponding to the on-site monitoring video. is the number of video frames corresponding to the on-site monitoring video, is the preset reference jitter amplitude, is a natural constant; The method of assessing the abnormal stability violation risk of the cover support structure of the target coal mine production zone includes: arbitrarily selecting a preset number of pixel points in the cover support structure of the target coal mine production zone of the first video frame of the on-site monitoring video for marking, recording them as each marked pixel point, and randomly connecting and combining each marked pixel point to obtain each closed morphological structure surface; In the subsequent video frames of the on-site monitoring video, each annotated pixel point in the cover support structure of the target coal mine production zone is continuously located, and each closed morphological structural surface is obtained according to the same connection combination method; According to the deformation degree of each closed morphological structural surface in each subsequent video frame of the on-site monitoring video and the corresponding closed morphological structural surface in the first video frame, the overall deformation coefficient and the continuous deformation coefficient of the cover support structure of the target coal mine production zone in the on-site monitoring video are calculated respectively, and the product of the two is used as the abnormal stability violation risk assessment index of the cover support structure of the target coal mine production zone; The determination of whether there is an abnormal stability violation in support in the target coal mine production zone includes: if the abnormal stability violation risk assessment index of the operating support structure or the cover support structure of the target coal mine production zone is greater than the preset support structure abnormal stability violation risk assessment index tolerance threshold, then it is determined that there is an abnormal stability violation in support in the target coal mine production zone; otherwise, it is determined that there is no abnormal stability violation in support in the target coal mine production zone.

2. According to claim 1, the method for identifying illegal behaviors in coal mine production safety based on video analysis, Features: The method of retrieving the best edge pixel point matching each key feature point in the edge contour of the video frame includes: constructing a neighborhood corresponding to each key feature point in the design contour of the working support structure according to a predefined principle, calculating the gradient direction and gradient amplitude of each pixel point in the neighborhood, dividing a number of gradient direction intervals according to a preset direction angle, allocating a corresponding gradient direction interval to each pixel point in the neighborhood corresponding to the key feature point, performing weighted accumulation of the gradient direction and gradient amplitude of each pixel point in the same gradient direction interval to generate a direction histogram, and then concatenating the direction histograms in the neighborhood into a vector after normalization processing, using the vector as a feature descriptor of the key feature point, thereby obtaining the feature descriptor corresponding to each key feature point in the design contour template of the working support structure; The position coordinates of each key feature point in the design outline of the working support structure marked on the design drawing of the target coal mine production partition are obtained, and a potential matching label with the key feature point is assigned to each pixel point in the edge outline of the video frame image whose distance with a key feature point is less than or equal to a preset distance value. The feature descriptor of each pixel point in the edge outline of the video frame image with a potential matching label with the key feature point is obtained, and the similarity is calculated with the feature descriptor of the key feature point, and the pixel point corresponding to the maximum similarity is selected as the best edge pixel point matching the key feature point in the edge outline of the video frame image, thereby retrieving the best edge pixel point matching each key feature point in the edge outline of the video frame image.

3. The method for identifying illegal behaviors in coal mine production safety based on video analysis according to claim 1, Features: The determination of whether there is abnormal operation violation by personnel in the target coal mine production zone includes: evaluating the equipment lighting violation risk, operation distance violation risk and operation posture violation risk of the operating personnel located above the operating support structure of the target coal mine production zone in the on-site monitoring video, and recording the corresponding evaluation indicators and their preset tolerance thresholds as , and ; If satisfied If the condition is met, it is judged that there is no abnormal operation violation by personnel in the production area of ​​the target coal mine; otherwise, it is judged that there is abnormal operation violation by personnel in the production area of ​​the target coal mine.

4. According to claim 3, the method for identifying illegal behaviors in coal mine production safety based on video analysis, Features: Said The analysis process includes: determining the position and irradiation direction of the mining lamp of the operator located above the operating support structure of the target coal mine production area in each video frame of the on-site monitoring video; outlining the mining lamp irradiation area of ​​the operator in each video frame of the on-site monitoring video according to the scope of the mining lamp irradiation area stored in the WEB cloud; taking the average pixel grayscale value of the mining lamp irradiation area as the reference light brightness; and calculating the average light brightness of the mining lamp irradiation area of ​​the operator in the on-site monitoring video. , according to the formula Assess the risk of equipment lighting violations for workers located above the support structure of the target coal mine production area in the on-site monitoring video, It is the standard brightness of light for safe operation in the preset coal mine production area.

5. The method for identifying illegal behaviors in coal mine production safety based on video analysis according to claim 3, Features: Said The analysis process includes: tracking the activity position of the operating personnel located above the operating support structure of the target coal mine production zone in each video frame in the on-site monitoring video and calculating the relative distance between the operating personnel and the boundary of the operating support structure of the target coal mine production zone, comparing the relative distance with the preset operation boundary safety distance, obtaining the proportion of the operating personnel's out-of-bounds distance in each video frame in the on-site monitoring video, and accumulating them to evaluate the operating distance violation risk of the operating personnel located above the operating support structure of the target coal mine production zone in the on-site monitoring video.

6. The method for identifying illegal behaviors in coal mine production safety based on video analysis according to claim 3, Features: Said The analysis process includes: identifying the posture of the workers located above the operating support structure of the target coal mine production zone in each video frame in the on-site monitoring video, matching them with the posture sets of dangerous working postures of coal mine workers stored in the WEB cloud, and selecting the maximum matching degree to evaluate the risk of violation of the operating posture of the workers located above the operating support structure of the target coal mine production zone in the on-site monitoring video.

Images