Mine underground personnel risk identification and early warning method and system based on visual detection

Abstract

The present application relates to the field of image recognition, and more particularly to a coal mine underground personnel risk identification and early warning method and system based on visual detection. The method comprises the following steps: based on the visual camera in the coal mine underground roadway, continuously collecting images of the underground operation area, and constructing a standardized image sequence; performing frame-by-frame depth target detection and human body posture estimation on the standardized image sequence to generate a key node coordinate dataset; performing skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct a skeleton structure change trajectory; and performing graph neural convolution propagation calculation on the skeleton structure change trajectory to extract personnel action evolution features. The present application effectively improves the accuracy, real-time performance and early warning linkage capability of personnel dangerous behavior identification in complex coal mine environments, and reduces the probability of safety accidents and the cost of manual supervision.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method and system for risk identification and early warning of personnel in underground coal mines based on visual detection. Background Technology

[0002] With the continuous advancement of intelligent coal mine construction, underground safety monitoring is gradually shifting from traditional manual inspection methods to intelligent visual perception and automated early warning. Due to the characteristics of the underground coal mine environment, such as low illumination, high dust levels, confined spaces, and high personnel mobility, traditional safety supervision methods based on manual monitoring or simple sensor detection suffer from insufficient real-time performance, limited monitoring range, and low accuracy in identifying dangerous behaviors. While some coal mines have introduced video surveillance technology to monitor the underground environment, existing visual detection methods mostly remain at the stage of target recognition or simple behavior judgment, lacking the ability to deeply analyze the continuous changes in human movements. In complex coal dust environments and under low illumination conditions, traditional video surveillance is easily affected by factors such as image blurring, uneven brightness, target occlusion, and noise interference, leading to decreased stability in personnel detection. Simultaneously, existing behavior recognition technologies typically rely on single-frame image features for judgment, lacking the ability to analyze the temporal changes of the human skeleton and model the evolution of movements, thus making it difficult to accurately identify complex and dangerous behaviors underground. In addition, traditional early warning methods mostly use fixed threshold triggering mechanisms, which lack dynamic quantification and adaptive linkage capabilities for different risk levels, making them prone to false alarms, missed alarms, and untimely early warning responses. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a method and system for risk identification and early warning of personnel in underground coal mines based on visual detection, thereby solving at least one of the aforementioned technical problems.

[0004] To achieve the above objectives, this invention provides a method for risk identification and early warning of personnel in underground coal mines based on visual detection, comprising the following steps: Step S1: Continuously acquire images of the underground working area using visual cameras in the coal mine roadways, and construct a standardized image sequence; Step S2: Perform frame-by-frame depth target detection and human pose estimation on the standardized image sequence to generate a key node coordinate dataset; Step S3: Perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory; Step S4: Perform graph neural convolutional propagation calculation on the trajectory of skeletal structure changes to extract the evolutionary features of human movements; Step S5: Analyze the trends of movement changes and limb movement patterns of personnel's movement evolution characteristics, and mark typical risk behaviors of personnel; Step S6: Conduct comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

[0005] This specification provides a visual detection-based underground personnel risk identification and early warning system for coal mines, used to execute the visual detection-based underground personnel risk identification and early warning method described above, including: The image acquisition module is used to continuously acquire images of the underground working area based on a vision camera in the underground roadway of the coal mine, and to construct a standardized image sequence. The target detection module is used to perform frame-by-frame depth target detection and human pose estimation on standardized image sequences, and generate a dataset of key node coordinates. The continuous tracking module is used to perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory. The graph neural convolution module is used to perform graph neural convolution propagation calculations on the trajectory of changes in the skeletal structure and extract the evolutionary features of human actions. The behavior analysis module is used to analyze the trends of movement changes and limb movement patterns of personnel's action evolution characteristics, and to mark typical risky behaviors of personnel. The early warning module is used for comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

[0006] The specific benefits of this invention are as follows: By continuously acquiring images of the underground working area and combining brightness optimization, coal dust defogging, and noise suppression processing, the image clarity and stability under low illumination and high dust conditions can be effectively improved, enhancing the identifiability of personnel targets and environmental features, and providing a high-quality, standardized image data foundation for personnel detection and risk analysis. Through frame-by-frame depth target detection and human posture estimation, the location of underground workers and key structural nodes of the human body can be accurately identified, enabling independent identification of multiple personnel and extraction of action information, improving the accuracy of human body recognition and the stability of continuous tracking in complex environments. By performing dynamic graph neural connections and multi-frame continuous tracking analysis on the key node coordinate dataset, the dynamic change relationship of the human skeleton structure can be established, accurately reflecting the trajectory and posture change trend of personnel, thereby improving the temporal perception capability and behavioral continuity analysis capability of complex action processes.

[0007] By employing graph neural convolutional propagation computation to extract deep features from the trajectory of skeletal structural changes, the system can simultaneously learn the spatial correlation and temporal evolution features of human movements, improving the accuracy of complex action recognition and enhancing the intelligent identification of abnormal and high-risk actions. Through comprehensive analysis of movement trends and limb movement patterns, it can accurately identify typical risky behaviors such as falls, boundary crossings, climbing, and abnormal stillness, reducing false alarms and missed alarms, and improving the real-time performance and reliability of identifying dangerous behaviors of underground personnel. By comprehensively quantifying the duration and severity of risky behaviors and combining this with an adaptive directional early warning mechanism, dynamic linkage alarms and precise warnings at different risk levels can be achieved, improving the efficiency of underground safety incident response and reducing the risk of accident spread and the burden on manual monitoring. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the steps of a method for identifying and warning of risks to personnel in underground coal mines based on visual detection, according to the present invention. Figure 2 This is a detailed flowchart illustrating the implementation steps of step S1. Figure 3 This is a detailed flowchart illustrating the implementation steps of step S2; Figure 4 This is a detailed flowchart illustrating the implementation steps of step S3; Figure 5 This is a schematic diagram of the skeleton topology. Detailed Implementation

[0009] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0010] This application provides a method and system for identifying and warning of risks to underground personnel in coal mines based on visual detection. The executing entities of the method and system include, but are not limited to, the following: mechanical equipment, data processing platforms, cloud server nodes, network upload devices, etc., which can be considered as general computing nodes in this application. The data processing platform includes, but is not limited to, at least one of the following: an information management system, a cloud data management system.

[0011] Please see Figures 1 to 5 This invention provides a method for risk identification and early warning of personnel in underground coal mines based on visual detection, comprising the following steps: Step S1: Continuously acquire images of the underground working area using visual cameras in the coal mine roadways, and construct a standardized image sequence; Step S2: Perform frame-by-frame depth target detection and human pose estimation on the standardized image sequence to generate a key node coordinate dataset; Step S3: Perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory; Step S4: Perform graph neural convolutional propagation calculation on the trajectory of skeletal structure changes to extract the evolutionary features of human movements; Step S5: Analyze the trends of movement changes and limb movement patterns of personnel's movement evolution characteristics, and mark typical risk behaviors of personnel; Step S6: Conduct comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

[0012] In the embodiments of the present invention, see Figure 1 The diagram below illustrates the steps of a visual detection-based method for identifying and warning of risks to underground personnel in coal mines according to the present invention. In this example, the steps of the visual detection-based method for identifying and warning of risks to underground personnel in coal mines include: Step S1: Continuously acquire images of the underground working area using visual cameras in the coal mine roadways, and construct a standardized image sequence; Step S2: Perform frame-by-frame depth target detection and human pose estimation on the standardized image sequence to generate a key node coordinate dataset; Step S3: Perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory; Step S4: Perform graph neural convolutional propagation calculation on the trajectory of skeletal structure changes to extract the evolutionary features of human movements; Step S5: Analyze the trends of movement changes and limb movement patterns of personnel's movement evolution characteristics, and mark typical risk behaviors of personnel; Step S6: Conduct comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

[0013] In one specific embodiment, four low-light industrial vision cameras are installed on the top of the underground transport roadway and personnel work area in a coal mine to continuously acquire images of the underground area. The vision cameras use wide dynamic range CMOS sensors with a dynamic range of 120 dB, an image resolution of 1920×1080, a video acquisition frame rate of 30 fps, and lens temperature maintained within the range of 20℃ to 35℃. The acquired image stream of underground personnel operations is divided into image subsequences every 2 seconds, with each subsequence containing 60 frames. Brightness analysis is performed on the underground personnel operation image stream, converting the images from the RGB color space to the YCbCr color space and extracting the brightness component Y. A 32×32 pixel window is used to perform local brightness statistics on the images. Assuming the average pixel brightness in a certain local area is 58 and the standard target brightness value is set to 85, the brightness deviation value ΔL is:

[0014] Since the average brightness is 70% lower than the standard brightness, the exposure gain is increased and the gamma correction value is adjusted to 1.5. Subsequently, the CLAHE local brightness equalization method is used to enhance the image, with the local histogram cropping threshold set to 3.0, thus generating a brightness-optimized image stream. Next, coal dust blur feature recognition is performed on the brightness-optimized image stream. Fourier spectrum analysis shows that the high-frequency energy value of the image is 52% of the normal baseline value, lower than the 60% threshold, therefore, it is determined that there is significant coal dust occlusion in the current area. Furthermore, the dark channel prior method is used to calculate the coal dust concentration parameter. Assuming the dark channel change rate is 0.38, the formula for estimating the visual coal dust concentration feature value is:

[0015] in, This indicates the coal dust concentration value. Indicates the rate of change of the dark channel. This is the concentration proportionality coefficient, with a value of 120. When... At that time, the calculated coal dust concentration was 45.6 mg / m³, which belongs to a medium-to-high concentration coal dust environment. Subsequently, an improved dark channel dehazing algorithm was used for image restoration, and high-frequency noise suppression was achieved by combining 5×5 bilateral filtering and nonlocal mean filtering with a search radius of 15 pixels, thus constructing a standardized image sequence.

[0016] Frame-by-frame depth target detection was performed on standardized image sequences. Images were uniformly scaled to 640×640 pixels and input into a YOLO target detection network, with transfer learning trained using a downhole environment dataset. In a given frame, three workers were detected, with corresponding bounding box confidence scores of 0.92, 0.88, and 0.81. The confidence threshold was set to 0.65–0.85, preferably 0.7, all higher than the target confidence threshold of 0.7, thus classifying them as valid personnel targets. A non-maximum suppression algorithm with an intersection-union ratio (IUU) threshold of 0.5 was further used to eliminate duplicate bounding boxes, and unique identifiers (IDs) were assigned to the three workers: ID1, ID2, and ID3. Subsequently, the corresponding bounding boxes underwent boundary expansion processing, with an expansion ratio of 10%, resulting in independent personnel image boxes. These personnel image boxes were then uniformly scaled to 256×192 pixels and input into the HRNet human pose estimation model for human key node recognition. For the person corresponding to ID2, the coordinates of the head, shoulder, elbow, knee, and ankle nodes were extracted. The coordinates of the head node are (612, 214), the left shoulder node is (598, 260), the right shoulder node is (628, 262), the left knee node is (604, 438), and the right ankle node is (621, 516). Subsequently, subpixel interpolation was used to refine the node coordinates, controlling the localization error of key nodes within the range of 1 to 3 pixels, and forming a key node coordinate dataset.

[0017] Furthermore, based on the connection relationships of the human skeleton, dynamic graph neural connections are performed on the key node coordinate dataset to construct a skeleton topology graph. Let the set of human skeleton nodes be:

[0018] in, Represents key structural nodes of the human body; the set of connecting edges of the skeleton is:

[0019] The human skeleton topology graph is constructed using the nodes and edges mentioned above. Subsequently, multi-frame continuous tracking analysis was performed on 30 consecutive frames of human skeleton data. Let the coordinates of the left knee node in frame t be... The coordinates of the left knee node in frame t+1 are: The formula for calculating the change in nodal displacement is:

[0020] When the coordinates of the left knee node in frame 15 are (604, 438) and the coordinates in frame 16 are (615, 462), the displacement change is:

[0021]

[0022]

[0023] Because the knee displacement change exceeded a preset threshold of 20 pixels per unit time, the human body was determined to have a rapid posture change. Temporal variation fitting was performed on the motion trajectories of skeleton nodes in consecutive time frames to construct the skeleton structure change trajectory.

[0024] Graph convolutional propagation is performed on the trajectory of changes in the skeleton structure. Let A be the adjacency matrix of the skeleton topology graph and X be the node feature matrix; then the graph convolutional propagation formula is:

[0025] in, , Represents the identity matrix. Degree matrix, This represents the weight parameters of the l-th layer convolution. Through graph convolution propagation, motion correlation features between the shoulder, knee, and ankle of the human body are extracted, yielding local pose features. Subsequently, these local pose features are input into a spatiotemporal graph convolutional behavior recognition model, which uses temporal convolution to dynamically model and extract the evolutionary features of human movements. If a rapid descent of the human body's center of gravity, a sharp bending of the knee angle, and an ankle support point offset exceeding 30 pixels are detected within 20 consecutive frames, it is determined that the person is exhibiting a tendency to fall.

[0026] Furthermore, the evolutionary characteristics of human movement are analyzed by examining trends in movement changes and limb movement patterns. Let the rate of change of the human body's center of gravity be... The rate of change of the knee angle The formula for calculating the risk behavior score is:

[0027] in, and These represent weighting coefficients, with values ​​of 0.6 and 0.4 respectively. When the detection yields... , When this happens, the risk behavior score is:

[0028]

[0029]

[0030] Because the risk score is higher than the risk threshold of 0.75, the person is determined to have typical fall risk behaviors, and the risk behavior is marked.

[0031] A comprehensive risk quantification is performed on typical risky behaviors of personnel. Assuming the duration of the risky behavior is 12 seconds and the risk level coefficient is 0.85, the formula for calculating the behavioral risk index is:

[0032] in, Indicates a behavioral risk index. Indicates duration. This represents the risk level coefficient. Substituting the parameters, we get:

[0033] The preset risk assessment gradient rules use a quantification scale of 0-100, where 0-30 is defined as low risk, 30-60 as medium risk, and above 60 as high risk. The result is determined to be high risk. Therefore, a dangerous behavior warning signal is triggered, and dynamic warning processing is executed, including coordinated activation of downhole audible and visual alarm devices, broadcast devices, and dispatch terminals for targeted warnings. Image segments within 10 seconds before and after the corresponding time period are automatically captured and stored for subsequent risk tracing and accident analysis.

[0034] In this embodiment, see Figure 2 The diagram below illustrates the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Based on the visual camera in the underground roadway of the coal mine, continuous image acquisition of the underground working area is carried out to obtain the image stream of underground personnel working; Calculate brightness distribution parameters based on the image stream of personnel working underground; Based on the brightness distribution parameters, adaptive exposure compensation and local brightness equalization are performed to generate a brightness-optimized image stream. Dust blur feature recognition is performed based on brightness-optimized image stream, and visual concentration estimation is performed to generate coal dust concentration features; Based on the characteristics of coal dust concentration, the brightness-optimized image stream is dehazed and high-frequency noise is suppressed to construct a standardized image sequence.

[0035] In this embodiment, low-light explosion-proof vision cameras are installed on the top of the underground mining area, transport roadways, and equipment operation area. These cameras employ a wide dynamic range CMOS imaging module, with a minimum illumination of no more than 0.001 Lux, and are equipped with infrared supplementary lighting components to ensure continuous imaging capability in low-light environments. The vision cameras are deployed in a distributed manner at intervals of 15m to 25m, with at least 20% overlap in the field of view between adjacent cameras to ensure continuous image acquisition. Edge acquisition terminals continuously capture video of the underground area at a sampling frequency of 25fps to 30fps, with a preferred video resolution of 1920×1080 to ensure the integrity of personnel movement details and contour features. During acquisition, real-time information on changes in underground environmental brightness is obtained, and the exposure time and image gain parameters are dynamically adjusted based on changes in miner lamp movement, equipment reflections, and local shadows in the roadways. The exposure time is controlled between 1ms and 12ms, and the photosensitive gain is controlled between 6dB and 18dB to prevent image overexposure or ghosting. Simultaneously, the electronic image stabilization module is used to compensate for the displacement of continuous image frames. By calculating the offset of feature points between adjacent image frames, pixel-level correction is performed on the image jitter caused by mechanical vibration, so that the image jitter deviation is controlled within 2 pixels. The continuous images are cached and sorted according to a unified timestamp to form a stable and continuous image stream of underground personnel operations.

[0036] The acquired image stream of personnel working underground was processed to grayscale, and a corresponding brightness distribution matrix was constructed. Each frame of the image was divided into multiple local brightness analysis regions, preferably using a 32×32 pixel unit to establish a local brightness grid. The average brightness value, brightness variance, and brightness gradient change rate of each region were calculated. Since the illumination from miner lamps in the underground environment exhibits significant local concentration, the pixel ratio of bright and dark areas in the image was further statistically analyzed. When the proportion of bright areas exceeded 30%, the current image was considered to have a risk of local overexposure; when the overall average brightness was below 60 grayscale values, the current area was considered to be in a low-light environment. Simultaneously, based on the brightness histogram, probability statistics were performed on grayscale levels 0–255 to obtain the brightness peak position, dark area coverage ratio, and brightness dispersion, reflecting the current brightness and darkness distribution in the underground environment. To improve the stability of the brightness analysis, a time-domain brightness change analysis was performed on 10 consecutive frames of images. By calculating the brightness fluctuation curves between consecutive frames, instantaneous brightness abrupt changes caused by rapid movement of miner lamps and equipment reflections were identified. When the brightness change of consecutive frames exceeds 20%, the corresponding brightness anomaly information is automatically recorded, and brightness distribution parameters including the average brightness of the area, the brightness gradient change rate, the brightness dispersion, and the dynamic brightness fluctuation value are generated.

[0037] Adaptive exposure compensation processing is applied to the image stream of personnel working underground based on brightness distribution parameters. The exposure compensation coefficient is dynamically calculated based on the overall average brightness value of the image. When the average brightness is below a preset threshold, a nonlinear gain compensation method is used to increase the brightness of dark areas. When overexposure due to strong light reflection or direct mine lamp illumination is detected in local areas, a brightness compression algorithm is used to reduce the exposure intensity of bright areas, thus balancing the overall image brightness. The exposure compensation gain is controlled between 1.1 and 2.5 times to avoid significant noise amplification during brightness enhancement. Local brightness equalization processing is performed on the image, dividing it into multiple 64×64 pixel local enhancement windows, and establishing brightness mapping curves for each corresponding area. For dark areas, local contrast is increased to enhance the clarity of human body edges and equipment contours; for bright areas, the brightness gradient variation is reduced to weaken the interference of mine lamp reflections on visual recognition. Temporal smoothing is performed on the brightness changes between consecutive video frames. When the brightness change between two consecutive frames exceeds 15%, the exposure compensation parameters are gradually adjusted by time interpolation to prevent obvious flickering in the video and generate a brightness-optimized image stream with balanced and stable brightness and clear edge details.

[0038] To address image blurring caused by suspended coal dust particles in underground environments, dust blurring feature analysis was performed on brightness-optimized image streams. The Sobel edge gradient operator was used to extract image edge features, and the edge pixel density and average edge gradient within a unit area were statistically analyzed. When the average edge gradient was below a preset threshold, significant coal dust occlusion was identified. Furthermore, the contrast changes in local image regions were analyzed, and visually fogged areas formed by coal dust diffusion were identified by calculating the rate of change in brightness difference between adjacent regions. When the local contrast decrease exceeded 40%, a high concentration of coal dust was identified in the current area. By pre-collecting standard sample images under different coal dust concentration environments, a mapping relationship between edge attenuation rate, image transmittance, and actual coal dust concentration was established. Based on the edge attenuation coefficient, transmittance parameter, and local blurring degree in the current image, the coal dust concentration was dynamically estimated. When the image transmittance was below 0.65, the current environment was determined to be in a high coal dust concentration state. Simultaneously, the motion trajectory of coal dust particles is extracted by continuous frame difference method, and the diffusion speed and diffusion range of coal dust are analyzed to generate coal dust concentration features including visual coal dust concentration value, edge attenuation coefficient, image transmittance and coal dust diffusion speed.

[0039] Dynamic dehazing enhancement is performed on the brightness-optimized image stream based on coal dust concentration characteristics. When the coal dust concentration is high, the dehazing compensation coefficient is increased; when the coal dust concentration is low, the enhancement amplitude is reduced to avoid over-sharpening of the image. A dehazing method based on dark channel prior is used to restore transmittance in coal dust-obscured areas and perform detail enhancement on distant blurred areas, so that the outlines of underground personnel and equipment edges are clearly displayed again. High-frequency noise suppression processing is performed on the image. Since low-light environments and dehazing enhancement are prone to introducing random noise, a multi-scale noise filtering mechanism is used to perform layered smoothing processing on high-frequency texture areas. Random isolated noise points in the image are identified, and then the noise areas are adaptively filtered based on the spatial neighborhood pixel similarity, while preserving key texture information of human body edges and equipment structure areas. When the image signal-to-noise ratio is below 20dB, the noise suppression level is automatically increased to reduce image graininess. At the same time, temporal texture consistency analysis is performed on continuous video frames, and short-term random noise and flickering are eliminated through a continuous frame averaging mechanism, outputting a standardized image sequence with uniform brightness, clear edges, and low noise characteristics.

[0040] In this embodiment, see Figure 3 The diagram below illustrates the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Perform frame-by-frame depth target detection on standardized image sequences and mark all personnel working within the area; A unique identifier ID is assigned to all workers in the area; based on the unique identifier ID, independent image frame segmentation is performed to obtain multiple personnel image frames; Perform human pose estimation on multiple person image frames and extract key human structure nodes for each person; Calculate the image pixel coordinates of the key structural nodes of the human body to generate a key node coordinate dataset.

[0041] In this embodiment, the input image is scale-normalized to uniformly adjust the image resolution to 1280×720 to improve detection efficiency and ensure the integrity of human contour details. A feature extraction network extracts texture, edge, and human contour features layer by layer, and a multi-scale feature fusion method enhances the recognition ability of small targets at a distance. Due to the low illumination, coal dust obstruction, and equipment obstruction in the underground environment, a local enhancement detection mechanism is established for the head and shoulder area, torso area, and leg area. Regional feature aggregation improves the stability of personnel detection in complex environments. During detection, target areas are filtered according to a confidence threshold, set between 0.65 and 0.85. When the target detection probability exceeds the threshold, it is determined that a worker exists in that area. To avoid target overlap caused by multiple workers working together, a non-maximum suppression method is further used to filter overlapping target boxes. The overlap intersection-union threshold is preferably set to 0.45 to reduce duplicate detection. In each frame of the image, a corresponding personnel detection box is generated, and the location of all workers in the area is marked to form a frame-by-frame personnel detection result.

[0042] For each frame of the image, the personnel detection results are processed for target identity association. By combining target appearance features and motion trajectory features, a unique identifier ID is assigned to each worker. Human body color distribution features, contour size features, and spatial location features are extracted for each personnel detection box, and a corresponding personnel feature vector is established. Combining the target movement trajectory between consecutive video frames, similarity matching is performed on personnel targets in adjacent frames. When the target appearance similarity exceeds 85% and the motion trajectory offset distance is less than a preset threshold, they are determined to be the same personnel target, and the original unique identifier ID remains unchanged. For newly appearing personnel targets, a new unique identifier number is automatically assigned to ensure the stability of continuous identity tracking in a multi-person environment underground. After identity association is completed, independent image frame segmentation is performed on the corresponding personnel area based on the unique identifier ID. During image frame segmentation, the target area is cropped according to the human body contour boundary, and a background buffer area of ​​10-20 pixels around the target is retained to avoid posture recognition errors caused by human body edge truncation. The segmented personnel image frames are uniformly scaled to a fixed size, preferably 256×256 or 384×384 pixels, to ensure the consistency of human structure proportions in subsequent human pose estimation. Multiple independent personnel image frames with unique identifiers are then generated.

[0043] Human pose estimation is performed on each of the segmented personnel image frames, and key structural nodes are extracted through human skeleton structure analysis. Human region normalization is applied to the personnel image frames, establishing a standard human structural region using the center position of the human body as a reference coordinate. Using a human key point heatmap detection method, probability distribution analysis is performed on key parts such as the head, shoulders, elbows, knees, and ankles, and the positions of key nodes are determined through local peak search. Due to the presence of miner lamp occlusion, motion distortion, and partial occlusion in the underground environment, a local structural constraint mechanism is introduced for the joint areas of the human body, dynamically correcting the positions of key nodes based on the human skeleton's connection relationships. When the elbow area is occluded, the possible position of the elbow is calculated based on the relative angle between the shoulder and wrist to improve the stability of key node recognition under complex working conditions. Constraint associations are established for the left-right symmetrical structure of the human body, and abnormal posture changes are identified through shoulder height difference, knee angle, and ankle position offset. The confidence level for detecting key human body nodes is preferably controlled above 0.7. When the probability of key node recognition is lower than the threshold, compensation and correction are performed by combining the node motion trajectories in consecutive frames. The key human body structural nodes of each person are extracted, and the corresponding human skeleton structure information is formed.

[0044] Pixel coordinate calculations are performed on key human structural nodes, establishing a spatial coordinate system for these nodes using the top-left corner of the image as the origin. The center position of each key structural node is extracted, and its horizontal (X) and vertical (Y) coordinates are recorded to form two-dimensional pixel coordinate data. To improve the stability of key node coordinates, temporal smoothing is applied to the key node positions in consecutive video frames. By calculating the node displacement change trend over three to five consecutive frames, abrupt coordinate changes are filtered and corrected to reduce interference from equipment vibration and image noise on key node localization. Furthermore, relative distance parameters and joint angle parameters between key human nodes are calculated, such as the distance between the shoulder and elbow, the knee flexion angle, and the head tilt angle, to describe the characteristics of changes in human movement. For fast-moving targets, the possible positions of key nodes in the next moment are estimated in advance to reduce the loss of key nodes due to rapid movement. After extracting all key node coordinates, the corresponding node data is associated and stored according to the unique personnel ID, generating a key node coordinate dataset containing the personnel ID, key node coordinates, inter-node distance parameters, and joint angle parameters.

[0045] In this embodiment, see Figure 4 Here is a detailed flowchart of the implementation steps for step S3, which includes the following steps: Based on the connection relationships of the human skeleton, the key node coordinate dataset is topologically connected to construct a skeleton topology graph. Multi-frame continuous tracking analysis of the skeleton topology map is performed based on standardized image sequences to extract the position change data of the skeleton topology map in adjacent time frames; By fitting the temporal variation data of the skeleton topology map position change data, the trajectory of the skeleton structure change is constructed.

[0046] In this embodiment, a human skeleton graph topology is constructed based on the trajectory of changes in the human skeletal structure. Key structural nodes such as the head, shoulders, elbows, knees, and ankles are used as node units in a graph neural network, and connection edges are established between nodes based on the natural skeletal connections. Shoulder nodes are connected to elbow nodes, and knee nodes are connected to ankle nodes, thus forming a complete human skeleton graph structure. Each node corresponds to the two-dimensional pixel coordinates, movement speed, and node angle change value of a key part of the human body, and each connection edge corresponds to the spatial relationship between human bones. Graph neural convolution propagation calculation is performed on the human skeleton graph, and local posture change features are extracted by aggregating node neighborhood information. During graph convolution propagation, the spatial relationship weight between adjacent nodes is calculated for each skeleton node, and the connection edge weights are dynamically adjusted based on node distance and node movement direction. When the movement directions between the shoulder and elbow are consistent, the propagation weight of the corresponding edge structure is increased; when there are abnormal offsets between nodes, the connection edge weights are decreased to enhance the feature representation ability of abnormal movements. The optimal number of graph convolution propagation layers is set to 3 to 5. Each layer of graph convolution extracts spatial pose features at different levels. The first layer mainly extracts local joint motion information, the second layer extracts limb linkage features, and the third layer extracts the overall human posture change relationship. To improve the stability of action recognition in complex underground environments, and in conjunction with a dynamic node confidence correction mechanism, the feature propagation intensity is reduced for nodes with low recognition confidence to reduce the impact of occlusion and coal dust interference on human posture analysis. This generates local posture feature data that includes local joint motion features, limb direction change features, and overall human posture structure features.

[0047] Hierarchical behavioral association analysis is performed on the local posture features obtained after graph neural convolution propagation to establish the coordinated motion relationships between different limb structures during human movement. Low-level behavioral association extraction is performed on local posture features, obtaining single joint movement features by analyzing local motion changes between the shoulder, elbow, knee, and ankle. Squatting is identified by calculating the rate of change of knee flexion angle, and arm swinging is identified by calculating the change in movement direction between the shoulder and elbow. Mid-level behavioral association analysis is performed on the linkage relationships between multiple local joint movements, analyzing the temporal synchronization and spatial correlation between different limb movements by establishing a coordinated motion matrix between joints. When both knee nodes simultaneously contract their angles and the body's center of gravity continuously descends, a falling tendency is identified; when one shoulder and elbow rapidly rise and the body's center of gravity shifts, climbing or crossing movements are identified. Furthermore, high-level behavioral association extraction is performed on the overall skeletal motion state of the human body, performing overall behavioral analysis on complex continuous movements by statistically analyzing the trajectory of the body's center of gravity movement, changes in body tilt angle, and limb coordination. In the hierarchical association extraction process, corresponding weight parameters are established for features at different levels. The weight for low-level local action features is preferably set to 0.3, the weight for mid-level limb coordination features to 0.4, and the weight for high-level overall behavior features to 0.3, to balance local action details and overall behavior trends. Temporal smoothing is applied to the behavior change trends in continuous video frames. Short-term action change patterns are analyzed through a continuous action association window, with the window length preferably set to 15–30 frames, to reduce the impact of instantaneous action fluctuations on the behavior recognition results. This results in multi-level behavior association features that include local joint action features, limb coordination change features, and overall human behavior trend features.

[0048] Multi-level behavioral association features are input into a pre-trained spatiotemporal graph convolutional behavior recognition model to perform dynamic temporal analysis of continuous human movements. The input behavioral association features are used to construct an action sequence tensor in chronological order, and a dynamic graph structure with key human nodes is established based on the temporal dimension. The spatial graph convolution module aggregates features of the human skeleton structure within the same time frame, and the temporal convolution module further analyzes the patterns of human movement changes between consecutive time frames. During temporal convolution, a sequence of 20 to 40 consecutive frames of human movement is used as the temporal analysis window, and the evolution trend of the movement is extracted using a sliding time window method. When human movement changes rapidly within a short period, the temporal convolution response weights are increased to enhance the ability to identify abnormal behavioral changes. When the center of gravity of the human body drops rapidly within several consecutive frames and the knee angle continues to contract, a fall is identified as an evolutionary trend; when the human body maintains low-amplitude periodic movements for a long time, it is identified as equipment operation behavior. To improve the stability of action recognition in complex environments, a historical action state memory mechanism is further integrated to cache the features of previous action states and perform joint analysis with the current action change trend. This reduces recognition errors caused by short-term action occlusion or loss of key nodes. The velocity change rate, joint angle change rate, and body center of gravity shift during the action evolution process are dynamically calculated to extract human action evolution features that include continuous change features, action development trend features, and action instability features. This provides a dynamic behavioral data foundation for subsequent risk behavior identification and hazard warning analysis.

[0049] In this embodiment, step S4 includes the following steps: Graph neural convolutional propagation calculations are performed on the trajectory of changes in the skeletal structure to extract local pose features in the spatial dimension; The local pose features are extracted layer by layer according to hierarchical dimensions to obtain multi-level behavioral association features; The multi-level behavioral association features are input into a preset spatiotemporal graph convolutional behavior recognition model to perform temporal convolutional dynamic modeling and extract the evolution features of human actions.

[0050] In this embodiment, a human skeleton graph structure is constructed based on the trajectory of changes in the human skeletal structure. Key structural nodes such as the head, shoulders, elbows, knees, and ankles are used as node units in the graph structure, and connection edges are established between nodes based on the natural skeletal connections. Head nodes are connected to shoulder nodes, shoulder nodes to elbow nodes, and knee nodes to ankle nodes, thus forming a complete human skeleton topology. Each node corresponds to the two-dimensional pixel coordinates of a key part of the human body, the node's movement speed, and the node's direction change value. Each connection edge represents the spatial relationship between human bones. Graph neural convolution propagation calculation is performed on the human skeleton graph, and local posture change features are extracted through neighborhood node feature aggregation. During graph convolution propagation, the adjacency matrix between nodes is calculated, and spatial propagation weights are established based on the Euclidean distance and movement direction changes between nodes. When there is a synchronous movement trend between adjacent nodes, the propagation weight of the corresponding connection edge is increased; when there is a significant deviation in the node's movement direction, the weight of the connection edge is decreased to enhance the expressive ability of abnormal movement changes. The optimal number of convolutional layers in the graph is 3 to 5. The first layer extracts local joint motion features, the second layer extracts limb linkage features, and the third layer extracts overall skeletal spatial structure features. To improve recognition stability in complex environments, a node confidence correction mechanism is used to reduce the feature propagation intensity of key nodes with a recognition confidence below 0.6. This reduces the impact of coal dust occlusion, local shadows, and overlapping of people on skeletal feature extraction, generating local posture features that include joint movement direction, local limb changes, and human spatial posture relationships.

[0051] Multi-level behavioral correlation analysis is performed on local posture features to establish the coordinated motion relationships between different joint movements of the human body. Low-level behavioral feature extraction is performed on local joint movements, identifying basic human movement states by analyzing angle, displacement, and velocity changes between the shoulder, elbow, knee, and ankle. Squatting is identified by the rate of change in knee flexion angle, and arm swinging is identified by the change in the direction of movement between the shoulder and elbow. Mid-level behavioral correlation analysis is performed on the relationships between multiple local movements, constructing a joint coordinated motion matrix to analyze the synchronous changes and spatial linkages between different limb movements. When both knee nodes simultaneously contract and the body's center of gravity continuously descends, it is identified as a falling tendency; when one shoulder and elbow continuously rise and are accompanied by body tilting, it is identified as a climbing or crossing movement. Furthermore, high-level behavioral trend analysis is performed on the overall skeletal structure of the human body, statistically analyzing the trajectory of the body's center of gravity movement, the rate of change in body tilt angle, and the coordination of limb movements to model the overall behavior of complex continuous movements. In the hierarchical feature association process, corresponding weight parameters are set for behavioral features at different levels. The weight for low-level local action features is preferably set to 0.3, the weight for mid-level limb coordination features to 0.4, and the weight for high-level overall behavioral trend features to 0.3, ensuring a balance between local action details and overall behavioral trends. Temporal smoothing analysis is performed on behavioral changes in consecutive video frames. By establishing a continuous action analysis window of 15 to 30 frames, short-term action fluctuations are smoothed to reduce the interference of instantaneous action changes on the behavioral analysis results, forming a multi-level behavioral association feature system that includes local action association features, limb coordination change features, and overall behavioral trend features.

[0052] Multi-level behavioral association features are input into a pre-defined spatiotemporal graph convolutional behavior recognition model to perform dynamic time-series analysis on continuous human movements. The input human behavioral association features are used to construct an action sequence tensor in chronological order, and a time-dimensional graph structure is established based on the dynamic relationships between key human nodes. The spatial graph convolution module aggregates features of the human skeleton structure within the same time frame, and the temporal convolution module is further used to analyze the changing patterns of human movements between consecutive time frames. During temporal convolution, a continuous sequence of 20 to 40 frames of human movement is used as a time analysis window, and the trend of movement development is extracted using a sliding window approach. When human movements change rapidly within a short period, the temporal convolution response weights are increased to enhance the ability to identify the evolution of dangerous movements. When the center of gravity of a person rapidly decreases within several consecutive frames and the knee angle continuously contracts, it is identified as a fall trend. When a person maintains low-amplitude repetitive movements for a long time, if they are located in a permitted work area and exhibit periodic arm movements, it is identified as equipment operation; if they are located in a transportation area, a high-risk equipment area, or have no obvious operational movements and the timeout period continues, it is identified as an abnormal stillness. To improve the stability of action recognition in complex environments, historical action states are cached as features and jointly analyzed with current action change trends to reduce recognition errors caused by short-term occlusion and loss of key nodes. The body's center of gravity shift, joint angle change rate, and action speed change rate are dynamically calculated during action evolution. The preferred threshold for body center of gravity shift is set at 15%–25% of the body's height to identify imbalances and dangerous action trends. Human action evolution features, including continuous action change characteristics, action development trend characteristics, and action instability change characteristics, are extracted.

[0053] In this embodiment, the pre-defined spatiotemporal graph convolutional behavior recognition model adopts the ST-GCN (Spatial Temporal Graph Convolutional Network) spatiotemporal graph convolutional network to uniformly model and recognize the trajectory of skeletal structure changes. This model uses key nodes of the human skeleton as graph structure nodes and the natural skeletal connections as graph structure edge connections. The input data is constructed as an N×C×T×V×M spatiotemporal tensor, where N is the batch size, preferably 8–32; C is the number of node feature channels, taking 3 dimensions, including X-coordinate, Y-coordinate, and motion velocity; T is the time frame length, preferably 30–60 frames; V is the number of key nodes, preferably 17; and M is the number of simultaneously existing human targets, preferably 1–3, thus forming a standardized skeletal spatiotemporal input structure.

[0054] In terms of model structure, ST-GCN consists of alternating stacks of spatial graph convolutional modules and temporal convolutional modules. The number of spatial graph convolutional layers is preferably 8 to 10. Features of spatial relationships between human joints are propagated through an adjacency matrix, and the skeleton is divided into three regions: root nodes, centripetal nodes, and centrifugal nodes, each assigned different propagation weights to enhance the extraction of limb movement direction features. The temporal convolutional module uses a 9×1 one-dimensional convolutional kernel, performing sliding calculations in a 9-frame window with a stride of 1 to capture the continuous changing trends of human movements and enhance response weights for rapidly changing movements (such as falls and climbing). Simultaneously, residual connection structures are introduced between the graph convolutional layers to maintain the continuous transmission of low-level movement details and avoid information decay in deeper networks.

[0055] Regarding feature output and model parameters, after multi-layer spatiotemporal graph convolution and temporal modeling, the spatiotemporal features are compressed into a 256-dimensional action feature vector through a global average pooling layer. This vector is then input into a Softmax classification output layer for action recognition. The output categories include seven types of behavior: normal walking, bending over to work, equipment operation, falling, crossing boundaries, climbing, and abnormal stillness. Model training uses the SGD optimizer with a learning rate of 0.01, momentum of 0.9, weight decay of 0.0005, batch size of 16–32, and 80–120 training epochs. This spatiotemporal graph convolutional behavior recognition model achieves high-precision recognition and classification output of personnel action evolution features based on joint modeling of spatial skeleton structure relationships and temporal dynamic changes.

[0056] In this embodiment, the specific steps of step S5 are as follows: The evolutionary characteristics of human movements are analyzed to determine the trends in movement changes and limb movement patterns, resulting in a classification of movement types. The risk behavior is judged based on the action type classification results, and typical risk behaviors of personnel are marked. The typical risk behaviors of personnel include falling, crossing boundaries, climbing, and abnormal stillness.

[0057] In this embodiment, continuous motion trend analysis is performed on the evolution characteristics of human movements. Dynamic analysis of human movements is conducted through displacement changes, joint angle changes, and the trajectory of the human body's center of gravity. The positional changes of key nodes such as the head, shoulders, elbows, knees, and ankles in a continuous time series are statistically analyzed, and the motion velocity, acceleration, and rate of change of direction between each key node are calculated. The motion state of the lower limbs is analyzed by the angle changes between the knee and ankle nodes in continuous time frames, and the balance state of the upper body is analyzed by the tilt angle changes between the shoulder and head nodes. The overall center of gravity position of the human body is dynamically tracked, and a human motion trajectory curve is established based on the offset of the human skeleton's center point in continuous frames. When the center of gravity of the human body rapidly decreases within a short period and the offset exceeds 20% of the human height, an abnormal imbalance trend is determined. Human limb movement patterns are classified and analyzed, and pattern matching is performed on the joint linkage relationships in different movement processes by establishing human motion feature vectors. When both knees are continuously bent and the upper body leans forward, it is identified as a bending motion; when the shoulder and elbow joints exhibit periodic reciprocating motions, it is identified as an arm-swinging motion; when the body's overall center of gravity moves in a stable direction, it is identified as a normal walking motion. During motion analysis, a time window is established for continuous motion sequences, with the window length preferably set to 20 to 40 frames. The continuity of motion within the window is smoothed to reduce misidentification caused by instantaneous motion fluctuations. By establishing a motion change rate threshold, fast and slow motions are distinguished; for example, when the limb angle change rate exceeds 30° / s, it is determined to be a violent motion state. The system generates motion type classification results including normal walking, bending over while working, equipment operation, falls, boundary crossings, climbing, and abnormal stillness.

[0058] Further risk behavior analysis was conducted on the action type classification results, including fall identification analysis. This was performed by jointly judging the rate of descent of the body's center of gravity, changes in body tilt angle, and changes in knee flexion angle across consecutive time frames. A fall was identified when the descent of the body's center of gravity exceeded 30% of the body's height within five consecutive frames, and the body tilt angle exceeded 60°. A secondary confirmation was also made based on the duration of stillness after a fall; if the continuous stillness after a fall exceeded 3 seconds, the risk level assessment weight was increased.

[0059] For boundary crossings, the real-time location coordinates of personnel are compared with the boundary of the dangerous area by combining the information of the electronic fence of the underground dangerous area. Fixed industrial cameras are used to continuously monitor the underground roadway. The camera installation height is preferably between 2.5 meters and 6 meters, and the viewing angle is preferably between 30 degrees and 75 degrees. The camera's intrinsic and extrinsic parameter matrices are obtained through a checkerboard calibration board or feature point calibration method. The intrinsic parameters include focal length parameters fx and fy, principal point coordinates cx and cy, and radial distortion coefficients k1 to k3. The extrinsic parameters include the rotation matrix R and translation vector T, used to describe the spatial relationship between the camera coordinate system and the roadway world coordinate system. After calibration, the roadway floor is used as a unified reference plane, and the electronic fence of the dangerous area is geometrically modeled in the roadway coordinate system. It is defined as a polygonal region composed of at least four vertices, with vertex coordinates described in meters. For example, the accuracy of the region boundary point coordinates is preferably controlled within ±0.05 meters to ensure spatial judgment accuracy. Subsequently, a mapping relationship between the image plane and the tunnel plane is established using a homography transformation method. The homography matrix is ​​obtained by solving for the corresponding points of at least four sets of pixel coordinates and actual spatial coordinates. This matrix is ​​periodically corrected during operation, with a preferred correction period of 24 to 72 hours to reduce the accumulation of errors caused by slight camera displacement. In actual detection, the pixel coordinates of key human body nodes are input into the mapping relationship and converted into tunnel plane coordinates. The spatial positioning error is preferably controlled within the range of 0.1 to 0.3 meters. A spatial inclusion relationship is determined with the polygonal area of ​​the electronic fence. When the human body's plane coordinates enter the danger zone and remain there for more than 2 seconds, a boundary crossing risk assessment is triggered, thus achieving highly reliable boundary crossing detection based on real spatial geometric constraints. When the coordinates of a person's key nodes enter the preset danger zone and remain there for more than 2 seconds, a boundary crossing risk is determined.

[0060] For climbing behavior, it is identified by analyzing the rate of change in shoulder height, knee elevation angle, and the vertical movement trend of the body's center of gravity. Climbing behavior is identified when the body's center of gravity moves continuously upwards and the angle of one knee is consistently greater than 90°. For abnormal stationary behavior, the continuous displacement of key body nodes is analyzed. An abnormal stationary state is determined when the overall displacement amplitude is less than 5 pixels within a set time window and the duration exceeds a preset threshold. The current work area type is also considered for comprehensive judgment; for example, prolonged stationary states in transportation areas or high-risk equipment areas are classified with higher risk levels. During the risk behavior analysis, corresponding risk scoring parameters are established for different risk behaviors. The optimal risk weights are: fall behavior (0.9), boundary crossing (0.8), climbing behavior (0.7), and abnormal stationary behavior (0.6), to achieve graded identification of different dangerous behaviors. Typical risk behaviors are identified and labeled, and corresponding risk behavior classification results are generated.

[0061] In this embodiment, step S6 includes the following steps: Calculate the duration of the typical risk behaviors of the individuals; The risk level of typical risk behaviors of the personnel is assessed to obtain the risk level of each behavior type. A comprehensive risk quantification is performed on the duration and the degree of risk to generate a behavioral risk index; Based on the preset risk assessment gradient rules, the behavioral risk index is compared and analyzed to adaptively generate early warning signals for dangerous behaviors; Dynamic early warning processing is performed based on dangerous behavior warning signals, and source image fragments are extracted and stored; the dynamic early warning processing includes hierarchical linkage control of downhole audible and visual alarm devices, broadcasting devices and dispatch terminals to perform adaptive directional early warning.

[0062] In this embodiment, a corresponding behavior state marker is established for each risky behavior. When the evolution characteristics of a person's actions first meet the risky behavior judgment conditions, the current video frame is defined as the behavior start frame, and the corresponding timestamp information is recorded. The status of key human nodes, the trend of changes in the human center of gravity, and the continuity of actions in consecutive video frames are continuously tracked. When the same risky behavior characteristics are maintained within consecutive time frames, the risky behavior state remains unchanged. When a person falls and remains in a low posture with a body tilt angle exceeding 60°, the person is continuously judged as having fallen. When key human nodes remain within the electronic fence of the danger zone, the person remains in an out-of-bounds state. To reduce misjudgments caused by instantaneous movement fluctuations, a time-stability window is established for the continuity of the behavior. The window length is preferably set to 10 to 20 frames. The duration of the risky behavior is accumulated only when the risk conditions are met within consecutive windows. The duration of the risky behavior is calculated by converting the video frame number and the video sampling frame rate. For example, under a 30fps video stream, if the risky behavior state is maintained for 300 consecutive frames, the corresponding duration of the risky behavior is 10 seconds. By performing time-series correlation analysis on intermittent risk behaviors, when the time interval between two identical risk behaviors is less than a preset threshold, they are determined to belong to the same continuous risk event, and the duration is accumulated, thereby improving the accuracy of continuous risk behavior analysis in complex scenarios and generating duration data of typical risk behaviors of corresponding personnel.

[0063] The identified risk behaviors, such as falls, boundary crossings, climbing, and abnormal stillness, are analyzed for their degree of danger. Risk levels are categorized based on movement characteristics, the location of the behavior, and the duration of the movement. Basic risk level parameters are set according to the hazard attributes of different behaviors; for example, the basic risk value for falls is set at 0.9, for boundary crossings at 0.8, for climbing at 0.7, and for abnormal stillness at 0.6. The risk level is dynamically adjusted based on the location of the behavior. When a fall occurs in the operating area of ​​transportation equipment or near high-risk equipment, the corresponding risk level is increased; when a boundary crossing enters a high-risk electronic fence area, the danger level is further increased. A joint analysis of changes in human movement states is performed. The severity of risk is further refined by calculating the magnitude of the body's center of gravity shift, the rate of change of joint angles, and the trend of changes in body tilt angles. When the rate of descent of the body's center of gravity exceeds 25% of the body's height per second, it is considered a high-risk fall; when the body tilt angle during climbing is consistently greater than 70°, a significant risk of imbalance is identified. To improve the stability of risk analysis and to make a comprehensive judgment based on the duration of behavior, the corresponding risk level is increased as the duration of high-risk behavior increases. The risk level is quantified using a normalized method between 0 and 1, where 0 represents a low-risk state and 1 represents an extremely high-risk state, thus obtaining the risk level data for the corresponding behavior type.

[0064] A joint quantitative analysis of the duration and severity of risky behaviors is conducted, generating a behavioral risk index through weighted averaging. The duration of risky behaviors is time-normalized, converting the durations of different types of risky behaviors into a unified risk scale. When a fall lasts longer than 5 seconds, the time risk weight is increased; when an out-of-bounds behavior persists in a dangerous area for more than 10 seconds, the risk is considered to be rapidly increasing. The risk severity parameter and the behavior duration parameter are weighted and fused, with the risk severity weight preferably set to 0.6 and the behavior duration weight set to 0.4 to ensure a balance between the dangerous action itself and the duration of the danger. The risk index growth rate is increased for dangerous behaviors in densely populated areas. When a fall occurs in a densely populated area, the behavioral risk index is increased to avoid triggering collateral safety risks. To reduce risk fluctuations caused by instantaneous changes in action, the risk parameters within a continuous time window are dynamically averaged, with the window length preferably set to 5-10 seconds, thereby improving the stability of the risk index changes. The risk index is represented by a quantification of 0 to 100, where 0-30 is defined as the low-risk range, 30-60 as the medium-risk range, and above 60 as the high-risk range. As risky behavior continues to increase, the risk index dynamically grows over time, generating a behavioral risk index corresponding to the behavioral state.

[0065] Multi-level risk threshold rules are set based on the risk level of the underground operation area. The preferred warning threshold is 30 for low risk, 60 for medium risk, and 80 for high risk. When the behavioral risk index reaches the corresponding threshold, a corresponding level of dangerous behavior warning signal is triggered. The trend of risk index changes is dynamically analyzed; when the risk index increases rapidly and continuously within a short period, the warning response level is increased. When the risk index increases by more than 20% within 5 seconds, an enhanced warning mechanism is triggered in advance. Differentiated warning rules are set for different types of risky behaviors. For example, falls trigger immediate warnings, boundary crossings trigger area boundary warnings, and static abnormal behaviors are confirmed with a delay based on duration. To reduce the probability of false alarms, dangerous behaviors are continuously confirmed. A dangerous behavior warning signal is only officially generated when the risk status continues for more than a preset confirmation time window, preferably set to 2-5 seconds, to reduce false triggering caused by short-term erroneous actions. The same dangerous behaviors that occur repeatedly within a short period are cumulatively analyzed. When the same person exhibits dangerous behaviors multiple times consecutively, the warning level is automatically increased, and a corresponding level of dangerous behavior warning signal is generated.

[0066] Once a dangerous behavior warning signal is generated, the underground audible and visual alarm device, broadcasting device, and dispatch terminal are linked and controlled according to the risk level and spatial location of the dangerous behavior. When the risk index is in the low-risk range, a yellow warning light in the area is triggered and a short voice reminder is played; when the risk index enters the medium-risk range, a red audible and visual alarm is activated and a safety evacuation notice is broadcast to nearby workers; when the risk index reaches the high-risk range, real-time risk warning information is simultaneously pushed to the dispatch terminal, and a high-priority continuous alarm is triggered. The broadcast content is dynamically matched according to the type of dangerous behavior. For example, for a fall, the broadcast is "A person has fallen; please immediately check the situation on site"; for a boundary crossing, the broadcast is "Danger area intrusion; please evacuate immediately." Video images before and after the dangerous behavior are automatically traced and captured. Taking the trigger time of the dangerous behavior as the center, a 10-second video clip is captured forward and a 20-second video clip is captured backward, and the clips are archived and stored according to the personnel identification number, risk behavior type, and timestamp. At the same time, the corresponding key human body node data, risk index change curve, and behavior type classification results are saved to form a complete record of dangerous behavior events.

[0067] In this embodiment, a visual detection-based underground personnel risk identification and early warning system for coal mines is provided, used to execute the visual detection-based underground personnel risk identification and early warning method for coal mines as described above, including: The image acquisition module is used to continuously acquire images of the underground working area based on a vision camera in the underground roadway of the coal mine, and to construct a standardized image sequence. The target detection module is used to perform frame-by-frame depth target detection and human pose estimation on standardized image sequences, and generate a dataset of key node coordinates. The continuous tracking module is used to perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory. The graph neural convolution module is used to perform graph neural convolution propagation calculations on the trajectory of changes in the skeletal structure and extract the evolutionary features of human actions. The behavior analysis module is used to analyze the trends of movement changes and limb movement patterns of personnel's action evolution characteristics, and to mark typical risky behaviors of personnel. The early warning module is used for comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

[0068] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0069] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. The apparatus / device embodiments described above are merely illustrative, and the division of modules or units is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0070] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0071] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0072] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for risk identification and early warning of underground personnel in coal mines based on visual detection, characterized in that, Includes the following steps: Step S1: Continuously acquire images of the underground working area using visual cameras in the coal mine roadways, and construct a standardized image sequence; Step S2: Perform frame-by-frame depth target detection and human pose estimation on the standardized image sequence to generate a key node coordinate dataset; Step S3: Perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory; Step S4: Perform graph neural convolutional propagation calculation on the trajectory of skeletal structure changes to extract the evolutionary features of human movements; Step S5: Analyze the trends of movement changes and limb movement patterns of personnel's movement evolution characteristics, and mark typical risk behaviors of personnel; Step S6: Conduct comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

2. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 1, characterized in that, The specific steps of step S1 are as follows: Based on the visual camera in the underground roadway of the coal mine, continuous image acquisition of the underground work area is carried out to obtain the image stream of underground personnel working; Calculate brightness distribution parameters based on the image stream of personnel working underground; Based on the brightness distribution parameters, adaptive exposure compensation and local brightness equalization are performed to generate a brightness-optimized image stream. Dust blur feature recognition is performed based on brightness-optimized image stream, and visual concentration estimation is performed to generate coal dust concentration features; Based on the characteristics of coal dust concentration, the brightness-optimized image stream is dehazed and high-frequency noise is suppressed to construct a standardized image sequence.

3. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 1, characterized in that, The specific steps of step S2 are as follows: Perform frame-by-frame depth target detection on standardized image sequences and mark all personnel working within the area; A unique identifier ID is assigned to all workers in the area; based on the unique identifier ID, independent image frame segmentation is performed to obtain multiple personnel image frames; Perform human pose estimation on multiple person image frames and extract key human structure nodes for each person; Calculate the image pixel coordinates of the key structural nodes of the human body to generate a key node coordinate dataset.

4. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 3, characterized in that, The key structural nodes of the human body include at least the head, shoulders, elbows, knees, and ankles.

5. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 1, characterized in that, Step S3 is as follows: Based on the connection relationships of the human skeleton, the key node coordinate dataset is topologically connected to construct a skeleton topology graph. Multi-frame continuous tracking analysis of the skeleton topology map is performed based on standardized image sequences to extract the position change data of the skeleton topology map in adjacent time frames; By fitting the temporal variation data of the skeleton topology map position change data, the trajectory of the skeleton structure change is constructed.

6. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 1, characterized in that, The specific steps of step S4 are as follows: Graph neural convolutional propagation calculations are performed on the trajectory of changes in the skeletal structure to extract local pose features in the spatial dimension; The local pose features are extracted layer by layer according to hierarchical dimensions to obtain multi-level behavioral association features; The multi-level behavioral association features are input into a preset spatiotemporal graph convolutional behavior recognition model to perform temporal convolutional dynamic modeling and extract the evolution features of human actions.

7. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 1, characterized in that, The specific steps of step S5 are as follows: The evolutionary characteristics of human movements are analyzed to determine the trends in movement changes and limb movement patterns, resulting in a classification of movement types. The risk behavior is judged based on the action type classification results, and typical risk behaviors of personnel are marked.

8. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 7, characterized in that, Typical risk behaviors of the personnel mentioned include falling, crossing boundaries, climbing, and abnormal stillness.

9. The method for risk identification and early warning of underground personnel in coal mines based on visual detection according to claim 1, characterized in that, The specific steps of step S6 are as follows: Calculate the duration of the typical risk behaviors of the individuals; The risk level of typical risk behaviors of the personnel is assessed to obtain the risk level of each behavior type. A comprehensive risk quantification is performed on the duration and the degree of risk to generate a behavioral risk index; Based on the preset risk assessment gradient rules, the behavioral risk index is compared and analyzed to adaptively generate early warning signals for dangerous behaviors; Dynamic early warning processing is performed based on dangerous behavior warning signals, and source image fragments are extracted and stored; the dynamic early warning processing includes hierarchical linkage control of downhole audible and visual alarm devices, broadcasting devices and dispatch terminals to perform adaptive directional early warning.

10. A visual detection-based underground personnel risk identification and early warning system for coal mines, applied to the visual detection-based underground personnel risk identification and early warning method for coal mines as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to continuously acquire images of the underground working area based on a vision camera in the underground roadway of the coal mine, and to construct a standardized image sequence. The target detection module is used to perform frame-by-frame depth target detection and human pose estimation on standardized image sequences, and generate a dataset of key node coordinates. The continuous tracking module is used to perform skeleton topology connection and multi-frame continuous tracking analysis on the key node coordinate dataset to construct the skeleton structure change trajectory. The graph neural convolution module is used to perform graph neural convolution propagation calculations on the trajectory of changes in the skeletal structure and extract the evolutionary features of human actions. The behavior analysis module is used to analyze the trends of movement changes and limb movement patterns of personnel's action evolution characteristics, and to mark typical risky behaviors of personnel. The early warning module is used for comprehensive risk quantification and adaptive targeted early warning based on typical risk behaviors of personnel.

Images