Underground safety posture recognition and early warning method based on adaptive cascade skeleton compensation

Abstract

The invention belongs to the technical field of underground safety intelligent monitoring and early warning, and particularly relates to an underground safety posture recognition and early warning method based on self-adaptive cascade skeleton compensation, which comprises the following steps: S1, acquiring real-time video stream data of a coal mine underground hydraulic support operation area; s2, extracting key point coordinates and confidence of the skeleton frame by frame by using a lightweight attitude estimation network deployed at an edge computing terminal; s3, for each frame, counting the number of continuous missing frames from each key point to the current frame, and calculating the global missing key point proportion; s4, performing branch compensation on the current frame based on the diagnosis result of S3; s5, calling a preset operation safety risk index model, outputting a real-time risk score of the current frame, and identifying whether a violation behavior violating the violation posture rule exists or not; and S6, triggering a grading early warning mechanism. According to the method, the real-time performance of the edge terminal can be guaranteed, and meanwhile high-robustness recognition and accurate safety evaluation of the posture of the miner in the operation area of the hydraulic support are achieved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring and early warning technology for downhole safety, specifically relating to a method for downhole safety posture recognition and early warning based on adaptive cascaded skeleton compensation. Background Technology

[0002] In modern fully mechanized coal mining faces, hydraulic supports are core equipment for ensuring roof stability and operational safety, and their operation directly impacts the lives of on-site workers. To prevent mechanical injuries, pinching injuries, or even burial accidents caused by improper operation or dangerous postures, real-time and accurate safety monitoring of miners' operating postures in the support area is urgently needed. Traditional methods mainly rely on manual video monitoring, which not only suffers from problems such as delayed response and significant subjective judgment bias, but also easily leads to visual fatigue due to long-term monitoring, making it difficult to meet the all-weather, high-reliability safety supervision requirements of high-risk environments.

[0003] In recent years, with the development of computer vision and edge intelligence technologies, posture recognition methods based on human skeleton key point detection have been gradually introduced into underground safety monitoring scenarios. These methods construct skeleton models by extracting the coordinates of human joints, and then analyze whether operational behaviors comply with safety regulations. However, the underground environment in coal mines is extremely complex: during coal cutting by mining machines and the movement of hydraulic supports, there is often high concentration of coal dust, water mist from dust suppression sprays, alternating light intensity, and dynamic, large-area occlusion of workers by large equipment structural components. These factors severely interfere with visual perception systems, leading to a significant increase in the key point loss rate of mainstream posture estimation algorithms in actual deployment. The obtained skeleton sequences are often incomplete and temporally fragmented, making it difficult to support subsequent accurate risk assessment.

[0004] To address the issue of missing skeletons, existing repair technologies mainly fall into two categories: First, methods based on kinematic models, such as linear interpolation or Kalman filtering, while computationally lightweight and suitable for edge deployment, assume smooth and continuous human movement. This fails to effectively model the sudden, nonlinear, and large-amplitude movements that miners may make when operating the support (such as bending over, leaning forward, or climbing in a step, which are typical violations). This results in the compensation trajectory deviating significantly from the actual posture, leading to missed detection of safety hazards. Second, end-to-end skeleton reconstruction using deep generative models, while possessing strong recovery capabilities in occluded scenarios, suffers from a large number of model parameters and high inference latency, making it difficult to achieve real-time operation on intrinsically safe edge computing terminals underground, thus limiting its feasibility for engineering implementation.

[0005] The root cause of the aforementioned problems lies in the highly dynamic and unpredictable nature of underground occlusion, while safety monitoring demands both high precision and low latency. On the one hand, relying solely on lightweight kinematic models cannot capture the nonlinear characteristics of complex violations; on the other hand, high-fidelity depth generation models struggle to adapt to edge resource constraints. Therefore, achieving high-fidelity, low-latency, and adaptive skeleton compensation under severe occlusion conditions, and building a real-time risk quantification and early warning mechanism that complies with coal mine safety regulations, has become a key bottleneck for current technology implementation.

[0006] Therefore, how to achieve robust identification and accurate safety assessment of miners' postures in the hydraulic support operating area while ensuring the real-time performance of edge terminals has become an urgent problem to be solved. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, this invention provides a method for downhole safety posture recognition and early warning based on adaptive cascaded skeleton compensation. This method can achieve robust recognition and accurate safety assessment of miners' postures in the hydraulic support operating area while ensuring the real-time performance of edge terminals.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation includes the following steps:

[0010] S1. Obtain real-time video stream data of the hydraulic support operation area in underground coal mines;

[0011] S2. Based on the video stream data of S1, a lightweight pose estimation network deployed on the edge computing terminal is used to extract the key point coordinates and confidence scores of the miner's human skeleton frame by frame to form the original skeleton sequence.

[0012] S3. For each frame in the original skeleton sequence of S2, perform missing point diagnosis: determine whether the confidence of each key point in the frame is lower than the preset threshold, and count the number of consecutive missing frames for each key point up to the current frame, and calculate the global missing key point ratio of the frame.

[0013] S4. Based on the diagnostic results of S3, perform the following branch compensation logic on the current frame:

[0014] (a) If the number of consecutive missing frames of all key points in the current frame does not exceed the first threshold, and the proportion of missing key points in the frame does not exceed the second threshold, then the first level of compensation is initiated: an adaptive filtering algorithm based on the state space model is used to perform kinematic prediction and compensation for the missing key points in the current frame.

[0015] (b) Otherwise, initiate the second level of compensation, executing a coarse-to-fine joint guided refactoring process, including:

[0016] The first-level compensation is used to perform preliminary prediction and filling of the missing regions in the current frame, generating a coarse trajectory feature matrix for the current frame. Through the coordinate encoding layer, the coordinate features of key points that are determined to be visible by S3 in the current frame are obtained. MaskToken, corresponding to missing key points, and based on Generated 3D spatiotemporal location code The input representation is then spliced ​​and merged to form an enhanced input representation.

[0017] The enhanced input representation is fed into a mask autoencoder based on a spatiotemporal tubular mask. The encoder models the anatomical topology and temporal evolution of human joints, and the decoder performs manifold reconstruction to generate the complete skeleton of the current frame. The spatiotemporal tubular mask is configured to simulate a continuous occlusion scene in a well.

[0018] S5. Based on the complete skeleton of the current frame obtained after compensation by S4, the preset Operation Safety Risk Index (OSRI) model is called. This model quantifies the safety of the current operation posture according to the dangerous areas and violation posture rules defined in the coal mine safety regulations, outputs the real-time risk score of the current frame, and identifies whether there is any violation of the violation posture rules.

[0019] S6. Based on the real-time risk score output by S5 and whether the violation has been identified, trigger the graded early warning mechanism and execute the corresponding safety response measures.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. A closed-loop architecture of "diagnosis-tiered compensation-safety assessment" is constructed to achieve intelligent response to complex occlusion scenarios. This method uses S3 to diagnose missing frames in each frame, not only determining whether individual keypoints are missing, but also calculating the number of consecutive missing frames and the global missing ratio, thus providing multi-dimensional decision-making basis for subsequent compensation strategies. This design breaks through the coarse-grained processing method of "direct interpolation or skipping" in traditional pose estimation, realizing a transformation from "passive completion" to "active perception". Compared with existing technologies that rely solely on confidence thresholds to trigger compensation, this scheme can more accurately distinguish between instantaneous noise interference and persistent occlusion, avoiding frequent activation of computationally expensive deep models under slight perturbations.

[0022] 2. S4(a) First-level compensation achieves efficient and stable kinematic prediction and preliminary repair. When a minor missing keypoint is detected, an adaptive filtering algorithm (such as Kalman filtering) based on a state-space model is initiated for kinematic prediction. This algorithm uses historical trajectory information to smoothly predict missing keypoints in the current frame, effectively suppressing skeleton jitter or breakage caused by brief occlusion. Its advantages are: ① High computational efficiency, suitable for deployment on edge terminals; ② Uncertainty perception capability, dynamically adjusting prediction weights based on confidence level; ③ More importantly, its output is used as a coarse trajectory prior for second-level compensation, achieving "one compensation, dual purpose"—completing preliminary repair and providing structural guidance for deep reconstruction. This significantly differs from the traditional method of separating primary prediction and advanced reconstruction, improving the overall process's synergy and robustness.

[0023] 3. S4(b) innovatively adopts a "coarse-to-fine" joint guided reconstruction strategy for the second-level compensation. For severe or persistent occlusion scenes, the second-level compensation process is initiated. Its core is to construct an enhanced input representation that integrates three types of information: embedded features of visible keypoints. ; Mask Token corresponding to missing key points A coarse trajectory matrix generated based on the first-level compensation. Extracted three-dimensional spatiotemporal location code This "stitching and fusion" mechanism enables the mask autoencoder (MAE) to not only perceive locally visible information during the decoding stage, but also to perform inference using global motion trends and spatial structure. Compared to the traditional method of directly feeding the original missing data into the MAE, this design greatly enhances the model's contextual understanding of long-term occlusion (such as coal dust cover and equipment obstruction), making the reconstruction results more consistent with human anatomical topology and temporal evolution. Furthermore, the spatiotemporal tubular mask used is specifically configured to simulate continuous occlusion scenarios underground, further strengthening the model's adaptability to real industrial environments and avoiding the problem that general random masks cannot reflect actual occlusion patterns.

[0024] 4. OSRI Risk Model and Tiered Early Warning Mechanism: Achieving a Closed Loop from Attitude Recognition to Safety Decision-Making. Based on the complete skeleton after S4 compensation, the Operational Safety Risk Index (OSRI) model is invoked, combined with hazardous areas (such as push rods and scraper conveyors) and violation behavior rules (such as unsupported roof work and unauthorized standing positions) in coal mine safety regulations, to quantitatively assess miners' behavior and output real-time risk scores. This model not only identifies the existence of violations but also triggers different levels of early warning responses based on risk levels, forming an integrated "perception-assessment-response" safety protection system. Compared to a simple alarm system, this solution achieves a leap from qualitative judgment to quantitative assessment, improving the accuracy and interpretability of early warnings.

[0025] In summary, this method can achieve robust identification and accurate safety assessment of miners' postures in the hydraulic support operating area while ensuring the real-time performance of edge terminals. By constructing a complete technology chain of "diagnosis-driven, graded compensation, joint guidance, and risk quantification," it solves the core problems of posture estimation failure and difficulty in implementing safety supervision in underground high-dust and strong obstruction environments, providing an engineering-deployable technical path for intelligent safety monitoring in coal mines.

[0026] Preferably, in S3, the preset threshold is 0.4; the statistics of the number of consecutive missing frames are based on a sliding time window.

[0027] This setting, 1. sets the key point confidence threshold to 0.4, which can effectively distinguish between "weakly visible but usable" and "actually lost" key points under interference conditions such as low light and high occlusion in the well. It avoids incorrectly retaining noisy points due to an overly lenient threshold, or over-triggering the compensation mechanism due to an overly strict threshold, thereby improving the accuracy of skeleton data quality judgment.

[0028] 2. By combining a fixed confidence threshold with continuous missing statistics based on a sliding window, the missing diagnosis has both frame-level sensitivity and temporal context awareness, thus more accurately distinguishing between mild anomalies and severe failure scenarios, ensuring that the first and second level compensations are activated under appropriate conditions, and improving the overall system's adaptability and robustness.

[0029] Preferably, in S4(a), the adaptive filtering algorithm is a Kalman filter, and its process noise covariance is dynamically adjusted according to the confidence level of the key points in the previous frame.

[0030] This setup balances computational efficiency with compensation accuracy. While maintaining the low computational overhead advantage of Kalman filtering, it introduces perceived uncertainty information through a simple covariance adjustment mechanism. This allows for more accurate attitude compensation on edge devices than fixed-parameter filtering without adding complex networks or extra inference steps, effectively supporting the reliability of subsequent safety assessments.

[0031] Preferably, in S4(a), the implementation of the adaptive filtering algorithm includes:

[0032] Constructing the state vector of key points ,in The coordinates of the k-th keypoint in the current frame. Its velocity component;

[0033] Set the state prediction equation:

[0034] ;

[0035] In the formula, Here is the state transition matrix. The process noise covariance matrix;

[0036] Among them, process noise covariance Based on the confidence level of key points in the previous frame Dynamic adjustment: Confidence level when key points are detected Less than the confidence threshold At that time, using the predicted value As a compensation result, and to increase the process noise covariance Q at the next time step. k+1 This is to reflect the accumulation of forecast uncertainty.

[0037] With this setup, 1. by fusing position and velocity information to construct a state vector, the model can reasonably predict the missing key points of the current frame based on historical motion trends, effectively alleviating the problem of skeleton jitter or breakage caused by instantaneous occlusion, and significantly enhancing the smoothness of the output sequence.

[0038] 2. Implement dynamic adaptive adjustment of the compensation strategy. When the confidence level of key points is low, automatically switch to prediction mode and increase Q. k+1 Reflecting the uncertainty of future predictions, the system is more inclined to accept new observation data in subsequent frames, avoiding the accumulation of errors caused by "over-reliance on predictions".

[0039] 3. Enhanced robustness to complex occlusion scenarios. This mechanism is particularly suitable for brief but frequent occlusion events common in underground environments (such as coal dust drifting or equipment shaking). It can maintain trajectory continuity without observation input while retaining sensitivity to future observations, thus balancing stability and responsiveness.

[0040] Preferably, in S4(b), the key point coordinate features determined to be visible in the current frame by S3 are... MaskToken, corresponding to missing key points, and based on Generated 3D spatiotemporal location code The expression for splicing and merging is:

[0041] ;

[0042] In the formula, ⊕ represents the feature splicing operation.

[0043] Preferably, in S4(b), the spatiotemporal tubular mask is constructed during the training phase as an occlusion mask matrix that moves continuously along the time axis. ∈ Where T represents the number of time frames and J represents the number of key points of the human skeleton, used to simulate persistent regional occlusion caused by coal dust clumps or equipment structural components on the input skeleton sequence tensor.

[0044] This setup improves the model's generalization ability to persistent occlusion. By introducing a mask that moves continuously along the time axis during the training phase, the mask autoencoder can learn motion consistency and structural recovery capabilities under continuous occlusion across frames, thus enabling stable reconstruction of the skeleton even when faced with coal dust or prolonged occlusion by the robotic arm in a real-world underground environment.

[0045] 2. Enhance the model's accuracy in compensating for long-term missing local keypoints. Because the mask is continuous in the time dimension, the model is forced to rely on motion trend information (such as speed and direction) from historical frames for prediction. This prompts the cross-attention mechanism to make more effective use of cue features, significantly improving the accuracy of compensating for the continuous loss of a single keypoint across multiple frames.

[0046] 3. Achieve a high degree of simulation of real industrial scenarios. The design of the mask matrix directly corresponds to the typical occlusion patterns in underground coal mines—non-instantaneous, regional, and continuous occlusion (such as coal dust coverage or equipment obstruction), making the training process closer to actual application scenarios and improving the practicality and reliability of the model after deployment.

[0047] Preferably, the mask autoencoder is optimized during training using the following joint loss function:

[0048] ;

[0049] In the formula, The mean square error between the reconstructed coordinates and the true coordinates is used to measure the accuracy of the position reconstruction. The bone length consistency loss is used to constrain the rate of change of limb length after reconstruction to conform to the rigid structure of the human body. This is a temporal smoothing loss used to ensure the continuity of the motion trajectory; , , These are the weighting coefficients.

[0050] This approach, by designing a joint loss function that integrates spatial structural constraints and temporal dynamic constraints, deeply incorporates prior physical knowledge into the neural network training process. Its core value lies not only in pursuing pixel-level reconstruction accuracy but also in emphasizing the "interpretability" and "behavioral rationality" of the generated results, achieving a leap from "being able to complete" to "completing a likeness." This hybrid training paradigm of "data-driven + physically guided" is one of the key supports for achieving highly robust attitude estimation in the field of industrial safety monitoring.

[0051] Preferably, in S5, the dangerous area is a virtual boundary defined in three-dimensional space according to the coal mine safety regulations; the violation includes spatial interference between key points of the human skeleton and the virtual boundary, and / or posture violation behavior that violates the rule of the violation posture.

[0052] This setup achieves: 1. Precise safety risk assessment driven by regulations. It transforms the abstract safety requirements in coal mine safety regulations into calculable virtual boundaries in three-dimensional space, and combines this with real-time spatial interference detection of key points in the human skeleton. This upgrades risk identification from experience-based judgment to a technically sound, quantifiable, and traceable assessment, significantly improving the objectivity and compliance of violation identification.

[0053] 2. Supports unified assessment of multi-dimensional violations. By simultaneously detecting "spatial intrusion" (such as the body entering the danger zone of the support movement) and "postural violations" (such as bending over to operate, holding the support with one hand, and other non-standard actions), a safety assessment system covering both position and posture dimensions is constructed. Compared with traditional methods that only focus on a single behavioral characteristic, it can more comprehensively reflect the complex risks in actual operations.

[0054] 3. Enhance the interpretability and on-site guidance value of early warnings. Based on clear virtual boundaries and rule logic, output violation types (such as "left shoulder intrusion into the displacement zone" or "excessive spinal flexion"), which not only facilitates hierarchical alarms in the system, but also provides a clear basis for subsequent personnel training, accident retrospection, and process optimization, thereby improving the closed-loop capability and intelligence level of safety management.

[0055] This design deeply integrates industry safety regulations into the algorithm logic, making risk assessment both technically rigorous and engineering-practical, effectively supporting proactive and refined safety control in downhole operations.

[0056] Preferably, in S5, the operational safety risk index model is a calculation model based on multiple risk assessment factors, including: the spatial intrusion distance between key points of the human skeleton and the dangerous area, the posture violation score corresponding to the currently triggered posture violation, and the duration of the currently triggered posture violation.

[0057] This setup, by introducing risk assessment factors across multiple dimensions such as spatial intrusion distance, violation level, and duration, allows the OSRI model to transcend the binary judgment of "whether a violation has occurred" and to measure risk continuously and hierarchically. For example, slight proximity to a danger zone versus deep intrusion, and momentary violation versus prolonged stay will correspond to different risk scores, making the assessment results more closely reflect the actual level of security threat.

[0058] 2. The selected factors are all derived from the core concerns in the coal mine safety regulations (such as safety distance, standardization of actions, and exposure time). Their combination method conforms to the risk perception habits of on-site safety management personnel, making the algorithm output not only technically reasonable, but also having good interpretability and management adaptability, which is easy to integrate into the existing safety management system.

[0059] Preferably, in S5, the operational safety risk index model calculates the real-time risk score of the current frame using the following formula:

[0060] ;

[0061] In the formula, This is the set of key point coordinates of the human skeleton in the current frame after S4 compensation; A three-dimensional virtual electronic fence defined according to coal mine safety regulations; This is the distance risk function, used to calculate... Key points and The minimum Euclidean distance increases exponentially when the distance is less than the safety threshold. This is the posture violation function, used to quantify the severity of the current posture violation. Its output value is the maximum value among all the posture violation scores corresponding to the triggered posture violations in the preset coal mine typical violation posture feature mapping table MVPMT. The duration factor of the triggered posture violation is used to reflect the cumulative effect of risk. , , These are the weighting coefficients.

[0062] This setup enables: 1. Accurate identification and real-time alerts for intrusions into high-risk areas. By establishing a three-dimensional virtual electronic fence and calculating the Euclidean distance between key points of the skeleton and the fence, it can accurately detect whether a miner's body part has entered the equipment's movement trajectory or a dangerous blind spot, significantly improving the early warning capability for "entering a dangerous area" type of accidents.

[0063] 2. Effectively identify typical violation patterns and enhance the intelligence level of behavior supervision. For posture violations, it achieves automatic identification and continuous monitoring of non-contact violations, reducing blind spots in human inspections.

[0064] 3. Provides interpretable and adjustable real-time risk quantification indicators. The OSRI model integrates multiple risk factors through weighted coefficients to generate a continuously changing risk score. It not only supports system-level alarm threshold settings but can also be used for subsequent trend analysis and risk tracing, demonstrating good engineering practicality and scalability.

[0065] This technical solution deeply integrates physical space constraints, human posture semantics, and temporal behavioral characteristics to construct a closed-loop safety assessment mechanism for special underground coal mine operation scenarios. Its core value lies in breaking away from the traditional passive "rules + manual" supervision model and establishing an integrated proactive intelligent protection system encompassing "perception—identification—quantification—early warning." This methodology of "data-driven risk assessment and model-quantified safety assessment" is one of the key breakthroughs of this invention in achieving intelligent upgrades in the field of industrial safety.

[0066] Preferably, the process for determining whether the human posture violates the rule of violation includes:

[0067] Based on the current frame's 3D skeleton sequence obtained after S4 compensation. Real-time calculation of joint angles at key locations The skeleton is represented by multiple key points, each bone segment is defined by a line connecting two adjacent key points, and each joint consists of three consecutive key points. The corresponding two bone vectors are used to calculate the joint angle. The key region is a human anatomical joint area characterized by at least three consecutive key points. The joint angle... Calculated using the dot product formula of adjacent bone vectors:

[0068] ;

[0069] In the formula, and These are the vectors of two adjacent bones that make up the joint;

[0070] Traverse the preset coal mine typical violation posture feature mapping table MVPMT; the MVPMT table stores multiple predefined posture violation behavior rules in a multi-row format, each row corresponds to a specific posture violation behavior, and includes at least one of the angle constraint field, spatial position constraint field and time accumulation constraint field;

[0071] For each row, if an angle constraint field is set in the row, check whether the current joint angle meets the conditions defined in the field; if a spatial position constraint field is set, check whether the relevant key points are located in the preset danger zone; if a time accumulation constraint field is set, check whether the current violation status has continued to reach the duration defined in the field.

[0072] When all the set constraint fields in a row are satisfied, it is determined that the corresponding attitude violation has occurred, and the attitude violation score associated with that row is obtained.

[0073] Finally, the attitude violation function The output value is the maximum value among all attitude violation scores identified as occurring; if no attitude violation is triggered, then... =0;

[0074] The attitude violation function The output value is used by the operational safety risk index model to calculate the real-time risk score.

[0075] This setup enables: 1. Multi-dimensional and structured detection of complex and unsafe postures. By introducing joint angle calculation and the MVPMT rule table, this method can not only identify single actions (such as bending over), but also comprehensively judge three types of factors: abnormal angles, spatial boundary violations, and time accumulation, thereby accurately capturing complex violations such as "operating without overhead contact" and "illegal positioning." Compared with traditional methods that rely solely on simple spatial distance or threshold judgments, this solution has stronger semantic expression capabilities and misjudgment suppression capabilities.

[0076] 2. Enhance the interpretability and maintainability of violation identification. The MVPMT table stores rules in a structured table format, with each row corresponding to a violation and its constraints. This facilitates dynamic updates and configurations by safety managers based on coal mine regulations. For example, when adding a "two feet off the ground" risk, only a new row needs to be added and the corresponding angle and time constraints set, without modifying the model code, significantly improving the system's flexibility and usability.

[0077] 3. Supports high-precision, low-latency attitude violation assessment. Joint angle calculation is based on the completed skeleton, avoiding misjudgments caused by missing key points; at the same time, the rule matching process is a pure numerical comparison, with low computational overhead, and can run in real time at the edge. This design ensures that the OSRI model can maintain high accuracy while still meeting the response speed requirements of downhole scenarios.

[0078] This method deeply integrates human anatomical features with safety procedures to construct an automated violation detection mechanism based on geometric constraints and rule-based reasoning. Its core value lies in: not only achieving precise quantitative analysis of miners' operating postures, but also establishing a closed-loop transformation path of "safety specifications - mathematical expression - algorithm execution," enabling safety supervision to shift from "post-event accountability" to "pre-event warning."

[0079] Preferably, the key areas include, but are not limited to: the neck, shoulder, elbow, wrist, torso, hip, knee, ankle, and pelvis;

[0080] The MVPMT table includes, but is not limited to, the following posture violations and their corresponding judgment rules and posture violation scores:

[0081] Unauthorized work on an open ceiling: The angle constraint field is the neck flexion angle calculated based on the head key points, neck key points, and trunk key points. > 45°, the spatial position constraint field is that the head key point is located outside the boundary of the front beam support range, the time accumulation constraint field is that the duration T > 3s, and the associated posture violation score is 10;

[0082] Violation of posture rules: Angle constraint field not set, spatial position constraint field is that the center of the two foot key points is not located within the safety footing area specified by the support base, time accumulation constraint field is that the duration T>1s, and the associated posture violation score is 8;

[0083] Leaning into the chute: The angle constraint field includes the trunk flexion angle calculated based on the shoulder and hip keypoints. >60°, and the upper arm elevation angle calculated based on shoulder key points, elbow key points, and wrist key points. > 30°, the spatial position constraint field is that the head key point or hand key point is located in the danger zone directly above the scraper conveyor chute, the time accumulation constraint field is that the duration T > 0.5s, and the associated posture violation score is 10;

[0084] Across running equipment: The angle constraint field is the sum of the hip abduction angles of both legs and the knee flexion angle, calculated based on the hip keypoint, knee keypoint, and ankle keypoint. > 60°, the spatial position constraint field is that the pelvic key point is located above the running scraper or transfer machine, the time accumulation constraint field is not set, and the associated posture violation score is 10;

[0085] Improper operating posture: The angle constraint field is the trunk lateral bending / torsion angle calculated based on the key points of both shoulders and both hips. The angle is >30°, the spatial position constraint field is that the hand key point is located in the functional area of ​​the operating handle, the time accumulation constraint field is that the duration T > 5s, and the associated posture violation score is 5.

[0086] This setup enables: 1. Precise modeling and automatic identification of complex violations. By setting multi-dimensional constraints (angle + position + time) for each violation, the solution can accurately distinguish between easily confused scenarios such as "brief probing" and "continuous overhead movement," and "accidental approach" and "long-term straddling." For example, "leaning into a chute" requires not only a torso flexion angle greater than 60°, but also that the head or hands are in the danger zone for more than 0.5 seconds, significantly reducing the false alarm rate and improving identification accuracy.

[0087] 2. Provides interpretable and quantifiable risk assessment criteria. Each rule is associated with specific physical parameters and time thresholds, giving the system's output "attitude violation score" clear physical meaning and safety semantics, supporting subsequent risk tracing and liability determination. Compared to traditional binary judgments that only output "whether a violation occurred," this solution represents a leap from "black box detection" to "transparent assessment."

[0088] This method constructs a structured violation rule base (MVPMT) for underground coal mine operations, transforming abstract safety regulations into computable and executable geometric and spatiotemporal constraints. This enables precise, interpretable, and configurable automated identification of miners' operational postures. Its core value lies in establishing a closed-loop path from "safety regulations—mathematical expression—algorithm execution," truly enabling the intelligent monitoring system to "understand regulations, make judgments, and provide early warnings."

[0089] Preferably, in S6, the graded early warning mechanism is divided into three levels based on the real-time risk score OSRI(t), and the following security response measures are executed accordingly:

[0090] 1) When OSRI(t) exceeds the first threshold but does not reach the second threshold, it is judged as a mild risk. A low-frequency, intermittent vibration signal is output through the tactile feedback unit integrated in the miner's safety equipment to convey an abnormal posture prompt to the miner.

[0091] 2) When OSRI(t) reaches or exceeds the second threshold but does not reach the third threshold, it is determined to be a medium risk. The tactile feedback unit outputs a high-frequency vibration signal and simultaneously controls the smart miner lamp to enter the flashing mode to warn the miner to immediately correct his posture or leave the danger zone.

[0092] 3) When OSRI(t) reaches or exceeds the third threshold, it is judged as a serious risk. The edge computing device sends the highest priority emergency interlock command to the hydraulic support controller, causing the controller to cut off the power supply to the pilot magnet of the relevant action circuit and forcibly stop the dangerous action. The dangerous action includes pushing, lowering the column and / or deploying the side guard plate.

[0093] The locking state triggered by the emergency locking command can only be released after the miner has left the danger zone, the human posture has returned to a compliant state, and the OSRI(t) has dropped below the first threshold.

[0094] The frequency range of the low-frequency vibration signal is 5–30 Hz, and the frequency range of the high-frequency vibration signal is 80–200 Hz.

[0095] This setup enables: 1. Precise risk response grading and dynamic adaptation. By dividing warnings into three levels and corresponding response measures of varying intensities, the system can adopt a strategy of "mild alert, moderate warning, and heavy intervention" based on the actual level of risk. Compared to the traditional, crude response of "alarming at the first sign of an anomaly," this solution avoids vigilance fatigue caused by frequent false alarms, thus improving the effectiveness and acceptability of warnings.

[0096] 2. Integrating a multimodal feedback mechanism enhances the safety guidance capabilities of human-machine interaction. In the mild and moderate risk stages, a dual warning system of tactile vibration and visual flashing is employed, fully utilizing the miners' sensory channels for coordinated alerts. This multi-sensory linkage design significantly improves information transmission efficiency, effectively conveying safety warnings even in the high-noise, high-light interference of the underground environment.

[0097] 3. Implement mandatory safety interlocking of critical equipment, constructing a physical-level protective barrier. When the risk reaches a severe level, the system not only issues a warning but also directly sends a highest-priority emergency interlock command to the hydraulic support controller, forcibly cutting off the power supply to the action circuit, achieving physical-level blocking of dangerous actions such as "pushing" and "lowering the column." This mechanism breaks through the limitation of traditional monitoring systems that "can only alarm but cannot control equipment," truly realizing closed-loop control from "monitoring and early warning" to "active protection."

[0098] This method establishes a safety response system of "risk stratification—response grading—closed-loop control," transforming the risk scores output by intelligent algorithms into executable and verifiable safety actions. This achieves full-chain, multi-level, and adaptive protection for miners' operational behaviors. Its core value lies not only in enhancing the system's early warning capabilities but also in constructing an integrated safety closed loop of "perception—assessment—response—intervention," enabling intelligent monitoring to truly achieve the engineering effectiveness of "prevention before the event." Attached Figure Description

[0099] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0100] Figure 1 This is a flowchart of the method;

[0101] Figure 2 This is a schematic diagram of the branch compensation process in Example 1;

[0102] Figure 3 This is a schematic diagram of the mask autoencoder in Embodiment 1;

[0103] Figure 4 This is a schematic diagram of the system architecture in Example 2;

[0104] Figure 5 This is the logic diagram for determining the safe posture in Example 2. Detailed Implementation

[0105] The following detailed explanation illustrates the specific implementation methods:

[0106] Example 1

[0107] like Figure 1As shown, this embodiment discloses a downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation, including the following steps:

[0108] S1. Obtain real-time video stream data of the hydraulic support operation area in the coal mine.

[0109] S2. Based on the video stream data from S1, a lightweight pose estimation network deployed on an edge computing terminal is used to extract the key point coordinates and their confidence scores of the miner's human skeleton frame by frame, forming the original skeleton sequence.

[0110] The attitude estimation network is a lightweight neural network whose model size and computational complexity meet the resource constraints of downhole edge computing devices.

[0111] By employing a lightweight neural network, the number of model parameters and computational overhead are significantly reduced, enabling efficient operation on intrinsically safe edge computing terminals with limited computing power, power consumption, and heat dissipation. This avoids deployment failures or insufficient frame rates caused by excessively large models, ensuring the continuity and timeliness of video stream processing. Furthermore, lightweighting does not simply sacrifice performance; rather, it involves targeted optimization of the network structure under resource constraints (such as channel pruning and depthwise separable convolutions). While maintaining basic accuracy in keypoint detection, it meets real-time requirements, providing reliable and stable raw skeleton input for subsequent missing data diagnosis, adaptive compensation, and safety risk assessment. Compared to directly deploying general-purpose large-scale pose estimation models, this lightweight design is more suitable for the actual deployment environment in coal mines, effectively avoiding system failures caused by equipment incompatibility or excessive inference latency, and enhancing the practicality and scalability of the entire safety monitoring solution in complex industrial scenarios.

[0112] S3. For each frame in the original skeleton sequence of S2, perform missing point diagnosis: determine whether the confidence of each key point in the frame is lower than the preset threshold, and count the number of consecutive missing frames for each key point up to the current frame, and calculate the global missing key point ratio of the frame.

[0113] In specific implementation, the preset threshold is 0.4; the statistics of consecutive missing frames are based on a sliding time window. Setting the key point confidence threshold to 0.4 can effectively distinguish between "weakly visible but usable" and "actually lost" key points under interference conditions such as low light and high occlusion in the well, avoiding the incorrect retention of noise points due to an overly lenient threshold, or the over-triggering of the compensation mechanism due to an overly strict threshold, thereby improving the accuracy of skeleton data quality judgment. In addition, combining the fixed confidence threshold with the continuous missing statistics based on the sliding window, the missing diagnosis has both frame-level sensitivity and temporal context awareness, thereby more accurately distinguishing between mild anomalies and severe failure scenarios, ensuring that the first and second level compensations are activated under appropriate conditions, and improving the adaptability and robustness of the overall system.

[0114] S4, such as Figure 2 As shown, based on the diagnostic results of S3, the following branch compensation logic is executed for the current frame:

[0115] (a) If the number of consecutive missing frames of all key points in the current frame does not exceed the first threshold, and the proportion of missing key points in the frame does not exceed the second threshold, then the first level of compensation is initiated: an adaptive filtering algorithm based on the state space model is used to perform kinematic prediction and compensation for the missing key points in the current frame.

[0116] (b) Otherwise, initiate the second level of compensation, executing a coarse-to-fine joint guided refactoring process, including:

[0117] The first-level compensation is used to perform preliminary prediction and filling of the missing regions in the current frame, generating a coarse trajectory feature matrix for the current frame. Through the coordinate encoding layer, the coordinate features of key points that are determined to be visible by S3 in the current frame are obtained. MaskToken, corresponding to missing key points, and based on Generated 3D spatiotemporal location code The input representation is then concatenated and merged to form an enhanced input representation; the expression for concatenation and fusion is:

[0118] ;

[0119] In the formula, ⊕ represents the feature splicing operation.

[0120] The enhanced input representation is fed into a mask autoencoder based on a spatiotemporal tubular mask. The encoder models the anatomical topology and temporal evolution of human joints, and the decoder performs manifold reconstruction to generate the complete skeleton of the current frame. The spatiotemporal tubular mask is configured to simulate a continuous occlusion scene in a well. It should be noted that in this invention, 'joint' refers to the connection points of the human skeleton, represented by specifically numbered keypoints in pose estimation.

[0121] In practice, explosion-proof high-definition cameras installed at the working face collect raw video streams of the hydraulic support operation area in real time. A lightweight pose estimation network pre-deployed on an edge computing terminal is then used to extract the miners' movements from each frame of the image. Key points to obtain the original skeleton sequence matrix Each row contains the two-dimensional coordinates (x, y) of the keypoint and the confidence score c output by the model. Subsequently, the system enters the missing feature quantification and diagnosis stage. By analyzing a frame sequence of length W within a sliding time window, for each keypoint j, its confidence distribution within the current window is calculated, and the number of consecutive missing frames is extracted. That is, the confidence level is consistently lower than the preset threshold. The number of frames; simultaneously, calculate the percentage of global spatial missing data in the current frame to be processed. It is defined as the ratio of the number of keypoints with confidence levels below the threshold to the total number of keypoints J.

[0122] The system executes adaptive cascade scheduling logic based on the above-mentioned quantification characteristics. When the diagnostic results meet... and When the system determines that it is currently in "short-term random noise interference" mode, it calls the first-level fast compensation module (i.e., performs S4(a) to initiate the first-level compensation). When the diagnostic results show... or When the system determines that it is currently in the "severe physical occlusion or long-term loss" mode, it automatically activates the second-level deep reconstruction channel (i.e., performs S4(b) to start the second-level compensation).

[0123] In specific implementation, in S4(a), the adaptive filtering algorithm is a Kalman filter, and its process noise covariance is dynamically adjusted according to the confidence level of the key points in the previous frame.

[0124] Compared to traditional fixed-parameter Kalman filtering, this method retains the low computational overhead advantage of Kalman filtering while introducing perceptual uncertainty information through a simple covariance adjustment mechanism. Without adding complex networks or extra inference steps, it can achieve more accurate attitude compensation on edge devices than fixed-parameter filtering, effectively supporting the reliability of subsequent safety assessments.

[0125] The implementation of the adaptive filtering algorithm includes:

[0126] Constructing the state vector of key points ,in The coordinates of the k-th keypoint in the current frame. Its velocity component;

[0127] Set the state prediction equation:

[0128] ;

[0129] In the formula, Here is the state transition matrix. The process noise covariance matrix;

[0130] Among them, process noise covariance Based on the confidence level of key points in the previous frame Dynamic adjustment: Confidence level when key points are detected Less than the confidence threshold At that time, using the predicted value As a compensation result, and to increase the process noise covariance Q at the next time step. k+1 This is to reflect the accumulation of forecast uncertainty.

[0131] In this way, by fusing position and velocity information to construct a state vector, the model can reasonably predict missing keypoints in the current frame based on historical motion trends, effectively mitigating skeleton jitter or breakage caused by momentary occlusion and significantly enhancing the smoothness of the output sequence. Furthermore, when the confidence level of a keypoint is low, it automatically switches to prediction mode and increases the Q-value. k+1 Reflecting the uncertainty of future predictions, the system is more inclined to accept new observation data in subsequent frames, avoiding the accumulation of errors caused by "over-reliance on predictions." Furthermore, this mechanism is particularly suitable for brief but frequent occlusion events common in downhole environments (such as coal dust drifting or equipment shaking), maintaining trajectory continuity even without observational input while preserving sensitivity to future observations, thus balancing stability and responsiveness.

[0132] In practice, the spatiotemporal tubular mask is constructed during the training phase as an occlusion mask matrix that moves continuously along the time axis. ∈ Where T represents the number of time frames and J represents the number of key points on the human skeleton, used to simulate persistent regional occlusion caused by coal dust clumps or equipment structural components on the input skeleton sequence tensor. The structure of the mask autoencoder (ST-MAE) is as follows: Figure 3 As shown.

[0133] By introducing a continuously moving mask along the time axis during the training phase, the mask autoencoder learns motion consistency and structural recovery capabilities under continuous occlusion across frames. This allows for stable skeleton reconstruction even in real-world underground environments, such as when faced with coal dust or prolonged occlusion by a robotic arm. Because the mask is continuous in the time dimension, the model is forced to rely on motion trend information (such as speed and direction) from historical frames for prediction. This enables the cross-attention mechanism to more effectively utilize cue features, significantly improving the accuracy of compensation for the continuous loss of individual keypoints across multiple frames. Furthermore, the design of the mask matrix directly corresponds to typical occlusion patterns in coal mines—non-instantaneous, regional, and continuous occlusion (such as coal dust cover or equipment obstruction)—making the training process more closely resemble real-world application scenarios and enhancing the practicality and reliability of the model after deployment.

[0134] To facilitate a better understanding of the second-level compensation by those skilled in the art, the following explanation is provided.

[0135] The second-level compensation process includes:

[0136] First, the missing regions are initially linearly filled using first-level compensation to generate a coarse trajectory feature matrix. ;

[0137] Subsequently, the visible joint coordinates are fused with the missing keypoint mask token containing coarse trajectory information using a coordinate encoding layer. The fusion formula is expressed as follows:

[0138] ;

[0139] Wherein, PE stands for three-dimensional spatiotemporal location code;

[0140] Then, the fused feature vectors are input into the mask autoencoder (ST-MAE), which captures the anatomical topological relationships between human joints and the spatiotemporal evolution of movements through a multi-layer self-attention mechanism.

[0141] The masked autoencoder (ST-MAE) is optimized during training using the following joint loss function:

[0142] ;

[0143] In the formula, The mean square error (i.e., reconstruction loss) between the reconstructed coordinates and the true coordinates is used to measure the accuracy of position reconstruction. The bone length consistency loss is used to constrain the rate of change of limb length after reconstruction to conform to the rigid structure of the human body. This is a temporal smoothing loss used to ensure the continuity of the motion trajectory; , , These are the weighting coefficients.

[0144] In this way, by designing a joint loss function that integrates spatial structural constraints and temporal dynamic constraints, prior physical knowledge is deeply integrated into the neural network training process. Its core value lies not only in pursuing pixel-level reconstruction accuracy but also in emphasizing the "interpretability" and "behavioral rationality" of the generated results, achieving a leap from "being able to complete" to "completing a likeness." This hybrid training paradigm of "data-driven + physics-guided" is one of the key supports for achieving highly robust attitude estimation in the field of industrial safety monitoring.

[0145] S5. Based on the complete skeleton of the current frame obtained after compensation in S4, the preset Operational Safety Risk Index (OSRI) model is invoked. This model quantifies the safety of the current operating posture according to the dangerous areas and violation posture rules defined in the coal mine safety regulations, outputs the real-time risk score of the current frame, and identifies whether there are any violations of the violation posture rules.

[0146] In specific implementation, the dangerous area is a virtual boundary defined in three-dimensional space according to the coal mine safety regulations; the violation includes spatial interference between key points of the human skeleton and the virtual boundary, and / or posture violations that violate the rules of the violation posture.

[0147] This approach transforms the abstract safety requirements in coal mine safety regulations into calculable virtual boundaries in three-dimensional space. Combined with real-time spatial interference detection of key points in the human skeleton, risk identification is upgraded from experience-based judgment to a clearly defined, quantifiable, and traceable technical assessment, significantly improving the objectivity and compliance of violation identification. By simultaneously detecting "spatial intrusion" (such as the body entering the dangerous zone of support displacement) and "postural violations" (such as bending over or using one hand to hold the support), a safety assessment system covering both position and posture dimensions is constructed. Compared to traditional methods that only focus on a single behavioral characteristic, this system more comprehensively reflects the complex risks in actual operations. Furthermore, outputting violation types based on clear virtual boundaries and rule logic (such as "left shoulder intrusion into the displacement zone" or "excessive spinal flexion") not only facilitates tiered alarms but also provides a clear basis for subsequent personnel training, accident retrospection, and process optimization, enhancing the closed-loop capability and intelligence level of safety management. This design deeply integrates industry safety regulations into the algorithm logic, making risk assessment both technically rigorous and engineeringally practical, effectively supporting proactive and refined safety control in underground operations.

[0148] In specific implementation, the Operational Safety Risk Index (OSRI) model is a calculation model based on multiple risk assessment factors, including: the spatial intrusion distance between key points of the human skeleton and the dangerous area, the posture violation score corresponding to the currently triggered posture violation, and the duration of the currently triggered posture violation.

[0149] In this way, by introducing risk assessment factors across multiple dimensions, such as spatial intrusion distance, violation level, and duration, the OSRI model can transcend the binary judgment of "whether a violation occurred" and provide a continuous and hierarchical measurement of risk. For example, slight proximity to a danger zone versus deep intrusion, and momentary violation versus prolonged stay, will correspond to different risk scores, making the assessment results more closely reflect the actual level of safety threat. Furthermore, the selected factors are all derived from core concerns in coal mine safety regulations (such as safety distance, action standardization, and exposure time), and their combination aligns with the risk perception habits of on-site safety management personnel. This ensures that the algorithm output is not only technically sound but also possesses good interpretability and management adaptability, facilitating integration into existing safety management systems.

[0150] In practice, the operational safety risk index model calculates the real-time risk score of the current frame using the following formula:

[0151] ;

[0152] In the formula, This is the set of key point coordinates of the human skeleton in the current frame after S4 compensation; A three-dimensional virtual electronic fence (including hydraulic support push rods, scraper conveyor and coal cutting machine cutting section) defined according to coal mine safety regulations; This is the distance risk function, used to calculate... Key points and The minimum Euclidean distance increases exponentially when the distance is less than the safety threshold. This is the posture violation function, used to quantify the severity of the current posture violation. Its output value is the maximum value among all the posture violation scores corresponding to the triggered posture violations in the preset coal mine typical violation posture feature mapping table MVPMT. The duration factor of the triggered posture violation is used to reflect the cumulative effect of risk. , , These are the weighting coefficients.

[0153] By establishing a three-dimensional virtual electronic fence and calculating the Euclidean distance between key points of the skeleton and the fence, it is possible to accurately detect whether a miner's body part has entered the equipment's movement trajectory or a dangerous blind spot, significantly improving the early warning capability for "entering a dangerous area" type accidents. The OSRI model integrates multiple risk factors through weighted coefficients to generate a continuously changing risk score, which not only supports system-level alarm threshold settings but can also be used for subsequent trend analysis and risk tracing, possessing good engineering practicality and scalability.

[0154] This technical solution deeply integrates physical space constraints, human posture semantics, and temporal behavioral characteristics to construct a closed-loop safety assessment mechanism for special underground coal mine operation scenarios. Its core value lies in breaking away from the traditional passive "rules + manual" supervision model and establishing an integrated proactive intelligent protection system encompassing "perception—identification—quantification—early warning." This methodology of "data-driven risk assessment and model-quantified safety assessment" is one of the key breakthroughs of this invention in achieving intelligent upgrades in the field of industrial safety.

[0155] In specific implementation, the process of determining whether the human posture violates the rule of violation includes:

[0156] Based on the current frame's 3D skeleton sequence obtained after S4 compensation. Real-time calculation of joint angles at key locations The skeleton is represented by multiple key points, each bone segment is defined by a line connecting two adjacent key points, and each joint consists of three consecutive key points. The corresponding two bone vectors are used to calculate the joint angle. The key region is a human anatomical joint area characterized by at least three consecutive key points. The joint angle... Calculated using the dot product formula of adjacent bone vectors:

[0157] ;

[0158] In the formula, and These are the vectors of two adjacent bones that make up the joint;

[0159] Traverse the preset coal mine typical violation posture feature mapping table MVPMT; the MVPMT table stores multiple predefined posture violation behavior rules in a multi-row format, each row corresponds to a specific posture violation behavior, and includes at least one of the angle constraint field, spatial position constraint field and time accumulation constraint field;

[0160] For each row, if an angle constraint field is set in the row, check whether the current joint angle meets the conditions defined in the field; if a spatial position constraint field is set, check whether the relevant key points are located in the preset danger zone; if a time accumulation constraint field is set, check whether the current violation status has continued to reach the duration defined in the field.

[0161] When all the set constraint fields in a row are satisfied, it is determined that the corresponding attitude violation has occurred, and the attitude violation score associated with that row is obtained.

[0162] Finally, the attitude violation function The output value is the maximum value among all attitude violation scores identified as occurring; if no attitude violation is triggered, then... =0;

[0163] The attitude violation function The output value is used by the operational safety risk index model to calculate the real-time risk score.

[0164] By introducing joint angle calculation and the MVPMT rule table, this method can not only identify single actions (such as bending over), but also comprehensively judge three types of factors: abnormal angles, spatial boundary violations, and time accumulation, thereby accurately capturing complex violations such as "working under the ceiling" and "illegal standing positions." Compared with traditional methods that rely solely on simple spatial distance or threshold judgments, this solution has stronger semantic expression capabilities and false positive suppression capabilities. Furthermore, the MVPMT table stores rules in a structured table format, with each row corresponding to a violation and its constraints, facilitating dynamic updates and configurations by safety managers according to coal mine regulations. For example, when adding a "both feet off the ground" risk, only a new row needs to be added and the corresponding angle and time constraints set, without modifying the model code, significantly improving the system's flexibility and practicality. Moreover, the joint angle calculation is based on the completed skeleton, avoiding false positives caused by missing key points; simultaneously, the rule matching process is a pure numerical comparison, with low computational overhead, allowing for real-time operation at the edge. This design ensures that the OSRI model maintains high accuracy while still meeting the response speed requirements of underground scenarios. By deeply integrating human anatomical features with safety procedures, this method constructs an automated violation detection mechanism based on geometric constraints and rule-based reasoning. Its core value lies in: not only achieving precise quantitative analysis of miners' operating postures, but also establishing a closed-loop transformation path of "safety specifications—mathematical expression—algorithm execution," enabling safety supervision to shift from "post-event accountability" to "pre-event early warning."

[0165] In specific implementation, the key parts include, but are not limited to: neck, shoulder, elbow, wrist, torso, hip, knee, ankle and pelvis;

[0166] The MVPMT table includes, but is not limited to, the following posture violations and their corresponding judgment rules and posture violation scores:

[0167] Unauthorized work on an open ceiling: The angle constraint field is the neck flexion angle calculated based on the head key points, neck key points, and trunk key points. > 45°, the spatial position constraint field is that the head key point is located outside the boundary of the front beam support range, the time accumulation constraint field is that the duration T > 3s, and the associated posture violation score is 10;

[0168] Violation of posture rules: Angle constraint field not set, spatial position constraint field is that the center of the two foot key points is not located within the safety footing area specified by the support base, time accumulation constraint field is that the duration T>1s, and the associated posture violation score is 8;

[0169] Leaning into the chute: The angle constraint field includes the trunk flexion angle calculated based on the shoulder and hip keypoints. >60°, and the upper arm elevation angle calculated based on shoulder key points, elbow key points, and wrist key points. > 30°, the spatial position constraint field is that the head key point or hand key point is located in the danger zone directly above the scraper conveyor chute, the time accumulation constraint field is that the duration T > 0.5s, and the associated posture violation score is 10;

[0170] Across running equipment: The angle constraint field is the sum of the hip abduction angles of both legs and the knee flexion angle, calculated based on the hip keypoint, knee keypoint, and ankle keypoint. > 60°, the spatial position constraint field is that the pelvic key point is located above the running scraper or transfer machine, the time accumulation constraint field is not set, and the associated posture violation score is 10;

[0171] Improper operating posture: The angle constraint field is the trunk lateral bending / torsion angle calculated based on the key points of both shoulders and both hips. The angle is >30°, the spatial position constraint field is that the hand key point is located in the functional area of ​​the operating handle, the time accumulation constraint field is that the duration T > 5s, and the associated posture violation score is 5.

[0172] To facilitate understanding by those skilled in the art, the above five types of posture violations are summarized in Table 1 (which can be regarded as part of the MVPMT table in specific implementation).

[0173] Table 1 illustrates the five posture violations in the MVPMT table.

[0174]

[0175] By setting multi-dimensional constraints (angle + position + time) for each violation, this scheme can accurately distinguish between easily confused scenarios such as "brief probing" and "continuous overhead movement," and "accidental approach" and "long-term straddling." For example, "leaning into the chute" requires not only a torso flexion angle greater than 60°, but also that the head or hands are in the danger zone for more than 0.5 seconds, significantly reducing the false alarm rate and improving recognition accuracy. Furthermore, each rule is associated with specific physical parameters and time thresholds, giving the system's output "posture violation score" clear physical meaning and safety semantics, supporting subsequent risk tracing and responsibility determination. Compared to traditional binary judgments that only output "whether a violation occurred," this scheme achieves a leap from "black box detection" to "transparent assessment." This method constructs a structured violation rule base (MVPMT) for underground coal mine operations, transforming abstract safety procedures into computable and executable geometric and spatiotemporal constraints, achieving accurate, interpretable, and configurable automated recognition of miners' operating postures. Its core value lies in: opening up a closed-loop path of "safety specifications - mathematical expression - algorithm execution", enabling the intelligent monitoring system to truly have the ability to "understand the procedures, make judgments, and provide early warnings".

[0176] S6. Based on the real-time risk score output by S5 and whether the violation has been identified, trigger the graded early warning mechanism and execute the corresponding safety response measures.

[0177] Compared with existing technologies, this invention constructs a closed-loop architecture of "diagnosis-graded compensation-safety assessment" to achieve intelligent response to complex occlusion scenarios. By using S3 to diagnose missing frames in each frame, it not only determines whether individual keypoints are missing but also calculates the number of consecutive missing frames and the global missing ratio, thus providing multi-dimensional decision-making basis for subsequent compensation strategies. This design breaks through the coarse-grained processing method of "direct interpolation or skipping" in traditional pose estimation, realizing a transformation from "passive completion" to "active perception." Compared with existing technologies that rely solely on confidence thresholds to trigger compensation, this scheme can more accurately distinguish between instantaneous noise interference and persistent occlusion, avoiding frequent activation of computationally expensive deep models under slight perturbations.

[0178] This invention achieves on-demand allocation of computing resources at the edge by employing a missing feature quantification diagnostic mechanism. Utilizing a "cascaded compensation" architecture, most short-term jitter caused by light fluctuations or slight coal dust is handled by the computationally inexpensive Kalman filter, with the large-parameter MAE model only activated when real physical occlusion occurs. This not only ensures real-time performance on edge devices with limited computing power underground (processing latency less than 50ms) but also improves the system's attitude recognition robustness by more than 35%. Furthermore, this invention introduces a "coarse-to-fine" prior guidance mechanism in the deep reconstruction module. By using the coarse trajectory predicted by kinematics as the hint input to the MAE network, it successfully solves the problem of generative algorithms easily reconstructing distorted movements when there is large-area occlusion and extremely sparse input information underground. Experiments demonstrate that this mechanism can effectively utilize the symmetry and interconnectivity of human anatomy, accurately reconstructing fine movements such as miners operating handles and bending over to avoid obstacles even with a keypoint missing rate of 50%. Furthermore, this invention constructs an OSRI assessment system deeply coupled with the "Coal Mine Safety Regulations," transforming general posture recognition into business logic recognition specifically for hydraulic support operations. Through real-time interactive calculations between a 3D virtual electronic fence and a compensation skeleton, centimeter-level accuracy is achieved in identifying high-risk behaviors such as leaning over to move supports and illegal crossings, significantly reducing the false alarm rate of safety warnings and providing a technically valuable means for achieving inherent safety in coal mine working faces.

[0179] Example 2

[0180] To facilitate a better understanding of the technical content of one embodiment of this method by those skilled in the art, the following illustrative description is provided.

[0181] In this illustration, to implement the method of the present invention, a posture recognition system deployed in a mining intrinsically safe edge computing box is used, such as... Figure 4 As shown, the system input is the RTSP video stream from the explosion-proof camera at the fully mechanized mining face, and the output is alarm signals and control commands; the core processing flow includes five functional modules: data acquisition, skeleton extraction, missing data diagnosis, cascade compensation, and safety judgment.

[0182] The specific implementation process of the method is as follows:

[0183] (1) Step S1: Video acquisition and preprocessing

[0184] An explosion-proof underground camera is used to acquire real-time video streams of the hydraulic support operating area in the fully mechanized mining face. The video stream undergoes preprocessing such as distortion correction and noise reduction to improve image quality.

[0185] (2) Step S2: Primary skeleton extraction

[0186] Extracting key points of the miner's human skeleton from video frames using a lightweight pose estimation network (Openpose). ,in Confidence level for key points.

[0187] (3) Step S3: Quantitative diagnosis of missing features

[0188] To achieve optimal results with limited computing power, the system first performs a "physical examination" on the skeleton quality.

[0189] Define a sliding time window W = 30 frames. For the skeleton of frame t... The system calculates two key indicators:

[0190] ① Number of consecutive missing frames For each joint j, backtrack the number of consecutive frames where its confidence c < 0.4. If any key part (such as the wrist or ankle) has a confidence level of 0.4, then... >10 frames (approximately 0.4 seconds) are marked as long-term missing.

[0191] ②Spatial occlusion rate : Calculate the percentage of invisible joints in the current frame. For example, when coal dust completely obscures the lower body, It could reach 50%.

[0192] (4) Step S4: Adaptive Cascade Compensation Mechanism

[0193] This invention designs a dual-channel compensation path that dynamically switches based on diagnostic results:

[0194] A. First-level channel: Fast repair based on adaptive Kalman filtering

[0195] Applicable scenarios: For short-term frame drops caused by lighting flicker or slight motion blur. <10).

[0196] Algorithm implementation:

[0197] Construct a constant velocity model. Define the state vector. The state transition matrix F is set as follows:

[0198] ;

[0199] Adaptive update strategy: When a lost observation is detected, no measurement update is performed; only a time update (prediction step) is executed. Simultaneously, the process noise covariance matrix Q-amplification factor is increased. >1 (for example) This process reflects the expansion of prediction errors over time. The calculation time is less than 1ms, ensuring the real-time baseline of the system.

[0200] B. Second-level channel: Deep spatiotemporal reconstruction based on ST-MAE

[0201] Applicable scenarios: For long-term, large-area deficiencies caused by obstruction of hydraulic support columns or dense coal dust coverage.

[0202] Network structure: Asymmetric Transformer architecture is adopted.

[0203] Input embedding: The skeleton sequence is divided into spatiotemporal patches. To address the convergence difficulty under severe occlusion, this embodiment introduces a "hint-guided" mechanism. The coarse predicted trajectory generated by the first-level channel (although inaccurate, it provides a general location) is encoded as... This is then added to the input Token.

[0204] Tube Masking Strategy: Instead of random masking during training, “tubular” masks are generated. For example, a 10×10 pixel cloud of coal dust is simulated to move across the screen at a speed of v, and the key points it occludes are spatiotemporally continuous tubular regions. This forces the network to learn the global dependencies of human motion (e.g., inferring the position of the occluded right arm from the swing of the left arm).

[0205] Loss function design: To prevent the generated skeleton from exhibiting "long arms and legs" distortions, a bone length constraint loss is introduced. :

[0206] ;

[0207] in This is the standard limb length for the miner (obtained from calibration in the initial few frames). This ensures that the generated pose conforms to human biomechanical constraints.

[0208] (5) Step S5: Hydraulic support operation compliance judgment (OSRI model). The logic for safe posture judgment is as follows: Figure 5 As shown.

[0209] The discrimination system is based on the compensated complete skeleton sequence. A compliance analysis of the safety procedures for operating hydraulic supports was conducted.

[0210] The OSRI calculation formula is the same as in Example 1, and its principle and effect will not be repeated here.

[0211] (6) Step S6: Warning Execution

[0212] Based on the real-time risk score output by S5 and whether the violation is identified, a graded early warning mechanism is triggered, and corresponding safety response measures are executed.

[0213] As an example, in this embodiment, if 50 < OSRI < 80: trigger a yellow warning (voice prompt "Pay attention to your position");

[0214] If OSRI > 80 (e.g., a hand is detected while the support is in motion): trigger a red alarm and output a relay signal to cut off the power supply to the support (lock).

[0215] Example 3

[0216] Unlike Embodiment 1, in S6 of this embodiment, the graded early warning mechanism is divided into three levels based on the real-time risk score OSRI(t), and the following security response measures are executed accordingly:

[0217] 1) When OSRI(t) exceeds the first threshold but does not reach the second threshold, it is judged as a mild risk. The tactile feedback unit integrated in the miner's safety equipment outputs a low-frequency, intermittent vibration signal (such as simulating a "shoulder tap") to convey an abnormal posture prompt to the miner. The frequency range of the low-frequency vibration signal is 5–30 Hz. Under this safety response measure, it will not interfere with the miner's normal operation.

[0218] 2) When OSRI(t) reaches or exceeds the second threshold but does not reach the third threshold, it is determined to be a moderate risk. The tactile feedback unit outputs a high-frequency vibration signal and simultaneously controls the smart miner lamp to enter a flashing mode (such as a yellow flashing mode) to warn the miner to immediately correct his posture or leave the danger zone; the frequency range of the high-frequency vibration signal is 80–200Hz.

[0219] 3) When OSRI(t) reaches or exceeds the third threshold, it is determined to be a serious risk. The edge computing device (via industrial Ethernet or CAN bus) sends a high-priority emergency interlock command to the hydraulic support controller, causing the controller to cut off the power supply to the pilot magnets of the relevant action circuit, forcibly stopping the dangerous action. The dangerous action includes pushing, lowering the column, and / or deploying the side guard plate. After receiving the command, the hydraulic support controller can cut off the power supply to the pilot magnets of the relevant action valve group within milliseconds, forcibly stopping dangerous actions such as pushing, lowering the column, or deploying or retracting the side guard plate, thus physically blocking the path of the accident.

[0220] The locking state triggered by the emergency locking command can only be released after the miner has left the danger zone, the human posture has returned to a compliant state, and the OSRI(t) has dropped below the first threshold.

[0221] This introduces a tactile feedback channel, ensuring that warning messages are 100% perceived by miners even in the extreme underground environment. More importantly, through deep integration with heavy equipment, an insurmountable "electronic safety barrier" is established. This ability to proactively intervene physically within milliseconds of a potential danger significantly enhances the system's safety protection level and practical value, representing a key innovative feature that distinguishes it from existing video surveillance systems.

[0222] By categorizing early warnings into three levels and corresponding response measures of varying intensities, the system can adopt a strategy of "mild reminder, moderate warning, and severe intervention" based on the actual level of risk. Compared to the traditional crude response of "alarming at the first sign of an anomaly," this solution avoids vigilance fatigue caused by frequent false alarms, improving the effectiveness and acceptability of early warnings. Furthermore, by integrating a multimodal feedback mechanism, the system enhances the safety guidance capabilities of human-machine interaction. In the mild and moderate risk stages, tactile vibration and visual flashing are used as dual warning methods, fully utilizing the miners' sensory channels for coordinated alerts. This multi-sensory linkage design significantly improves information transmission efficiency, effectively conveying safety warnings even in the high-noise, high-light interference of underground environments. Moreover, it can achieve mandatory safety interlocking of critical equipment, constructing a physical-level protective barrier. When the risk reaches a severe level, the system not only issues a warning but also directly sends a highest-priority emergency interlocking command to the hydraulic support controller, forcibly cutting off the power supply to the action circuit, achieving physical-level blocking of dangerous actions such as "pushing" and "lowering the column." This mechanism breaks through the limitations of traditional monitoring systems that "can only alarm but cannot control equipment," and truly realizes closed-loop control from "monitoring and early warning" to "active protection."

[0223] This method establishes a safety response system of "risk stratification—response grading—closed-loop control," transforming the risk scores output by intelligent algorithms into executable and verifiable safety actions. This achieves full-chain, multi-level, and adaptive protection for miners' operational behaviors. Its core value lies not only in enhancing the system's early warning capabilities but also in constructing an integrated safety closed loop of "perception—assessment—response—intervention," enabling intelligent monitoring to truly achieve the engineering effectiveness of "prevention before the event."

[0224] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for downhole safety posture recognition and early warning based on adaptive cascaded skeleton compensation, characterized in that, Includes the following steps: S1. Obtain real-time video stream data of the hydraulic support operation area in underground coal mines; S2. Based on the video stream data of S1, a lightweight pose estimation network deployed on the edge computing terminal is used to extract the key point coordinates and confidence scores of the miner's human skeleton frame by frame to form the original skeleton sequence. S3. For each frame in the original skeleton sequence of S2, perform missing point diagnosis: determine whether the confidence of each key point in the frame is lower than the preset threshold, and count the number of consecutive missing frames for each key point up to the current frame, and calculate the global missing key point ratio of the frame. S4. Based on the diagnostic results of S3, perform the following branch compensation logic on the current frame: (a) If the number of consecutive missing frames of all key points in the current frame does not exceed the first threshold, and the proportion of missing key points in the frame does not exceed the second threshold, then the first level of compensation is initiated: an adaptive filtering algorithm based on the state space model is used to perform kinematic prediction and compensation for the missing key points in the current frame. (b) Otherwise, initiate the second level of compensation, executing a coarse-to-fine joint guided refactoring process, including: The first-level compensation is used to perform preliminary prediction and filling of the missing regions in the current frame, generating a coarse trajectory feature matrix for the current frame. Through the coordinate encoding layer, the coordinate features of key points that are determined to be visible by S3 in the current frame are obtained. MaskToken, corresponding to missing key points, and based on Generated 3D spatiotemporal location code The input representation is then spliced ​​and merged to form an enhanced input representation. The enhanced input representation is fed into a mask autoencoder based on a spatiotemporal tubular mask. The encoder models the anatomical topology and temporal evolution of human joints, and the decoder performs manifold reconstruction to generate the complete skeleton of the current frame. The spatiotemporal tubular mask is configured to simulate a continuous occlusion scene in a well. S5. Based on the complete skeleton of the current frame obtained after compensation by S4, the preset operation safety risk index model is called. This model quantifies the safety of the current operation posture according to the dangerous areas and violation posture rules defined in the coal mine safety regulations, outputs the real-time risk score of the current frame, and identifies whether there is any violation of the violation posture rules. S6. Based on the real-time risk score output by S5 and whether the violation has been identified, trigger the graded early warning mechanism and execute the corresponding safety response measures.

2. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 1, characterized in that: In S4(a), the adaptive filtering algorithm is a Kalman filter, and its process noise covariance is dynamically adjusted according to the confidence level of the key points in the previous frame. The implementation of the adaptive filtering algorithm includes: Constructing the state vector of key points ,in The coordinates of the k-th keypoint in the current frame. Its velocity component; Set the state prediction equation: ； In the formula, Here is the state transition matrix. The process noise covariance matrix; Among them, process noise covariance Based on the confidence level of key points in the previous frame Dynamic adjustment: Confidence level when key points are detected Less than the confidence threshold At that time, using the predicted value As a compensation result, and to increase the process noise covariance Q at the next time step. k+1 This is to reflect the accumulation of forecast uncertainty.

3. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 1, characterized in that: In S4(b), the key point coordinate features that were determined to be visible in the current frame by S3 are... MaskToken, corresponding to missing key points, and based on Generated 3D spatiotemporal location code The expression for splicing and merging is: ； In the formula, ⊕ represents the feature splicing operation.

4. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 1, characterized in that: In S4(b), the spatiotemporal tubular mask is constructed during the training phase as an occlusion mask matrix that moves continuously along the time axis. ∈ Where T represents the number of time frames and J represents the number of key points of the human skeleton, used to simulate the persistent regional occlusion caused by coal dust clumps or equipment structural components on the input skeleton sequence tensor. The mask autoencoder is optimized during training using the following joint loss function: ； In the formula, The mean square error between the reconstructed coordinates and the true coordinates is used to measure the accuracy of the position reconstruction. The bone length consistency loss is used to constrain the rate of change of limb length after reconstruction to conform to the rigid structure of the human body. This is a temporal smoothing loss used to ensure the continuity of the motion trajectory; , , These are the weighting coefficients.

5. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 1, characterized in that: In S5, the dangerous area is a virtual boundary defined in three-dimensional space according to the coal mine safety regulations; the violation includes spatial interference between key points of the human skeleton and the virtual boundary, and / or posture violation behavior that violates the rule of the violation posture.

6. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 5, characterized in that: In S5, the operational safety risk index model is a calculation model based on multiple risk assessment factors, including: the spatial intrusion distance between key points of the human skeleton and the dangerous area, the posture violation score corresponding to the currently triggered posture violation, and the duration of the currently triggered posture violation.

7. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 6, characterized in that: In S5, the operational safety risk index model calculates the real-time risk score of the current frame using the following formula: ； In the formula, This is the set of key point coordinates of the human skeleton in the current frame after S4 compensation; A three-dimensional virtual electronic fence defined according to coal mine safety regulations; This is the distance risk function, used to calculate... Key points and The minimum Euclidean distance increases exponentially when the distance is less than the safety threshold. This is the posture violation function, used to quantify the severity of the current posture violation. Its output value is the maximum value among all the posture violation scores corresponding to the triggered posture violations in the preset coal mine typical violation posture feature mapping table MVPMT. The duration factor of the triggered posture violation is used to reflect the cumulative effect of risk. , , These are the weighting coefficients.

8. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 7, characterized in that: The process for determining whether a human posture violates the rule of violation includes: Based on the current frame's 3D skeleton sequence obtained after S4 compensation. Real-time calculation of joint angles at key locations The skeleton is represented by multiple key points, each bone segment is defined by a line connecting two adjacent key points, and each joint consists of three consecutive key points. The corresponding two bone vectors are used to calculate the joint angle. The key region is a human anatomical joint area characterized by at least three consecutive key points. The joint angle... Calculated using the dot product formula of adjacent bone vectors: ； In the formula, and These are the vectors of two adjacent bones that make up the joint; Traverse the preset coal mine typical violation posture feature mapping table MVPMT; the MVPMT table stores multiple predefined posture violation behavior rules in a multi-row format, each row corresponds to a specific posture violation behavior, and includes at least one of the angle constraint field, spatial position constraint field and time accumulation constraint field; For each row, if an angle constraint field is set in the row, check whether the current joint angle meets the conditions defined in the field; if a spatial position constraint field is set, check whether the relevant key points are located in the preset danger zone; if a time accumulation constraint field is set, check whether the current violation status has continued to reach the duration defined in the field. When all the set constraint fields in a row are satisfied, it is determined that the corresponding attitude violation has occurred, and the attitude violation score associated with that row is obtained. Finally, the attitude violation function The output value is the maximum value among all attitude violation scores identified as occurring; if no attitude violation is triggered, then... =0; The attitude violation function The output value is used by the operational safety risk index model to calculate the real-time risk score.

9. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 8, characterized in that: The key areas include, but are not limited to: neck, shoulders, elbows, wrists, torso, hips, knees, ankles, and pelvis; The MVPMT table includes, but is not limited to, the following posture violations and their corresponding judgment rules and posture violation scores: Unauthorized work on an open ceiling: The angle constraint field is the neck flexion angle calculated based on the head key points, neck key points, and trunk key points. > 45°, the spatial position constraint field is that the head key point is located outside the boundary of the front beam support range, the time accumulation constraint field is that the duration T > 3s, and the associated posture violation score is 10; Violation of posture rules: Angle constraint field not set, spatial position constraint field is that the center of the two foot key points is not located within the safety footing area specified by the support base, time accumulation constraint field is that the duration T>1s, and the associated posture violation score is 8; Leaning into the chute: The angle constraint field includes the trunk flexion angle calculated based on the shoulder and hip keypoints. >60°, and the upper arm elevation angle calculated based on shoulder key points, elbow key points, and wrist key points. > 30°, the spatial position constraint field is that the head key point or hand key point is located in the danger zone directly above the scraper conveyor chute, the time accumulation constraint field is that the duration T > 0.5s, and the associated posture violation score is 10; Across running equipment: The angle constraint field is the sum of the hip abduction angles of both legs and the knee flexion angle, calculated based on the hip keypoint, knee keypoint, and ankle keypoint. > 60°, the spatial position constraint field is that the pelvic key point is located above the running scraper or transfer machine, the time accumulation constraint field is not set, and the associated posture violation score is 10; Improper operating posture: The angle constraint field is the trunk lateral bending / torsion angle calculated based on the key points of both shoulders and both hips. The angle is >30°, the spatial position constraint field is that the hand key point is located in the functional area of ​​the operating handle, the time accumulation constraint field is that the duration T > 5s, and the associated posture violation score is 5.

10. The downhole safety posture recognition and early warning method based on adaptive cascaded skeleton compensation according to claim 1, characterized in that: In S6, the graded early warning mechanism is divided into three levels based on the real-time risk score OSRI(t), and the following security response measures are executed accordingly: 1) When OSRI(t) exceeds the first threshold but does not reach the second threshold, it is judged as a mild risk. A low-frequency, intermittent vibration signal is output through the tactile feedback unit integrated in the miner's safety equipment to convey an abnormal posture prompt to the miner. 2) When OSRI(t) reaches or exceeds the second threshold but does not reach the third threshold, it is determined to be a medium risk. The tactile feedback unit outputs a high-frequency vibration signal and simultaneously controls the smart miner lamp to enter the flashing mode to warn the miner to immediately correct his posture or leave the danger zone. 3) When OSRI(t) reaches or exceeds the third threshold, it is judged as a serious risk. The edge computing device sends the highest priority emergency interlock command to the hydraulic support controller, causing the controller to cut off the power supply to the pilot magnet of the relevant action circuit and forcibly stop the dangerous action. The dangerous action includes pushing, lowering the column and / or deploying the side guard plate. The locking state triggered by the emergency locking command can only be released after the miner has left the danger zone, the human posture has returned to a compliant state, and the OSRI(t) has dropped below the first threshold.

Images