Aerial work fatigue analysis and anti-falling early warning method based on space-time diagram convolutional network

Abstract

The invention discloses an aerial work fatigue analysis and anti-falling early warning method based on a space-time diagram convolutional network, and belongs to the technical field of artificial intelligence safety monitoring. According to the method, mechanical association dynamic modeling is achieved for the first time, an adaptive joint weight map convolutional layer is used for the first time, mechanical association between joint points in high-altitude actions is dynamically learned, in addition, a multi-scale cavity time sequence convolution module is designed, instantaneous unbalance actions and progressive fatigue characteristics are synchronously captured, and slip early warning can be implemented in advance compared with a traditional method; a four-level behavior chain is established, and biomechanical parameters and visual features are fused in the four-level behavior chain, so that the false alarm rate is obviously reduced; a multi-modal graph attention mechanism MGAT is adopted, a fatigue-state joint importance re-calibration mechanism is focused, and the problem that a traditional model is not sensitive to action deformation caused by fatigue is solved; cross-scale time sequence physical fusion is adopted, the problem that a traditional model cannot capture a long-period fatigue trend and short-time physical sudden change at the same time is solved, and the detection rate of micro-amplitude high-risk actions is increased.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence safety monitoring technology, and in particular relates to a method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks. Background Technology

[0002] Currently, the field of high-altitude operation safety monitoring faces the following three major technical bottlenecks:

[0003] 1. Limitations of dynamic behavior recognition: Traditional visual models rely on fixed joint association rules, which are difficult to adapt to sudden mechanical changes such as safety rope pulling and strong wind disturbances, and have a delayed response to instantaneous high-risk actions such as slipping and loss of balance.

[0004] 2. Lack of fatigue quantification: Existing systems can only identify single-point behaviors (such as closing eyes or yawning), lacking the ability to model the chain evolution of "progressive fatigue → behavioral degradation → accident triggering", resulting in a high false alarm rate;

[0005] 3. Multimodal fusion barrier: The mismatch between the spatiotemporal scales of bioelectrical signals and visual data leads to a disconnect between muscle fatigue state and movement deformation characteristics, resulting in a high rate of missed detection of key risk points, such as the precursor to a fall caused by slight tremors.

[0006] Existing patents include: CN114693532A, which uses basic ST-GCN motion recognition but does not model inter-joint mechanical dependencies, such as the tension of safety ropes, resulting in a false negative rate of >35% for minute movements; CN112950705A, which uses pressure sensing and posture estimation but does not integrate bioelectrical signals, resulting in no response in the early stages of fatigue and a progressive fatigue detection delay of ≥8 minutes; and WO202015758A1, which uses inertial sensors and threshold judgment but cannot predict behavioral sequences, leading to frequent false triggers and a false alarm rate as high as 22.7%. Published patent CN113592939A, which first introduced LSTM behavioral chain prediction and supports 3-level fatigue state classification, but relies on pure visual input, lacks biomechanical parameters, does not solve the problem of cross-scale temporal modeling, and still exhibits a fragmentation of long / short cycle features.

[0007] Therefore, there is an urgent need for a new technical solution to address this problem. Summary of the Invention

[0008] The technical problem to be solved by this invention is to provide a fatigue analysis and fall prevention early warning method for high-altitude operations based on spatiotemporal graph convolutional networks to solve the technical problems in the existing technology, such as lack of mechanical correlation in high-altitude operation safety monitoring, delayed response to instantaneous high-risk actions, lack of fatigue quantification, high false alarm rate, and high missed detection rate of key risk points due to lack of integration of bioelectrical signals.

[0009] A fatigue analysis and fall prevention early warning method for high-altitude operations based on spatiotemporal graph convolutional networks includes the following steps, which are performed sequentially:

[0010] Step 1: Acquire physiological signals of high-altitude workers, real-time posture data of the human body during high-altitude work, human bio-electromyography signals, and videos of high-altitude behavior;

[0011] Step 2: Time-based data synchronization of real-time human posture data and human bio-electromyography signals during high-altitude work; and extraction of selected human key points and joint coordinates from high-altitude behavior videos using the OpenPose human posture estimation algorithm.

[0012] Step 3: Use timestamp alignment engine technology to ensure that the time corresponding to the three sets of modal data are consistent, based on the synchronized real-time posture data of human body working at height, human bio-electromyography signals, and human key points and joint coordinates.

[0013] Step 4: Construct a fatigue detection model

[0014] First, a spatiotemporal feature extraction ST-GCN is employed to simultaneously extract spatial and temporal features from three sets of modal data, obtaining human posture and action features for fatigue detection. The ST-GCN uses a collaborative architecture of multimodal graph attention mechanism MGAT and multi-scale dilated temporal convolution. Considering the impact of safety rope tension on the center of gravity, an adaptive joint weight graph convolutional layer is incorporated into the graph convolutional network (GCN) to dynamically learn the mechanical relationships between joints in high-altitude movements. Multi-scale dilated temporal convolution is used to simultaneously capture instantaneous imbalance action features and progressive fatigue features from the three sets of timestamp-aligned modal data. The multimodal graph attention mechanism MGAT focuses on the fatigue-state joint importance recalibration mechanism, enhancing the model's sensitivity to fatigue-induced motion deformation recognition. Simultaneously, cross-scale temporal-physical fusion captures both long-term fatigue trends and short-term physical abrupt changes.

[0015] Then, a four-level behavioral chain is established, including physiological signals, loss of control, postural instability, and fall risk, and corresponding threshold conditions are set.

[0016] Then, the human posture and movement characteristics are analyzed step by step according to the four-level behavior chain and it is determined which of the four-level behavior chain they belong to;

[0017] Then, the instantaneous imbalance action features and progressive fatigue features obtained are respectively divided into two warning methods, namely instantaneous fatigue warning and chronic fatigue monitoring, through the risk-focused fusion multimodal graph attention mechanism MGAT method, thereby obtaining real-time fatigue value and fall risk value;

[0018] Step 5: Based on the set fatigue and fall risk warning thresholds, conduct fatigue level assessment and fall prevention warning for high-altitude operations using real-time fatigue values ​​and fall risk values, and implement corresponding warnings and protective measures.

[0019] The high-altitude workers wear wearable smart bracelets to perform their work. The real-time posture data of the human body during high-altitude work is obtained through the IMU sensor in the wearable smart bracelet. The features extracted by the graph convolutional network (GCN) and temporal convolution include the acceleration changes of various parts of the body, the angular velocity changes of body rotation, the frequency of human posture changes, and the vibration frequency of the body.

[0020] The human bio-electromyography (EMG) signal is obtained through a wearable smart bracelet. The features obtained by the spatiotemporal feature extraction ST-GCN include muscle activation intensity, muscle activation frequency, muscle fatigue index, and muscle synergy. The muscle fatigue index is characterized by the spectral features of the EMG signal, including median frequency and average power frequency.

[0021] The high-altitude behavior video was captured by a camera and processed by the OpenPose human pose estimation algorithm to extract key points and joint coordinates of the human body and obtain the following features: joint angle changes, movement frequency and amplitude, posture stability, movement symmetry and movement delay; wherein, the joint angle changes include the angle changes of the elbow, knee and waist; the posture stability includes the degree of swaying of the body's center of gravity or selected key points; the movement symmetry refers to the symmetry of the movements on the left and right sides of the body.

[0022] Features obtained by extracting key points and joint coordinates of the human body, and then processed by the spatiotemporal feature extraction ST-GCN, include fatigue-related facial expression changes, head posture changes, eye features, and body posture features. The fatigue-related facial expression changes include changes in blinking frequency and yawning frequency. The head posture changes include the tilt angle of the head and the degree of head shaking. The eye features include changes in eye closure time and gaze direction. The body posture features include the degree of curvature of each specified body part.

[0023] The physiological signals are detected and obtained by the wearable smart bracelet, including heart rate, heart rate variability (HRV), skin conductance (EDA), and electroencephalogram (EEG).

[0024] The chronic fatigue monitoring uses an LSTM long short-term memory structure model based on long-period data, while the instantaneous fatigue early warning uses a TCN temporal convolutional network model based on short-period data.

[0025] Through the above design scheme, the present invention can bring the following beneficial effects:

[0026] 1. This invention is the first to achieve dynamic modeling of mechanical correlations. In the spatiotemporal feature extraction ST-GCN, it is the first to use an adaptive joint weight graph convolutional layer to dynamically learn the mechanical correlations between joints in high-altitude movements (such as the effect of safety rope pulling on the center of gravity). In addition, a multi-scale dilated temporal convolutional module is designed to simultaneously capture instantaneous imbalance movements (such as slipping) and progressive fatigue features (such as continuous eye closure), which can implement slip warnings earlier than traditional methods.

[0027] 2. The world's first four-level fatigue behavior transformation quantitative model and fatigue quantification system were established, and a behavior chain prediction model was established. The four-level behavior chain integrates biomechanical parameters (such as joint angular velocity / muscle electrical signals) and visual features, which significantly reduces the false alarm rate.

[0028] 3. The Multimodal Graph Attention (MGAT) mechanism is adopted, which focuses on the recalibration mechanism of joint importance in fatigue state, and overcomes the problem that traditional models are not sensitive to motion deformation caused by fatigue (such as small leg tremors but high risk).

[0029] 4. Employ cross-scale temporal-physical fusion: This overcomes the problem that traditional models cannot simultaneously capture long-term fatigue trends and short-term physical mutations (such as short-term motion deformation caused by prolonged physical exertion), thereby improving the detection rate of micro-amplitude high-risk movements. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0031] Figure 1 This is a basic principle framework diagram of the high-altitude operation fatigue analysis and fall prevention early warning method based on spatiotemporal graph convolutional network of the present invention;

[0032] Figure 2 This is a flowchart of the multivariate data input layer in the high-altitude operation fatigue analysis and fall prevention early warning method based on spatiotemporal graph convolutional network of the present invention;

[0033] Figure 3 This is a flowchart of the core processing engine in the high-altitude operation fatigue analysis and fall prevention early warning method based on spatiotemporal graph convolutional network of the present invention.

[0034] Figure 4 This is a flowchart of the early warning decision layer in the high-altitude operation fatigue analysis and fall prevention early warning method based on spatiotemporal graph convolutional networks of the present invention. Detailed Implementation

[0035] like Figures 1 to 4 As shown, the basic principle framework of the fatigue analysis and fall prevention early warning method for high-altitude operations based on spatiotemporal graph convolutional networks is as follows:

[0036] I. Multi-modal data input layer: Real-time data synchronization based on multimodal data information such as RGB video stream, IMU sensor, and surface electromyography signals;

[0037] The system collects data primarily through three different channels:

[0038] 1. IMU sensor: measures real-time posture data of the human body working at height;

[0039] 2. sEMG electromyography signal: Bio-electromyography signals acquired by the sensor;

[0040] 3. RGB video stream: Video of staff behavior captured by the camera;

[0041] For motion posture and biosignal data acquired by IMU sensors and sEMG, time-based synchronization and alignment are performed using the PTP protocol. For RGB video streams, the OpenPose human pose estimation algorithm is used to extract the joint coordinates of key human points and construct a human skeleton model. For the above three sets of multimodal videos, a timestamp alignment engine is used to ensure that the time corresponding to the three sets of modal data is consistent. Subsequently, fatigue detection-related feature information is extracted and input into the core processing engine.

[0042] II. Core Processing Engine: Based on spatiotemporal feature extraction ST-SGN, four-level behavior chain step-by-step analysis, risk-focused fusion multimodal graph attention mechanism MGAT and other technologies, a fatigue detection model is constructed to analyze the behavior of high-altitude personnel, calculate the real-time fatigue value of high-altitude workers, and the fall risk value for early warning.

[0043] This technical process mainly includes the following modules:

[0044] Spatiotemporal feature extraction ST-GCN: By combining graph convolutional network GCN and temporal convolution, it can simultaneously extract spatial features (such as the relationship between joints) and temporal features (such as the temporal changes of movements). It can extract human posture and movement features for fatigue detection through data information flow.

[0045] Fourth-level behavioral chain analysis:

[0046] A hierarchical analysis method, from physiological signals to fall risk, is used to predict and prevent potentially dangerous behaviors. The four-level behavioral chain is constructed as follows:

[0047] Level1: physiological signal;

[0048] Level 2: Operation out of control;

[0049] Level 3: Attitude instability;

[0050] Level 4: Fall risk;

[0051] Risk value calculation:

[0052] The MGAT method, which is based on risk-focused fusion multimodal graph attention mechanism, divides risk value calculation into chronic fatigue monitoring and instantaneous fatigue early warning. Chronic fatigue monitoring uses an LSTM model based on long-period data, while instantaneous fatigue early warning uses a TCN model based on short-period data.

[0053] Early warning decision layer: Assess the fatigue level of personnel at height calculated by the fatigue detection model, predict fall risk, and trigger early warning devices; construct an early warning decision system based on the set fatigue assessment and fall risk prediction values.

[0054] Physical feedback execution layer: The early warning device interfaces with the monitoring platform and sensor control device to realize fatigue early warning for personnel at high altitudes.

[0055] The feature information extracted by the core processing engine mainly includes the aspects shown in the table below:

[0056]

[0057] The main technical processes and methods of the four-level behavior chain are as follows:

[0058]

[0059] In summary, this invention proposes an intelligent safety protection system that aims to solve the fall risk problem caused by chronic fatigue and sudden loss of control in high-altitude work scenarios. Through multimodal perception, dynamic risk modeling and closed-loop control of active protection, it achieves a fundamental shift from "post-event response" to "pre-event prevention".

[0060] Example:

[0061] The present invention will be further described in detail below with reference to specific embodiments. This embodiment takes a high-altitude operation scenario as an example to illustrate the specific implementation process of the system. The system only uses an RGB camera, an IMU sensor, and an sEMG sensor as data sources, and other data source formats can also be replaced between groups.

[0062] 1. Setting and feature extraction of adaptive joint weight graph convolutional layers

[0063] In the spatiotemporal feature extraction ST-GCN, we introduce an adaptive joint weight graph convolutional layer to dynamically learn the mechanical relationships between joints in high-altitude movements.

[0064] (1) Graph structure definition: Based on the 2D joint coordinates of the human body (a total of 18 key points) extracted from the RGB video stream by OpenPose, a human skeleton graph structure is constructed. Nodes represent joints, and edges represent the physical connections between joints.

[0065] (2) Adaptive weight learning:

[0066] ① Traditional GCNs use a fixed adjacency matrix, while this invention uses a learnable weight matrix W∈R N×N (Where N is the number of joints) Dynamically adjust the relationship between joints.

[0067] ② Simulation of mechanical relationships: In order to simulate mechanical relationships such as "the effect of safety rope tension on the center of gravity" without force sensors, we use IMU data and joint kinematics for indirect calculation.

[0068] a. Center of gravity estimation: The 2D center of gravity of the human body is estimated in real time by extracting the coordinates of the 2D joints of the human body from the RGB video stream and combining them with the known weight ratio of each segment of the human body (preset human body model).

[0069] b. Learning of Anomalous Mechanical Relationships: Anomalies in lumbar acceleration and angular velocity measured by IMU sensors (worn at the waist) are used as important contextual features. When the system detects anomalous abrupt changes in the lumbar IMU data (such as sudden lateral acceleration), the adaptive graph convolutional layer uses an attention mechanism to focus on learning the dynamic connection weights between lumbar joints and supporting leg joints (such as ankles and knees). This allows the network to perceive the impact of a lateral force (such as a safety rope pull or strong wind) on the lower limb supporting joints.

[0070] (3) Feature acquisition and use:

[0071] ① OpenPose features: Extract joint angle changes (by calculating the angle between adjacent joint vectors), motion frequency and amplitude (by joint displacement time FFT), posture stability (by center of gravity sway variance), motion symmetry (by the correlation between left and right joint motions), and motion delay (by the motion response time difference) from the RGB video stream.

[0072] ②IMU sensor characteristics: By wearing IMU modules on the waist and limbs, the sensor collects data on acceleration changes (mean and variance), angular velocity changes (amplitude spectrum), attitude change frequency (dominant frequency analysis), and vibration frequency (high-frequency component energy).

[0073] ③sEMG characteristics: Electromyographic signals are acquired through surface electrodes to calculate muscle activation intensity (signal amplitude integral), muscle activation frequency (zero crossing rate), muscle fatigue index (median frequency shift), and muscle synergy (cross-muscle coherence).

[0074] ④ Multimodal fusion features: By concatenating and normalizing, the above features are integrated into the input feature vector h of each joint. i ∈R d R d This is the set of feature vectors for all joints.

[0075] (4) Adaptive joint weight graph convolutional layer updates node features through graph convolution operation.

[0076] For each joint node, the input feature vector h i The formula for feature updating is:

[0077]

[0078] In this formula: It is the feature vector of the adjacent joint node j in the current layer. Let represent the set of adjacent joint nodes, i.e., all joint nodes connected to joint node i. W is the learnable weight matrix of the graph convolutional layer, representing the strength of the relationships between different joint nodes. A ij σ represents the relationship weights between joint nodes i and j, typically obtained dynamically through an adaptive weight learning process. σ is an activation function (such as ReLU) used to introduce nonlinear transformations, enabling the model to learn complex feature representations.

[0079] 2. A collaborative architecture of multimodal graph attention mechanism (MGAT) and multi-scale dilated temporal convolution.

[0080] (1) MGAT structure:

[0081] ① Input: Multimodal feature vector h i (Output from OpenPose, IMU, and sEMG features via graph convolutional layers).

[0082] ② Attention calculation: For joint i and its neighboring joints Calculate attention score:

[0083] e ij =LeakyReLU(aT[Wh i ||Wh j ]);

[0084]

[0085] Where W is the trainable weight matrix, a is the attention vector, and a T represents the transpose of 'a', and || represents the concatenation operation.

[0086] ③ Feature Update: The output feature of joint node i is:

[0087]

[0088] MGAT uses weight α ij Focus on key joint nodes under fatigue conditions (such as the knee joint when the legs tremble) to improve sensitivity to movement deformation.

[0089] (2) Multi-scale dilated temporal convolution:

[0090] ① Architecture: Three parallel dilated convolutional branches are used, with dilation rates of 1, 2, and 4, and a kernel size of 3 for each. The outputs of the three parallel dilated convolutional branches are fused through concatenation.

[0091] Z = Concat(Conv) d＝1 (H),Conv d＝2 (H),Conv d＝4 (H));

[0092] Where Z represents the fused output feature, and H represents the spatiotemporal feature sequence output by MGAT.

[0093] ② Collaborative Mechanism: MGAT handles spatial dimension joint relationships, while multi-scale dilated convolution handles temporal dimension sequences. The output feature Z simultaneously includes short-term abrupt changes (such as abrupt changes in joint acceleration corresponding to a slip) and long-term trends (such as an increase in the frequency of eye closure).

[0094] (3) Collaborative working mechanism: The spatial feature sequence processed by MGAT is input into multi-scale dilated temporal convolution. The branch with a dilation rate of 1 focuses on capturing instantaneous imbalance actions (such as slipping) represented by sudden changes in IMU angular velocity and abrupt changes in joint angle; the branch with a dilation rate of 4 focuses on capturing progressive fatigue features (such as continuous eye closure) represented by a slow decrease in sEMG muscle fatigue index and a gradual increase in the frequency of eye closure in RGB video.

[0095] 3. The four-level fatigue behavior transformation quantification model and data source are unified.

[0096] We established a four-level fatigue behavior transformation quantification model and achieved fatigue quantification through chain analysis. All analyses were based solely on RGB, IMU, and sEMG data.

[0097] The detailed process of the fourth-level behavior chain is as follows:

[0098] (1) Level 1: Abnormal physiological signals:

[0099] ① Data source: sEMG, RGB video stream (used for facial features, replacing heart rate / eye tracker).

[0100] ②Feature extraction:

[0101] a. median EMG frequency (muscle fatigue).

[0102] b. RGB video stream: Use a preprocessing module to extract blink frequency and yawn frequency (instead of EEG / HRV).

[0103] c. Analysis method: The probability P of physiological abnormality was calculated using a random forest model. phy .

[0104] (2) Level 2: Operation out of control:

[0105] ①Data source: IMU, RGB video stream (using hand joint tracking to replace force sensors and eye trackers).

[0106] ②Feature extraction:

[0107] a. Amplitude variance (IMU acceleration amplitude variance).

[0108] b. Operational accuracy: Track the relative position of hand joints to the tool / work area using RGB video stream, and calculate the smoothness of the motion trajectory and the deviation from the target.

[0109] c. Gaze dispersion: Estimate head pose and approximate gaze direction using RGB video stream, and calculate the frequency and range of their changes.

[0110] ③Analysis method: Fusion P phy Based on the current features, the probability of operational runaway P is predicted using an LSTM model. ctrl .

[0111] (3) Level 3: Attitude instability

[0112] ①Data source: RGB camera, IMU.

[0113] ②Feature extraction:

[0114] a. Body center of gravity offset: The 2D center of gravity position is estimated through the RGB video stream, and its movement speed and range are calculated.

[0115] b. Joint angle variance: calculated based on OpenPose joint coordinates.

[0116] c. Attitude sway spectrum energy: Frequency domain analysis is performed on the acceleration signal of the IMU (waist) to extract low-frequency sway energy.

[0117] ③Analysis method: Fusion P ctrl Based on the current features, the attitude instability probability P is calculated using the TCN model. stab .

[0118] (4) Level 4: Fall Risk

[0119] ① Data source: RGB camera, IMU (replacing depth camera and LiDAR).

[0120] ②Feature extraction:

[0121] a. Estimated distance between the center of gravity and the edge of the support surface: This is a key algorithmic compensation. We use monocular depth estimation techniques from RGB video streams (such as using pre-trained models like MiDaS) to infer coarse depth information from 2D images, thereby estimating the relative distance between the worker's center of gravity and the edge of the work platform in 3D space.

[0122] b. Environmental risk score: By using a predefined electronic fence for the work area (marking a safe zone in the camera view), the environmental risk score is triggered to increase when the estimated center of gravity or foot joint position of a person continues to approach the virtual boundary.

[0123] ③Analysis method: Fusion P stab Based on the estimated environmental characteristics described above, a fall risk value R is output through a fully connected network. fall ∈[0,1].

[0124] (5) Fatigue Quantification System:

[0125] Overall fatigue value F total The calculation is as follows:

[0126] F total =w1P phy +w2P ctrl +w3P stab +w4R fall ;

[0127] The weights w1 to w4 were obtained through training with historical data, and their initial values ​​were set to 0.2, 0.3, 0.3, and 0.2, respectively.

[0128] 4. Threshold conditions for early warning decision-making level

[0129] Early warning decision-making level based on real-time fatigue value F total and fall risk value R fall The following threshold conditions are used to trigger an alert:

[0130] (1) Fatigue warning:

[0131] When F total ≥T fatigue When this is triggered, a Level 1 warning is issued (e.g., a platform audio prompt: "High fatigue level, please rest"). fatigue For the set fatigue warning threshold, T fatigue The value here is set to 0.7.

[0132] (2) Fall risk Rfall Warning:

[0133] When R fall ≥T fall When this occurs, a secondary warning (such as a sensor vibration alarm) is triggered, T fall To set the fall risk threshold, T fall The value here is set to 0.8.

[0134] When R fall ≥T fall 'And if it lasts for 3 seconds, a level 3 warning will be triggered (such as linking an audible and visual alarm and sending an emergency message to the monitoring center), T fall 'T' is the threshold for sending emergency messages. fall The value here is set to 0.9.

[0135] (3) Dynamic threshold adjustment:

[0136] The threshold is dynamically adjusted based on environmental factors (such as ambient light intensity analysis via RGB images) and working duration. For example, during nighttime operations, T... fatigue When the wind speed drops to 0.6 and the wind speed is greater than level 5, T fall It dropped to 0.75.

Claims

1. A method for fatigue analysis and fall prevention early warning in high-altitude operations based on spatiotemporal graph convolutional networks, characterized by: Includes the following steps, And the following steps are performed in sequence. Step 1: Acquire physiological signals of high-altitude workers, real-time posture data of the human body during high-altitude work, human bio-electromyography signals, and videos of high-altitude behavior; Step 2: Time-based data synchronization of real-time human posture data and human bio-electromyography signals during high-altitude work; and extraction of selected human key points and joint coordinates from high-altitude behavior videos using the OpenPose human posture estimation algorithm. Step 3: Use timestamp alignment engine technology to ensure that the time corresponding to the three sets of modal data are consistent, based on the synchronized real-time posture data of human body working at height, human bio-electromyography signals, and human key points and joint coordinates. Step 4: Construct a fatigue detection model First, a spatiotemporal feature extraction ST-GCN is used to simultaneously extract spatial and temporal features from three sets of modal data to obtain human posture and action features for fatigue detection. The spatiotemporal feature extraction ST-GCN adopts a collaborative architecture of multimodal graph attention mechanism MGAT and multi-scale dilated temporal convolution. Considering the influence of safety rope tension on the center of gravity, an adaptive joint weight graph convolutional layer is set in the graph convolutional network GCN to dynamically learn the mechanical relationship between joints in high-altitude movements. Multi-scale dilated temporal convolution is used to simultaneously capture instantaneous imbalance action features and progressive fatigue features from the three sets of modal data after time-stamp alignment. The Multimodal Graph Attention (MGAT) mechanism focuses on the recalibration mechanism of joint importance in fatigue state, enhancing the model's sensitivity to the recognition of motion deformation caused by fatigue. At the same time, it captures long-term fatigue trends and short-term physical mutations through cross-scale temporal physical fusion. Then, a four-level behavioral chain is established, including physiological signals, loss of control, postural instability, and fall risk, and corresponding threshold conditions are set. Then, the human posture and movement characteristics are analyzed step by step according to the four-level behavior chain and it is determined which of the four-level behavior chain they belong to; Then, the instantaneous imbalance action features and progressive fatigue features obtained are respectively divided into two warning methods, namely instantaneous fatigue warning and chronic fatigue monitoring, through the risk-focused fusion multimodal graph attention mechanism MGAT method, thereby obtaining real-time fatigue value and fall risk value; Step 5: Based on the set fatigue and fall risk warning thresholds, conduct fatigue level assessment and fall prevention warning for high-altitude operations using real-time fatigue values ​​and fall risk values, and implement corresponding warnings and protective measures.

2. The method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The high-altitude workers wear wearable smart bracelets to perform their work. The real-time posture data of the human body during high-altitude work is obtained through the IMU sensor in the wearable smart bracelet. The features extracted by the graph convolutional network (GCN) and temporal convolution include the acceleration changes of various parts of the body, the angular velocity changes of body rotation, the frequency of human posture changes, and the vibration frequency of the body.

3. The method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The human bio-electromyography signal is obtained through a wearable smart bracelet, and the features obtained by the spatiotemporal feature extraction ST-GCN include muscle activation intensity, muscle activation frequency, muscle fatigue index, and muscle synergy. The muscle fatigue index is characterized by the spectral characteristics of electromyographic signals, including median frequency and average power frequency.

4. The method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The high-altitude behavior video was captured by a camera, and the OpenPose human pose estimation algorithm was used to extract human key points and joint coordinates to obtain the following features: The factors include joint angle changes, movement frequency and amplitude, postural stability, movement symmetry, and movement delay; wherein, the joint angle changes include changes in the angles of the elbows, knees, and waist; the postural stability includes the degree of swaying of the body's center of gravity or selected key points; and the movement symmetry refers to the symmetry of movements on the left and right sides of the body.

5. The method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks according to claim 4, characterized in that: Features obtained by extracting key points and joint coordinates of the human body, and then processed by the spatiotemporal feature extraction ST-GCN, include fatigue-related facial expression changes, head posture changes, eye features, and body posture features. The fatigue-related facial expression changes include changes in blinking frequency and yawning frequency. The head posture changes include the tilt angle of the head and the degree of head shaking. The eye features include changes in eye closure time and gaze direction. The body posture features include the degree of curvature of each specified body part.

6. The method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The physiological signals are detected and obtained by the wearable smart bracelet, including heart rate, heart rate variability (HRV), skin conductance (EDA), and electroencephalogram (EEG).

7. The method for fatigue analysis and fall prevention early warning of high-altitude operations based on spatiotemporal graph convolutional networks according to claim 1, characterized in that: The chronic fatigue monitoring uses an LSTM long short-term memory structure model based on long-period data, while the instantaneous fatigue early warning uses a TCN temporal convolutional network model based on short-period data.

Images