A Deep Learning Analysis Method for Tower Climbing Actions Integrating Multi-Dimensional Sensor Data

Abstract

This invention discloses a deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data, comprising the following steps: S1, Multi-dimensional sensor data acquisition: Nine-axis Bluetooth sensors are deployed on the limbs of the tower climber to collect human limb movement and posture data; status sensors are deployed at the collar, clamping mechanism, and fastening mechanism of the tower climbing safety protection device to collect the working status data of the protection device; the two types of data are time-stamped and synchronized to form a multi-dimensional sensor dataset. This invention achieves dual-dimensional data fusion and deep feature extraction of human limb movements and protection device status, with a comprehensive discrimination accuracy of 99.0%, filling the technical gap in integrated "action-protection" analysis and reducing the safety hazard of "standardized limb movements but malfunctioning protection devices" from the source.

Description

Technical Field

[0001] The present invention relates to the field of power operation safety training and monitoring, and particularly to a deep learning analysis method for tower climbing actions that integrates multi-dimensional sensing data. Background Art

[0002] Climbing of power transmission towers is a core basic operation for power operation and maintenance personnel. The standardization of its actions and the rationality of the use of protective devices directly determine the safety factor of tower climbing operations and are also key contents of power safety training. In the prior art, the analysis of tower climbing actions mostly adopts a single method of collecting human body limb sensing data. Action posture data is obtained by deploying nine-axis sensors on the limbs of operators, and deep learning models such as Transformer are combined to achieve action type recognition and standardization discrimination. Although unattended tower climbing training has been achieved, the cooperative analysis of the status data of tower climbing safety protection devices and human body limb actions has been ignored, resulting in potential safety hazards such as "standard limb actions but ineffective protection devices"; at the same time, the existing data preprocessing uses fixed step filtering and fixed window segmentation methods, which cannot adapt to the action characteristics of different rhythms such as climbing, rope tying, and operation of protection devices, easily leading to loss or confusion of data features; in addition, traditional deep learning models only extract the temporal features of human body limb actions, without integrating the associated features of the status of protection devices, the discrimination dimension is single, and the feedback mechanism only targets limb actions, making it impossible to achieve real-time early warning of abnormal protection device status.

[0003] Another type of prior art focuses on the mechanical structure optimization of tower climbing safety protection devices. The connection stability between the protection device and the tower foot nails is improved through the cooperation of collar rings, clamping mechanisms, and fastening mechanisms. However, the working status data of the protection device and the human body tower climbing action data are not analyzed联动, and the status monitoring of the protection device and the standardization discrimination of actions are independent of each other, making it impossible to achieve integrated safety control of "action-protection".

[0004] In summary, the prior art has technical defects such as single data collection dimension, poor adaptability of data preprocessing, insufficient model feature extraction ability, incomplete discrimination system, and non-linked feedback mechanism, resulting in insufficient accuracy and comprehensiveness of tower climbing action standardization analysis, and it is difficult to meet the high safety and high-precision requirements of power tower climbing training and actual operations. To solve the above problems, the present invention proposes a deep learning analysis method for tower climbing actions that integrates multi-dimensional sensing data, realizing two-dimensional data fusion of human body limb actions and protection device status, adaptive data preprocessing, deep extraction of multi-dimensional features, three-dimensional standardization discrimination, and linked real-time feedback, filling the technical gap of "action-protection" integrated analysis. Summary of the Invention

[0005] The main purpose of the present invention is to provide a deep learning analysis method for tower climbing actions that integrates multi-dimensional sensing data, which can effectively solve the problems in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data includes the following steps:

[0008] S1. Multi-dimensional sensor data acquisition: Nine-axis Bluetooth sensors are deployed on the limbs of tower climbers to collect human limb movement and posture data. Status sensors are deployed at the collar, clamping mechanism, and fastening mechanism of the tower climbing safety protection device to collect the working status data of the protection device. The two types of data are time-stamped and calibrated to form a multi-dimensional sensor dataset.

[0009] S2. Adaptive data preprocessing: The multi-dimensional sensor dataset is subjected to adaptive moving average filtering, multi-dimensional data dimension expansion, categorical data segmentation, and multi-label dataset labeling in sequence to obtain standardized model training and testing samples.

[0010] S3. Construction of a deep learning model for fusing features: Using Transformer as the base network, a protection device state feature fusion layer, a two-dimensional position encoding module and a three-element multi-head attention mechanism are added to construct a deep learning model for joint analysis of tower climbing action and protection device state. Standardized samples are input into the model to complete the training, resulting in an action analysis model with multi-dimensional feature extraction capabilities.

[0011] S4. Multi-dimensional action standardization analysis: Construct a three-dimensional discrimination matrix of tower type, limb action, and protective device status, establish a tower climbing standard action library and a protective device standard status library corresponding to different tower types, input the real-time collected and preprocessed multi-dimensional sensor data into the trained model, output the limb action category and the protective device status category, and complete the comprehensive standardization score of tower climbing action by comparing the similarity with the standard library.

[0012] S5. Real-time Linkage Feedback: Based on the application scenario, the scores for the standardization of limb movements, the status scores of protective devices, and the corresponding correction suggestions are linked and fed back through visual / voice / audio-visual alarms to achieve real-time correction of the operator's movements.

[0013] Preferably, the nine-axis Bluetooth sensor mentioned in step S1 is an integrated sensor that includes XYZ three-axis acceleration, XYZ three-axis angular velocity, and XYZ three-axis angle. A total of 6 sensors are deployed and worn on the left upper arm, right upper arm, left wrist, right wrist, left ankle, and right ankle respectively. The acquisition frequency is 10Hz to acquire real-time displacement, rotation, and motion posture data of the human limbs.

[0014] The status sensors include pressure sensors, displacement sensors, and angle sensors. The pressure sensors are deployed on the inner side of the collar of the protective device, the clamping surface of the semi-circular frame, and the contact surface of the trapezoidal movable block to collect contact pressure data between the protective device and the tower foot spikes. The displacement sensors are deployed on the sliding block and the moving frame of the protective device to collect displacement data of the components of the protective device. The angle sensors are deployed on the semi-circular frame and the manual lever of the protective device to collect rotation angle data of the components of the protective device. The acquisition frequency of the status sensors is consistent with that of the nine-axis Bluetooth sensor, and all sensors are equipped with Bluetooth communication modules to transmit the collected data to the computing server in real time.

[0015] The timestamp synchronization calibration unifies the acquisition clocks of all sensors, ensuring that human limb data and protective device status data acquired at the same time point have the same timestamp, guaranteeing the temporal correlation of the data, and controlling the calibration error within ±0.01s.

[0016] Preferably, the adaptive moving average filtering in step S2 abandons the fixed step distance filtering method of the prior art and dynamically adjusts the moving step distance according to the action type characteristics of the sensor data: for the climbing action of human limbs, because the movement amplitude is large and the rhythm is slow, the moving step distance is set to 5, and one position is moved in the same direction each time.

[0017] For the operation of protective devices for human limbs, due to the small range of motion and fast pace, the step distance is set to 3, and each time it moves one position in the same direction;

[0018] For the status data of the protective device, due to the small data fluctuation and high stability requirements, the movement step size is set to 4, and it moves one position in the same direction each time.

[0019] The filtering algorithm formula is: ,in Here, n represents the raw sensor data, n is the dynamically adjusted step size, and t is the current time point. The filtered data is processed by this algorithm to remove clutter peaks and noise interference, making the characteristics of multi-dimensional sensor data more prominent.

[0020] Preferably, the multi-dimensional data dimension expansion in step S2 includes two parts: first, calculating the sum of the acceleration, angular velocity, and angle data from the nine-axis sensor of the human limb along the XYZ axes, with the formula for the sum calculation being: Where X, Y, and Z are the original data for each dimension, A is the sum value, and supplementary feature values ​​describe only the size of the data and not its direction.

[0021] Secondly, the status data of the protective device is normalized, and the contact pressure, component displacement, and component rotation angle data are mapped to the [0,1] interval. Then, the normalized status data of the protective device is added to each row of the human limb data to achieve dimensional fusion of the two-dimensional data.

[0022] The categorized data segmentation sets different sliding window sizes according to the action type: for human limb climbing action, since the action completion time is about 2 seconds, combined with the 10Hz acquisition frequency, the sliding window size is set to 20 sampling points.

[0023] For the operation of human limb protective devices and changes in the status of the protective devices, since the action completion time is about 1.5 seconds, the sliding window size is set to 15 sampling points;

[0024] The window overlap rate for all data segmentation is set to 80% to ensure that each segment contains complete action and state information and to avoid feature loss.

[0025] The multi-label dataset annotation combines tower type, human limb movement type, and protective device status type for triple label annotation, ultimately obtaining a dataset of no less than 19,207 data slices, which are then divided into training set, validation set, and test set according to a ratio of 70%, 20%, and 10%, respectively.

[0026] Preferably, the improved Transformer deep learning model described in step S3 is improved in three aspects based on the original Encoder-Decoder structure: First, a protective device state feature fusion layer is added after the Embedding module to concatenate and fuse the human limb feature vector and the protective device state feature vector. The fused vector dimension is the sum of the human limb feature vector dimension and the protective device state feature vector dimension, thus achieving the initial fusion of dual-dimensional features.

[0027] Second, the location coding module is improved by adopting a two-dimensional location coding, which not only encodes the temporal location information of human limb data, but also encodes the correlation location information between the protective device status data and human limb data. The location coding matrix is ​​generated by a sine-cosine function, with even columns activated by a sine function and odd columns activated by a cosine function, thus making up for the defect of Transformer network that cannot directly obtain the location relationship of temporal data.

[0028] Third, the original multi-head attention mechanism is improved into a ternary multi-head attention mechanism, which includes a human body action self-attention sub-module, a protective device state self-attention sub-module, and an action-protective device association attention sub-module. The three sub-modules work in parallel to extract the temporal features of human limb actions, the change features of the protective device state, and the linkage features between human actions and the protective device state, respectively. Then, the output features of the three sub-modules are concatenated through the Concat operation to achieve deep extraction of multi-dimensional features.

[0029] The Encoder module of the model consists of six stacked repeating sub-blocks, each containing a ternary multi-head attention mechanism and a feedforward neural network. The Decoder module consists of six stacked repeating sub-blocks, each containing a masked ternary multi-head attention mechanism, a ternary multi-head attention mechanism, and a feedforward neural network. The modules are connected through residual networks to ensure the training stability of the model.

[0030] Preferably, the three-dimensional discrimination matrix in step S4 uses the tower type as the row dimension, the human limb movement type as the column dimension, and the protective device status type as the depth dimension. Each element in the matrix is ​​the standard weight value of the coordination between a certain human limb movement and a certain status of the protective device under the corresponding tower type. The weight value is determined through statistical analysis of a large amount of actual standard operation data, and the value range is [0,1]. The closer the weight value is to 1, the more standard the coordination between the movement and the status.

[0031] The tower climbing standard action library and the protective device standard status library are constructed separately for three types of towers: cylindrical cement towers, cylindrical metal towers, and high-voltage iron towers. The standard action library contains standard limb action characteristic data for operations such as climbing up and down the tower, tying long ropes, tying short ropes, and wearing safety helmets for each type of tower. The standard status library contains standard status characteristic data for protective devices such as collar fastening pressure, semi-circular frame clamping angle, trapezoidal movable block abutment pressure, and sliding block displacement for each type of tower.

[0032] The similarity comparison uses a cosine similarity algorithm to calculate the cosine similarity value between the real-time feature vector output by the model and the corresponding feature vector in the standard library. The closer the similarity value is to 1, the closer the real-time action / state is to the standard. The comprehensive standardization score is obtained by weighted summation of the similarity value of human limb action and the similarity value of the protective device state. The weight coefficient is determined according to the degree of influence of the two on the safety of climbing the tower. The weight coefficient of human limb action is set to 0.6, and the weight coefficient of protective device state is set to 0.4. The comprehensive score is out of 100 points. A score ≥ 80 is considered standard, 60 ≤ score < 80 is considered basically standard, and a score < 60 is considered non-standard.

[0033] Preferably, the real-time feedback in step S5 is designed according to two application scenarios: one is a virtual reality tower climbing training scenario, where the operator wears VR glasses and the central control server displays the standardization score of human limb movements and the status score of protective devices in the VR glasses in the form of a visual interface. At the same time, it provides specific correction suggestions for non-standard limb movements and the status of protective devices, such as "the left upper arm is not raised enough, please adjust to 60°" and "the contact pressure of the protective device collar is not enough, please retighten". The operator can correct in real time while practicing.

[0034] Secondly, in the physical tower climbing operation / training scenario, the operator does not need to wear VR glasses. The system provides the operator with a comprehensive score and correction suggestions through the voice module. When the protective device is in a non-standard state, the sound and light alarm module is immediately triggered, emitting a red light and a buzzer alarm sound to remind the operator to stop climbing the tower immediately and correct the protective device state. When the human body movement is non-standard and the protective device state is basically standard, only a voice reminder is issued to achieve hierarchical feedback.

[0035] Meanwhile, all scoring data and feedback information are displayed in real time on the central control server's screen, facilitating remote monitoring by administrators.

[0036] Preferably, the model also includes a self-learning optimization step: storing the multi-dimensional sensor data, corresponding operation scenarios, and discrimination results collected during the actual application process to form a model iteration dataset; periodically preprocessing the iteration dataset according to the method of claim 4, and inputting it into the trained improved Transformer model for incremental training; and dynamically adjusting the weight values ​​of the three-dimensional discrimination matrix and the feature data of the specification library according to the standard requirements of the actual operation to optimize the model's feature extraction capability and discrimination accuracy, so that the model can adapt to the operating habits of different operators and the tower climbing operation scenarios in different environments. The model iteration cycle is set to 1 month, and the discrimination accuracy of the model increases by no less than 2% after each iteration.

[0037] Preferably, a multi-dimensional sensor data anomaly calibration step is included between step S1 and step S2: an anomaly detection model based on isolated forest is established to detect anomalies in the collected raw multi-dimensional sensor data and identify abnormal data points caused by sensor process deviations, worker limb tremors, and mechanical vibration of protective devices.

[0038] For the detected abnormal data points, linear interpolation is used to compensate for the data and supplement the missing feature values;

[0039] To address the sensor drift problem, a zero-point drift compensation model is established. Based on the sensor's historical data, the zero-point deviation of the sensor is corrected in real time to ensure the authenticity and purity of multi-dimensional sensor data. The detection accuracy of abnormal data points is no less than 95%, and the error of the compensated data analysis is controlled within ±5%.

[0040] Preferably, this method is compatible with three mainstream tower types in power systems: cylindrical cement towers, cylindrical metal towers, and high-voltage iron towers. It can also be seamlessly integrated with existing tower climbing safety protection devices and virtual reality tower climbing training systems. No large-scale modification of existing equipment is required; only the addition of status sensors and corresponding communication modules is needed to achieve the collection and analysis of multi-dimensional sensor data.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data in this invention addresses several shortcomings of existing technologies through multi-dimensional data fusion, adaptive data preprocessing, an improved Transformer model for fused features, a three-dimensional discriminant matrix, scenario-based linkage feedback, and model self-learning optimization. It has the following significant advantages:

[0043] It achieves two-dimensional data fusion and deep feature extraction of human limb movements and protective device status, with a comprehensive judgment accuracy of 99.0%, filling the technical gap in integrated analysis of "movement-protection" and reducing the safety hazard of "normal limb movements but ineffective protective devices" from the source.

[0044] The adaptive preprocessing method adapts to the action features of different rhythms, and the anomaly detection and compensation model ensures the authenticity and purity of the data, providing a high-quality data foundation for the high accuracy of the model.

[0045] The scenario-based hierarchical linkage feedback mechanism takes into account both the visual guidance of virtual reality training and the sound and light alarm warning of actual tower climbing operations, which not only improves the efficiency and pertinence of tower climbing training, but also realizes real-time safety monitoring of actual operations.

[0046] The model self-learning optimization step continuously adapts to the different operating habits and work scenarios of different operators through short-cycle incremental training, thereby achieving dynamic improvement in model performance and ensuring the long-term effectiveness of the method.

[0047] It can seamlessly integrate with existing virtual reality tower climbing training systems and tower climbing safety protection devices, requiring no large-scale modification of existing equipment. Only the addition of status sensors and communication modules is needed to achieve multi-dimensional data collection and analysis. It is suitable for tower climbing training and actual operation safety monitoring in various power operation and maintenance units, and has broad engineering application value. Attached Figure Description

[0048] Figure 1 This is a flowchart of a deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data, as proposed in this invention. Detailed Implementation

[0049] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0050] Example 1

[0051] Basic Integrated Version of Deep Learning Analysis Method for Tower Climbing Motion

[0052] This embodiment is a basic two-dimensional data fusion version, which only achieves basic data fusion of the status of human limbs and protective devices. It adopts a fixed data preprocessing method and a traditional Transformer model, and does not include data anomaly calibration and model self-learning optimization steps. The specific implementation steps are as follows:

[0053] Multi-dimensional sensor data acquisition: Six nine-axis Bluetooth sensors (left upper arm, right upper arm, left wrist, right wrist, left ankle, and right ankle) are deployed on the human limbs to collect limb acceleration, angular velocity, and angle data; one pressure sensor is deployed on the protective device collar and one on the trapezoidal movable block, for a total of two status sensors, to collect the contact pressure data of the protective device; the two types of data are time-stamped and synchronized, with a calibration error of ±0.05s, to form a basic multi-dimensional sensor dataset.

[0054] Fixed data preprocessing: All data were processed using a moving average filter with a fixed step size of 5; the three-axis summation was calculated for the nine-axis human body data; the protective device status data were not normalized and were directly spliced ​​into the human body data to achieve dimensional fusion; all data were segmented using a fixed window of 20 sampling points (80% overlap); double labeling was performed only based on pole type and limb movement, resulting in 19207 data slices, which were divided into training set, validation set, and test set in a 7:2:1 ratio.

[0055] Traditional Transformer model construction and training: The original Transformer is used as the base network, without a protective state feature fusion layer, and only a single temporal position encoding is used. The original multi-head attention mechanism (8 attention heads) is retained. The model's Encoder and Decoder each consist of 4 stacked sub-blocks, with Adam optimizer, learning rate of 0.001, batch size of 32, and 80 training epochs.

[0056] Two-dimensional motion standardization analysis: A two-dimensional discrimination matrix of pole type and limb motion is constructed, and only the similarity value of limb motion is used as the scoring basis (maximum score of 100 points), without including the status data of protective devices.

[0057] Single feedback: Whether it is virtual reality training or physical tower climbing operation, the single feedback method is voice, which only broadcasts the score of body movement, without giving specific correction suggestions, and there is no warning of the status of protective devices.

[0058] This embodiment is a simplified version, which only implements basic two-dimensional data fusion. It does not solve the problems of poor adaptability of fixed preprocessing, single model feature extraction, and limited discrimination dimensions. It is only suitable for basic tower climbing training with low precision requirements.

[0059] Example 2

[0060] Preprocessing Optimized Deep Learning Analysis Method for Tower Climbing Actions

[0061] This embodiment optimizes the data preprocessing method based on Embodiment 1, adds a data anomaly calibration step, still uses the traditional Transformer model, and does not achieve deep feature fusion of the protective device status and human movement. The specific implementation steps are as follows:

[0062] Multi-dimensional sensor data acquisition: The sensor deployment is the same as in Example 1, with 6 human nine-axis sensors + 2 protective device pressure sensors, and the timestamp synchronization calibration error is optimized to ±0.03s.

[0063] Multi-dimensional sensor data anomaly calibration: An isolated forest anomaly detection model was established to detect anomalies in the original data. A linear interpolation method was used to compensate for the anomaly data points. No zero-point drift compensation model was set. The anomaly data detection accuracy was about 85%.

[0064] Adaptive data preprocessing: Adaptive moving average filtering was used, with a climbing action step distance of 5 and a protective device operation step distance of 3. The three-axis summation value of the human body nine-axis data was calculated, and the protective device status data was normalized to achieve dimensional fusion. Categorical data segmentation was used, with 20 sampling points in the climbing action window and 15 sampling points in the protective device operation action window, with an overlap rate of 80% for both. Triple labeling (tower type - limb action - protective device status) was used to obtain 21,560 data slices, which were divided into datasets in a 7:2:1 ratio.

[0065] Traditional Transformer Model Construction and Training: The original Transformer model is used, with the same structure as in Example 1. The Encoder and Decoder each have 4 sub-blocks, the original multi-head attention mechanism is used, and the training parameters are the same as in Example 1.

[0066] Three-dimensional motion standardization analysis: Construct a three-dimensional discrimination matrix of pole type, limb motion, and protective device status. The comprehensive score is obtained by weighted summation of the similarity values ​​between human limb motion and protective device status (limb motion weight 0.7, protective device status weight 0.3).

[0067] Scenario-based single feedback: Virtual reality training scenarios use visual feedback (VR glasses display comprehensive scores), while physical tower climbing operation scenarios use voice feedback. Neither provides specific correction suggestions, and there are no audible or visual alarms when the protective devices are in abnormal condition.

[0068] This embodiment optimizes data preprocessing and discrimination dimensions, and improves the degree of data standardization. However, because it adopts the traditional Transformer model, it does not achieve deep feature fusion of human body movements and protective device status, and the model's feature extraction capability is limited. It is suitable for tower climbing training with medium accuracy requirements.

[0069] Example 3

[0070] A comprehensive and optimized deep learning analysis method for tower climbing actions

[0071] This embodiment achieves full-dimensional data fusion, adaptive data preprocessing, deep feature fusion, three-dimensional discrimination, scenario-based hierarchical linkage feedback, and model self-learning optimization. The specific implementation steps are as follows:

[0072] Multi-dimensional sensor data acquisition:

[0073] Human limb sensing: Six nine-axis Bluetooth sensors are deployed on the left upper arm, right upper arm, left wrist, right wrist, left ankle, and right ankle to collect XYZ three-axis acceleration, angular velocity, and angle data, and comprehensively acquire limb displacement, rotation, and motion posture data;

[0074] Protective device sensing: Five pressure sensors are deployed on the inner side of the protective device collar (2), the semi-circular frame clamping surface (2), and the trapezoidal movable block contact surface (1); three displacement sensors are deployed on the sliding block (1) and the moving frame (2); and four angle sensors are deployed on the semi-circular frame (2) and the manual lever (2), for a total of 12 status sensors to collect data on the protective device contact pressure, component displacement, and component rotation angle.

[0075] Timestamp calibration: All sensors are uniformly calibrated, with a timestamp synchronization calibration error of ≤±0.01s, ensuring the temporal correlation between human limb data and protective device status data, forming a full-dimensional sensor dataset.

[0076] Multi-dimensional sensor data anomaly calibration:

[0077] Anomaly detection and compensation: An isolated forest anomaly detection model is established to identify abnormal data points caused by sensor process deviations, limb tremors, and mechanical vibrations, with an anomaly detection accuracy of ≥95%; linear interpolation is used to compensate for abnormal data points and supplement missing feature values;

[0078] Zero drift correction: A zero drift compensation model is established to correct the zero drift deviation in real time based on the historical data collected by the sensor. After compensation, the data analysis error is ≤±5%, ensuring the authenticity and purity of the data.

[0079] Adaptive data preprocessing;

[0080] Adaptive moving average filtering: This method dynamically adjusts the step size based on the action type: 5 steps for climbing actions (ascending / descending towers), 3 steps for safety device operation actions (tying ropes / pressing manual levers), and 4 steps for safety device status data. This adjustment is achieved through a filtering algorithm. Remove clutter peaks and noise to enhance the prominence of data features;

[0081] Multi-dimensional data fusion: Calculate the sum of the XYZ axes from the acceleration, angular velocity, and angle data of the nine-axis human body sensor. ), supplement feature values; normalize the status data of protective devices to the [0,1] interval, and splice it to each row of human limb data to achieve deep fusion of two-dimensional data;

[0082] Triple labeling: Triple labeling was performed based on 3 types of towers, 32 types of body movements, and 8 types of protective device status, resulting in 21,560 data slices, which were divided into training set (15,092 slices), validation set (4,312 slices), and test set (2,156 slices) according to a ratio of 70%:20%:10%.

[0083] Construction and training of an improved Transformer model with fused features:

[0084] Model Improvement: Based on the Transformer network, three core improvements were made: ① A protective device state feature fusion layer was added, which concatenates the human limb feature vector (128-dimensional) and the protective device state feature vector (64-dimensional) into a 192-dimensional fused feature vector;

[0085] ② Design a dual-dimensional location coding module that encodes both the temporal location of human limb data and the associated location of "human body-protection" data, generating a coding matrix through a sine-cosine function;

[0086] ③ The original multi-head attention mechanism is improved into a ternary multi-head attention mechanism, which includes three parallel sub-modules: human action self-attention, protective device status self-attention, and action-protection linkage attention, with 8 attention heads, to achieve multi-dimensional feature deep extraction;

[0087] Model structure: The Encoder module consists of 6 stacked repeating sub-blocks (triple multi-head attention + feedforward neural network), and the Decoder module consists of 6 stacked repeating sub-blocks (masked triple multi-head attention + triple multi-head attention + feedforward neural network). The modules are connected through a residual network to ensure training stability.

[0088] Training parameters: Adam optimizer, learning rate 0.001, batch size 32, 100 training epochs, early stopping mechanism (training stops if the validation set loss does not decrease for 10 consecutive epochs) to avoid overfitting.

[0089] Multi-dimensional analysis of behavioral norms:

[0090] Three-dimensional discrimination matrix construction: A three-dimensional discrimination matrix is ​​constructed with tower type as the row dimension, limb action as the column dimension, and protective device status as the depth dimension. The matrix elements are the normative weight values ​​([0,1]) of action-state coordination, which are determined by statistical analysis of actual normative operation data.

[0091] Standard database establishment: For the three types of towers, a standard action database for tower climbing (including standard limb characteristics such as rope length / shortness, tower climbing, wearing a safety helmet, etc.) and a standard status database for protective devices (including standard status characteristics such as collar pressure, clamping angle, and sliding block displacement, etc.) are established respectively.

[0092] Similarity comparison and comprehensive scoring: The cosine similarity algorithm is used to calculate the similarity value between the real-time feature vector and the feature vector in the standard library. The comprehensive score is a weighted sum of the similarity value of limb movement (weight 0.6) and the similarity value of protective device status (weight 0.4), with a full score of 100 points. ≥80 points is considered standard, 60≤score<80 points is considered basically standard, and <60 points is considered non-standard.

[0093] Linked real-time feedback: Designed scenario-based hierarchical feedback mechanism to adapt to both virtual reality training and physical tower climbing operations / training scenarios:

[0094] Virtual reality training scenario: Trainees wear VR glasses, and the central control server displays the comprehensive score, limb movement problems, and protective device status problems in the VR glasses in a visual interface, and provides specific correction suggestions (such as "the left upper arm is not raised enough, it is recommended to adjust it to 60°" and "the collar pressure is insufficient, it is recommended to retighten it").

[0095] In physical tower climbing scenarios: no VR glasses are required; feedback, scoring, and correction suggestions are provided via voice module; an audible and visual alarm (red light + buzzer alarm) is immediately triggered when the protective device is not in compliance with regulations, prompting the worker to stop work; when body movements are not in compliance with regulations but the protective device is in a basically compliant state, only a voice reminder is given; all data is displayed in real time on the central control display screen, supporting remote monitoring.

[0096] Model self-learning optimization:

[0097] Iterative dataset construction: Store multi-dimensional sensor data, operation scenarios, and discrimination results from real-world applications to form a model iterative dataset;

[0098] Incremental training: The iteration cycle is 1 month. The preprocessed iterative dataset is input into the model for incremental training, and the weight values ​​of the 3D discriminant matrix and the feature data of the canonical library are dynamically adjusted.

[0099] Performance requirements: The model's discrimination accuracy should improve by ≥2% after each iteration, continuously adapting to different operators' operating habits and work scenarios.

[0100] This embodiment fully applies all the core technical solutions of the present invention, realizing the optimization of the entire process of "data-preprocessing-model-discrimination-feedback-optimization". It achieves the best results in terms of recognition accuracy, discrimination comprehensiveness, feedback real-time performance and model adaptability, and is suitable for professional training in power tower climbing and safety monitoring of actual tower climbing operations with high precision requirements.

[0101] Example 4

[0102] A simplified deep learning analysis method for tower climbing motion using sensors.

[0103] This embodiment, based on the optimal embodiment 3, simplifies the deployment number of protective device status sensors. The remaining model structure, preprocessing method, discrimination system, and feedback mechanism are consistent with embodiment 3. It is used to verify the impact of the number of sensors on model performance. The specific implementation steps are as follows:

[0104] Multi-dimensional sensor data acquisition: The six nine-axis sensors for human limbs are consistent with those in Example 3; the protective device status sensors are simplified to six (one collar, one semi-circular frame, one trapezoidal movable block pressure sensor, one sliding block, one moving frame displacement sensor, and one semi-circular frame angle sensor), reducing the number of status sensors by 50%; the timestamp synchronization calibration error is ±0.01s.

[0105] The steps and parameters for data anomaly calibration, adaptive data preprocessing, improved Transformer model construction and training, three-dimensional discrimination, linkage feedback, and model self-learning optimization are completely consistent with those of the optimal embodiment 3.

[0106] This embodiment reduces equipment deployment costs by simplifying the number of sensors, but the model's accuracy in recognizing the status of protective devices is reduced due to insufficient data collection dimensions of protective device status. It is suitable for cost-sensitive tower climbing training with medium accuracy requirements.

[0107] Example 5

[0108] Deep learning analysis method for slow-iteration tower climbing action model

[0109] This embodiment extends the model self-learning iteration cycle based on Embodiment 3. The remaining sensor deployment, preprocessing methods, model structure, discrimination system, and feedback mechanism are consistent with Embodiment 3. It is used to verify the impact of the model iteration cycle on model performance. The specific implementation steps are as follows:

[0110] The steps and parameters for multi-dimensional sensor data acquisition, data anomaly calibration, adaptive data preprocessing, improved Transformer model construction and training, three-dimensional discrimination, and linkage feedback are completely consistent with those of the optimal embodiment 3.

[0111] Model self-learning optimization: The model iteration cycle was extended from 1 month in Example 3 to 3 months, while the other incremental training methods and parameters remained the same as in Example 3.

[0112] Because the iteration cycle is extended, the model adapts to new work scenarios and the operating habits of new workers more slowly in this embodiment. After long-term use, the improvement in the model's discrimination accuracy is lower than that in embodiment 3. It is suitable for tower climbing training scenarios with fixed work scenarios and stable workers.

[0113] Experimental Data and Results Analysis

[0114] To verify that Example 3 is the optimal example, the above five examples were tested uniformly. Test indicators were designed from four dimensions: model recognition performance, data processing efficiency, anomaly detection capability, and model adaptive capability. Complete experimental data was obtained and compared and analyzed.

[0115] I. Definition of Test Metrics

[0116] Model recognition performance: The core evaluation indicators include the accuracy of human limb movement recognition, the accuracy of protective device status recognition, and the accuracy of comprehensive judgment of tower climbing actions. All are in percentage form, and the higher the value, the better the performance.

[0117] Data processing efficiency: This includes data preprocessing time (unit: ms / 100 data slices) and model inference time (unit: ms / single sample). The lower the value, the faster the processing speed.

[0118] Anomaly detection capability: The evaluation index is the accuracy rate of anomaly detection (percentage). The higher the value, the stronger the ability to identify anomalies.

[0119] Model adaptability: The evaluation metric (percentage) is the total improvement in the model's overall discrimination accuracy after three consecutive iterations. A higher percentage indicates a stronger ability of the model to adapt to new scenarios.

[0120] Complete experimental test data for five sets of examples.

[0121] Test Dimensions Test metrics Example 1 Example 2 Example 3 Example 4 Example 5 Model recognition performance Human body movement recognition accuracy rate (%) 89.2 94.5 98.5 95.8 98.5 Accuracy rate of protective device status identification (%) 82.1 88.7 99.2 92.3 99.2 Overall accuracy rate (%) 86.5 92.8 99 94.6 98.2 Data processing efficiency Data preprocessing time (ms / 100 slices) 12.56 15.89 16.23 15.98 16.23 Model inference time (ms / sample) 8.32 9.56 9.89 9.76 9.89 Anomaly detection capability Anomaly detection accuracy (%) - 85.3 96.8 96.5 96.8 Model Adaptability The overall accuracy of the discrimination was improved by (%) after three iterations. - - 6.5 6.3 2.1 Note: "-" indicates that the corresponding function is not set in this embodiment and there is no test data.

[0122] Analysis of Experimental Results

[0123] The experimental data above clearly shows that Example 3 performed best in all core evaluation dimensions, as detailed below:

[0124] Model recognition performance: In Example 3, the accuracy rates for human limb movement recognition (98.5%), protective device status recognition (99.2%), and overall discrimination accuracy (99.0%) were the highest among the five examples, representing improvements of 9.3%, 17.1%, and 12.5% ​​respectively compared to Example 1; 4.0%, 10.5%, and 6.2% respectively compared to Example 2; 2.7%, 6.9%, and 4.4% respectively compared to Example 4; and 0.8% higher overall discrimination accuracy than Example 5. This is because Example 3 achieved full-dimensional data fusion and deep feature extraction, with complete sensor deployment and optimized model structure, enabling it to fully capture the linkage features between human movements and protective device status.

[0125] Data processing efficiency: The data preprocessing time (16.23ms / 100 slices) and model inference time (9.89ms / single sample) of Example 3 are slightly higher than those of Examples 1, 2, and 4, but the difference is within 1ms, which is within an acceptable range. The reason for the slight increase in time is that Example 3 adopted adaptive preprocessing and an improved ternary multi-head attention mechanism, which increased the amount of computation slightly, but in return, it achieved a significant improvement in recognition accuracy, realizing the optimal balance between "efficiency and accuracy".

[0126] Anomaly detection capability: The anomaly detection accuracy of Example 3 (96.8%) is much higher than that of Example 2 (85.3%), and basically the same as that of Examples 4 and 5 (difference ≤0.3%). This indicates that the anomaly detection model of Example 3 can accurately identify various abnormal data points, and the data compensation and zero-point drift correction effects are good, ensuring the data quality of the input model and laying the foundation for high recognition accuracy.

[0127] Model Adaptability: The improvement in overall discrimination accuracy after three iterations in Example 3 (6.5%) was the highest among the five examples, 0.2% higher than Example 4 and 4.4% higher than Example 5. This is because Example 3 uses a short iteration cycle of one month, which can quickly integrate new data from actual applications into the model, dynamically adjust the discrimination matrix and standard library, and adapt to different work scenarios and operating habits. In contrast, Example 5 has a long iteration cycle, resulting in a significant decrease in model adaptation speed.

[0128] Furthermore, in Example 4, due to the reduction in the number of sensors in the protective device, the accuracy of the protective device status recognition decreased significantly by 6.9%, and the overall discrimination accuracy decreased by 4.4%, indicating that a complete deployment of status sensors is the foundation for achieving high-precision discrimination. In Example 5, due to the extended iteration cycle, the overall discrimination accuracy improved by only 2.1% after 3 iterations, which is far lower than that of Example 3, indicating that short-cycle model self-learning optimization is the key to ensuring long-term high performance of the model.

[0129] In summary, Example 3 achieves the best performance in terms of model recognition accuracy, anomaly detection capability, and model adaptation capability. Although the data processing efficiency is slightly improved, it is within an acceptable range. It achieves the optimal balance between performance and efficiency and is the best example of this invention.

[0130] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data, characterized in that, Includes the following steps: S1. Multi-dimensional sensor data acquisition: Nine-axis Bluetooth sensors are deployed on the limbs of tower climbers to collect human limb movement and posture data. Status sensors are deployed at the collar, clamping mechanism, and fastening mechanism of the tower climbing safety protection device to collect the working status data of the protection device. The two types of data are time-stamped and calibrated to form a multi-dimensional sensor dataset. S2. Adaptive data preprocessing: The multi-dimensional sensor dataset is subjected to adaptive moving average filtering, multi-dimensional data dimension expansion, categorical data segmentation, and multi-label dataset labeling in sequence to obtain standardized model training and testing samples. S3. Construction of a deep learning model for fusing features: Using Transformer as the base network, a protection device state feature fusion layer, a two-dimensional position encoding module and a three-element multi-head attention mechanism are added to construct a deep learning model for joint analysis of tower climbing action and protection device state. Standardized samples are input into the model to complete the training, resulting in an action analysis model with multi-dimensional feature extraction capabilities. S4. Multi-dimensional action standardization analysis: Construct a three-dimensional discrimination matrix of tower type, limb action, and protective device status, establish a tower climbing standard action library and a protective device standard status library corresponding to different tower types, input the real-time collected and preprocessed multi-dimensional sensor data into the trained model, output the limb action category and the protective device status category, and complete the comprehensive standardization score of tower climbing action by comparing the similarity with the standard library. S5. Real-time Linkage Feedback: Based on the application scenario, the scores for the standardization of limb movements, the status scores of protective devices, and the corresponding correction suggestions are linked and fed back through visual / voice / audio-visual alarms to achieve real-time correction of the operator's movements.

2. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 1, characterized in that, The nine-axis Bluetooth sensor mentioned in step S1 is an integrated sensor that includes XYZ three-axis acceleration, XYZ three-axis angular velocity, and XYZ three-axis angle. A total of 6 sensors are deployed and worn on the left upper arm, right upper arm, left wrist, right wrist, left ankle, and right ankle. The acquisition frequency is 10Hz, which can acquire the displacement, rotation, and motion posture data of the human body in real time. The status sensors include pressure sensors, displacement sensors, and angle sensors. The pressure sensors are deployed on the inner side of the collar of the protective device, the clamping surface of the semi-circular frame, and the contact surface of the trapezoidal movable block to collect contact pressure data between the protective device and the tower foot spikes. The displacement sensors are deployed on the sliding block and the moving frame of the protective device to collect displacement data of the components of the protective device. The angle sensors are deployed on the semi-circular frame and the manual lever of the protective device to collect rotation angle data of the components of the protective device. The acquisition frequency of the status sensors is consistent with that of the nine-axis Bluetooth sensor, and all sensors are equipped with Bluetooth communication modules to transmit the collected data to the computing server in real time. The timestamp synchronization calibration unifies the acquisition clocks of all sensors, ensuring that human limb data and protective device status data acquired at the same time point have the same timestamp, guaranteeing the temporal correlation of the data, and controlling the calibration error within ±0.01s.

3. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 2, characterized in that, The adaptive moving average filtering described in step S2 abandons the fixed step distance filtering method of the existing technology and dynamically adjusts the moving step distance according to the action type characteristics of the sensor data: for the climbing action of human limbs, because the movement amplitude is large and the rhythm is slow, the moving step distance is set to 5, and one position is moved in the same direction each time. For the operation of protective devices for human limbs, due to the small range of motion and fast pace, the step distance is set to 3, and each time it moves one position in the same direction; For the status data of the protective device, due to the small data fluctuation and high stability requirements, the movement step size is set to 4, and it moves one position in the same direction each time. The filtering algorithm formula is: ,in Here, n represents the raw sensor data, n is the dynamically adjusted step size, and t is the current time point. The filtered data is processed by this algorithm to remove clutter peaks and noise interference, making the characteristics of multi-dimensional sensor data more prominent.

4. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 3, characterized in that, Step S2, the multi-dimensional data dimensional expansion, includes two parts: first, calculating the sum of the acceleration, angular velocity, and angle data from the nine-axis sensor of the human limb along the XYZ axes; the formula for calculating the sum is as follows: Where X, Y, and Z are the original data for each dimension, A is the sum value, and supplementary feature values ​​describe only the size of the data and not its direction. Secondly, the status data of the protective device is normalized, and the contact pressure, component displacement, and component rotation angle data are mapped to the [0,1] interval. Then, the normalized status data of the protective device is added to each row of the human limb data to achieve dimensional fusion of the two-dimensional data. The categorized data segmentation sets different sliding window sizes according to the action type: for human limb climbing action, since the action completion time is about 2 seconds, combined with the 10Hz acquisition frequency, the sliding window size is set to 20 sampling points. For the operation of human limb protective devices and changes in the status of the protective devices, since the action completion time is about 1.5 seconds, the sliding window size is set to 15 sampling points; The window overlap rate for all data segmentation is set to 80% to ensure that each segment contains complete action and state information and to avoid feature loss. The multi-label dataset annotation combines tower type, human limb movement type, and protective device status type for triple label annotation, ultimately obtaining a dataset of no less than 19,207 data slices, which are then divided into training set, validation set, and test set according to a ratio of 70%, 20%, and 10%, respectively.

5. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 1, characterized in that, The improved Transformer deep learning model described in step S3 is improved in three aspects based on the original Encoder-Decoder structure: First, a protective device state feature fusion layer is added after the Embedding module to concatenate and fuse the human limb feature vector and the protective device state feature vector. The fused vector dimension is the sum of the human limb feature vector dimension and the protective device state feature vector dimension, thus achieving the initial fusion of two-dimensional features. Second, the location coding module is improved by adopting a two-dimensional location coding, which not only encodes the temporal location information of human limb data, but also encodes the correlation location information between the protective device status data and human limb data. The location coding matrix is ​​generated by a sine-cosine function, with even columns activated by a sine function and odd columns activated by a cosine function, thus making up for the defect of Transformer network that cannot directly obtain the location relationship of temporal data. Third, the original multi-head attention mechanism is improved into a ternary multi-head attention mechanism, which includes a human body action self-attention sub-module, a protective device state self-attention sub-module, and an action-protective device association attention sub-module. The three sub-modules work in parallel to extract the temporal features of human limb actions, the change features of the protective device state, and the linkage features between human actions and the protective device state, respectively. Then, the output features of the three sub-modules are concatenated through the Concat operation to achieve deep extraction of multi-dimensional features. The Encoder module of the model consists of six stacked repeating sub-blocks, each containing a ternary multi-head attention mechanism and a feedforward neural network. The Decoder module consists of six stacked repeating sub-blocks, each containing a masked ternary multi-head attention mechanism, a ternary multi-head attention mechanism, and a feedforward neural network. The modules are connected through residual networks to ensure the training stability of the model.

6. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 1, characterized in that, The three-dimensional discrimination matrix mentioned in step S4 uses the tower type as the row dimension, the human limb action type as the column dimension, and the protective device status type as the depth dimension. Each element in the matrix is ​​the standard weight value of the coordination between a certain human limb action and a certain status of the protective device under the corresponding tower type. The weight value is determined through statistical analysis of a large amount of actual standard operation data, and the value range is [0,1]. The closer the weight value is to 1, the more standard the coordination between the action and the status is. The tower climbing standard action library and the protective device standard status library are constructed separately for three types of towers: cement towers, steel pipe towers, and angle iron towers. The standard action library contains standard limb action characteristic data for operations such as climbing up and down the tower, tying long ropes, tying short ropes, and wearing safety helmets for each type of tower. The standard status library contains standard status characteristic data for protective devices such as collar fastening pressure, semi-circular frame clamping angle, trapezoidal movable block abutment pressure, and sliding block displacement for each type of tower. The similarity comparison uses a cosine similarity algorithm to calculate the cosine similarity value between the real-time feature vector output by the model and the corresponding feature vector in the standard library. The closer the similarity value is to 1, the closer the real-time action / state is to the standard. The comprehensive standardization score is obtained by weighted summation of the similarity value of human limb action and the similarity value of the protective device state. The weight coefficient is determined according to the degree of influence of the two on the safety of climbing the tower. The weight coefficient of human limb action is set to 0.6, and the weight coefficient of protective device state is set to 0.

4. The comprehensive score is out of 100 points. A score ≥ 80 is considered standard, 60 ≤ score < 80 is considered basically standard, and a score < 60 is considered non-standard.

7. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 1, characterized in that, The real-time feedback linkage mentioned in step S5 is designed according to two application scenarios: one is a virtual reality tower climbing training scenario, where operators wear VR glasses and the central control server displays the standardization score of human limb movements and the status score of protective devices in the VR glasses in the form of a visual interface. At the same time, it provides specific correction suggestions for non-standard limb movements and protective device status, such as "the left upper arm is not raised enough, please adjust to 60°" and "the contact pressure of the protective device collar is not enough, please retighten". Operators can correct in real time while practicing. Secondly, in the physical tower climbing operation / training scenario, the operator does not need to wear VR glasses. The system provides the operator with a comprehensive score and correction suggestions through the voice module. When the protective device is in a non-standard state, the sound and light alarm module is immediately triggered, emitting a red light and a buzzer alarm sound to remind the operator to stop climbing the tower immediately and correct the protective device state. When the human body movement is non-standard and the protective device state is basically standard, only a voice reminder is issued to achieve hierarchical feedback. Meanwhile, all scoring data and feedback information are displayed in real time on the central control server's screen, facilitating remote monitoring by administrators.

8. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 1, characterized in that, It also includes a model self-learning optimization step: storing the multi-dimensional sensor data, corresponding operation scenarios and discrimination results collected during actual application to form a model iteration dataset, periodically preprocessing the iteration dataset according to the method of claim 4, and inputting it into the trained improved Transformer model for incremental training. At the same time, according to the standard requirements of actual operation, the weight values ​​of the three-dimensional discrimination matrix and the feature data of the standard library are dynamically adjusted to optimize the model's feature extraction capability and discrimination accuracy, so that the model can adapt to the operating habits of different operators and tower climbing operation scenarios in different environments. The model iteration cycle is set to 1 month.

9. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to claim 1, characterized in that, Between steps S1 and S2, there is also a multi-dimensional sensor data anomaly calibration step: establish an anomaly detection model based on isolated forest, perform anomaly point detection on the collected raw multi-dimensional sensor data, and identify abnormal data points caused by sensor process deviation, worker limb tremors, and mechanical vibration of protective devices. For the detected abnormal data points, linear interpolation is used to compensate for the data and supplement the missing feature values; To address the sensor drift problem, a zero-point drift compensation model is established. Based on the sensor's historical data, the zero-point deviation of the sensor is corrected in real time to ensure the authenticity and purity of multi-dimensional sensor data. The detection accuracy of abnormal data points is no less than 95%, and the error of the compensated data analysis is controlled within ±5%.

10. The deep learning analysis method for tower climbing actions that integrates multi-dimensional sensor data according to any one of claims 1-9, characterized in that, This method is applicable to three mainstream tower types in power systems: cylindrical cement towers, cylindrical metal towers, and high-voltage iron towers. It can also be seamlessly integrated with existing tower climbing safety protection devices and virtual reality tower climbing training systems. No large-scale modification of existing equipment is required; only the addition of status sensors and corresponding communication modules is needed to achieve the collection and analysis of multi-dimensional sensor data.

Images