An aerial work accident early warning method and device based on a hybrid neural network model, a terminal device, and a storage medium

Abstract

This invention discloses a method, device, terminal equipment, and storage medium for early warning of high-altitude work accidents based on a hybrid neural network model, belonging to the field of high-altitude work safety technology. The method comprises: acquiring time-series data from a sensor group; inputting the time-series data into a hybrid neural network model to output the current work scenario and current action category; inputting the time-series data, the current work scenario, and the current action category into a state machine to extract the current safety judgment threshold combination corresponding to the current work scenario and the current action category, and outputting the current safety status judgment result based on the time-series data and the current safety judgment threshold combination; and generating an accident warning based on the current safety judgment result. Therefore, by implementing this invention, the problem of insufficient accuracy in accident warnings in existing technologies can be solved.

Description

Technical Field

[0001] This invention relates to the field of high-altitude operation safety technology, and in particular to a method, device, terminal equipment and storage medium for early warning of high-altitude operation accidents based on a hybrid neural network model. Background Technology

[0002] In high-altitude power operations, safety belts are key protective equipment to ensure the personal safety of workers. In high-altitude work scenarios such as power poles and scaffolding, workers not only have problems with the use of protective equipment such as not wearing safety belts or fastening them improperly, but are also prone to dangerous behaviors such as loss of body balance, violent swaying, and instability due to shift in the center of gravity.

[0003] In existing technologies, the monitoring of safety status for high-altitude operations mainly relies on comparing real-time sensor data with fixed thresholds. Since sensor data can vary greatly under different work scenarios and different action categories, judging safety status based on fixed thresholds is prone to misjudgment, thus resulting in insufficient accuracy in accident warnings. Summary of the Invention

[0004] This invention provides a method, device, terminal equipment, and storage medium for early warning of high-altitude operation accidents based on a hybrid neural network model, which can solve the problem of insufficient accuracy of accident early warning in the prior art.

[0005] The high-altitude operation accident early warning method based on a hybrid neural network model provided by the present invention includes: acquiring sensor group time series data within the current time window; The time series data of the sensor group within the current time window is input into the hybrid neural network model so that the hybrid neural network model outputs the current work scenario and the current action category. The hybrid neural network model uses the time series data of the sensor group as training samples, the work scenario and action category corresponding to the training samples as sample labels, and the predicted work scenario and predicted action category as outputs. It is trained iteratively until the loss function converges. Input the sensor group timing data, current work scenario, and current action category within the current time window into the state machine, so that the state machine can extract the current safety judgment threshold combination corresponding to the current work scenario and current action category, and output the current safety status judgment result based on the sensor group timing data and current safety judgment threshold combination within the current time window. An accident warning is generated based on the current safety assessment results.

[0006] Furthermore, acquiring the sensor group time-series data within the current time window includes: Data from the sensor group is continuously collected according to a preset time sliding window; Based on a preset reference timing sequence, the data of the sensor group is calibrated to obtain the timing data of the sensor group within the current time window; The data from the sensor array includes: torso tilt angle, angular velocity, integral acceleration value, buckle status code, position data, and vibration amplitude.

[0007] Furthermore, the hybrid neural network model includes: a convolutional neural network, a recurrent neural network, a task scene classification head, and an action category classification head; The step of inputting the time-series data of the sensor group within the current time window into the hybrid neural network model, so that the hybrid neural network model outputs the current work scenario and the current action category, includes: Based on a convolutional neural network, feature vectors at each time step are extracted from the time-series data of the sensor group to generate spatial features at each time step. Based on a recurrent neural network, spatial features are recursively encoded time-by-time to obtain the spatiotemporal features within the current time window. The task scenario classification head calculates the probability distribution of the task scenario based on the spatiotemporal characteristics within the current time window, and outputs the current task scenario based on the probability distribution; where the task scenario is a pole, platform, or scaffold. Using the action category classification head, the probability distribution of action categories is calculated based on the spatiotemporal characteristics within the current time window, and the current action category is output based on the action category probability distribution; where the action category is climbing operation, moving and changing point operation, standing still operation, or leaning backward operation.

[0008] Furthermore, based on the combination of sensor group time-series data within the current time window and the current safety judgment threshold, the current safety status judgment result is output, including: Extract the torso tilt angle, angular velocity, integral acceleration value, and buckle status code of the last moment within the current time window; If the torso tilt angle is less than the corresponding current torso tilt angle safety threshold, the angular velocity is less than the current angular velocity safety threshold, the integral value of acceleration is less than the current integral value of acceleration safety threshold, and the buckle status code is a closed code, then the current safety status determination result is determined to be safe operation; If the torso tilt angle is less than the corresponding current torso tilt angle safety threshold, the angular velocity is less than the current angular velocity safety threshold, the acceleration integral value is less than the current acceleration integral safety threshold, and the buckle status code is an open code, then the current safety status determination result is that there is a risk of illegal fastening. Otherwise, the current safety status is determined to be that there is a risk of a fall.

[0009] Furthermore, based on the current safety assessment results, an accident warning is generated, including: When the current safety status is determined to be safe for operation, a first prompt message is generated to characterize the safe operation. When the current safety status assessment result indicates that there is a risk of illegal mooring, a second prompt message is generated to indicate the existence of the risk of illegal mooring. When the current safety status assessment result indicates that there is a risk of a fall accident, a third warning message is generated to indicate that there is a risk of a fall accident.

[0010] Furthermore, it also includes: After outputting the current safety status determination result, extract the location data of the last moment within the current time window as the real-time location; Based on the real-time location, obtain image information of the real-time location; Based on real-time location image information, the current security assessment result is reviewed.

[0011] Another embodiment of the present invention provides a high-altitude operation accident early warning device based on a hybrid neural network model, comprising: a data acquisition module, a classification module, a status determination module, and an accident early warning module; The data acquisition module is used to acquire the time series data of the sensor group within the current time window; The classification module is used to input the time-series data of the sensor group within the current time window into the hybrid neural network model, so that the hybrid neural network model outputs the current work scenario and the current action category; wherein, the hybrid neural network model uses the time-series data of the sensor group as training samples, uses the work scenario and action category corresponding to the training samples as sample labels, and uses the predicted work scenario and predicted action category as outputs, and performs iterative training until the loss function converges; The state determination module is used to input the sensor group time-series data, the current work scenario, and the current action category within the current time window into the state machine, so that the state machine can extract the current safety determination threshold combination corresponding to the current work scenario and the current action category, and output the current safety state determination result based on the sensor group time-series data and the current safety determination threshold combination within the current time window. The accident warning module is used to generate accident warnings based on the current safety assessment results.

[0012] Furthermore, it also includes: a judgment review module; The judgment and verification module is used to extract the location data of the last moment in the current time window as the real-time location after outputting the current security status judgment result; Based on the real-time location, obtain image information of the real-time location; Based on real-time location image information, the current security assessment result is reviewed.

[0013] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the high-altitude operation accident early warning method based on a hybrid neural network model provided by the present invention.

[0014] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the high-altitude operation accident early warning method based on a hybrid neural network model provided by the present invention.

[0015] The following benefits can be obtained by implementing the present invention: This invention discloses a high-altitude operation accident early warning method based on a hybrid neural network model. The method first acquires data from each sensor in the current time window, sorted by time. Then, it inputs the current time-series data into a hybrid neural network model so that the hybrid neural network model outputs the current operation scenario and the current action category. Subsequently, the state machine loads the corresponding safety judgment threshold combination according to the current operation scenario and the current action category, and performs a safety status judgment based on the current time-series data and the safety judgment threshold set. Finally, an accident early warning is generated based on the safety judgment result. Compared with existing methods, the hybrid neural network model is iteratively trained using time-series data pre-labeled with work scenarios and action categories until the loss function converges. The trained hybrid neural network model can distinguish different work scenarios and action categories based on different sensor data. The state machine then extracts corresponding judgment threshold combinations based on the classification results to determine the safety status. For example, in the tower scenario, the acceleration integral thresholds for climbing and moving / changing points are different; similarly, the maximum backward tilt angle thresholds for stationary standing in the scaffolding scenario and stationary standing in the platform scenario are different. This invention can adaptively select judgment thresholds based on the identified work scenario and action category, reducing misjudgments and improving the accuracy of accident warnings. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a high-altitude operation accident early warning method based on a hybrid neural network model provided in an embodiment of the present invention. Figure 2This is a schematic diagram of the structure of a high-altitude operation accident early warning device based on a hybrid neural network model provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0020] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0023] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0024] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0025] See Figure 1 To address the insufficient accuracy of accident early warning in existing technologies, an embodiment of the present invention provides a high-altitude operation accident early warning method based on a hybrid neural network model, comprising: 101. Obtain the timing data of the sensor group within the current time window.

[0026] In a preferred embodiment, acquiring the sensor group time-series data within the current time window includes: Data from the sensor group is continuously collected according to a preset time sliding window; Based on a preset reference timing sequence, the data of the sensor group is calibrated to obtain the timing data of the sensor group within the current time window; The data from the sensor array includes: torso tilt angle, angular velocity, integral acceleration value, buckle status code, position data, and vibration amplitude.

[0027] The sensor group includes: an attitude sensor, a buckle status sensor, a position sensor, and a vibration sensor. The attitude sensor, comprising a triaxial accelerometer and a triaxial gyroscope, is installed on the webbing of the safety belt on the worker's back and can collect torso tilt angle, angular velocity, and integral acceleration values ​​in real time. The buckle status sensor, using microswitches or Hall effect sensors, is installed at the safety belt hook and buckle to monitor the opening and closing status of the hook and encodes the buckle status using a preset coding rule, recording the current buckle status code in real time. The position sensor includes a satellite positioning module to record the worker's position data in real time. The vibration sensor includes a vibration-sensitive detection chip to sense real-time vibration amplitude. The sampling frequency of each sensor can be set according to actual needs and sensor model. In this embodiment, the sampling frequency of the attitude sensor is 50Hz, and the sampling frequency of the buckle status sensor is 10Hz to ensure data real-time performance and accuracy. The real-time sensing data collected by the sensor group can be transmitted to the microprocessor on the edge side via an internal bus.

[0028] Schematic, the sensor array may also include a human proximity sensor, which uses a capacitive or inductive proximity switch to help determine whether the hook is close to the iron hanging point component; the vibration sensor may also collect vibration acceleration to characterize high-frequency instantaneous vibration components, specifically to capture impact vibrations caused by bumps, collisions, and violent shaking; collect vibration frequency to characterize the speed of reciprocating vibration, distinguishing between slight shaking, violent impact, and machine resonance; and collect vibration pulse signals to characterize the instantaneous pulse vibration waveforms caused by sudden impacts and accidental falls.

[0029] Specifically, according to a set sliding time window, real-time monitoring data from various sensors deployed on the worker is continuously collected. Since the sampling frequencies of the sensors in the sensor group differ, a standard sampling time axis is defined within the time window based on a preset reference time sequence. This standard sampling time axis has a corresponding reference sampling frequency. For sensors with sampling frequencies lower than the reference frequency, a linear interpolation algorithm is used to calculate and fill in missing data corresponding to adjacent standard times based on two consecutive valid measured values, thus filling in time sequence gaps. For sensors with sampling frequencies higher than the reference frequency, an equal-interval downsampling method is used, extracting valid sampling points at fixed steps, eliminating redundant and duplicate data, and adapting the data sampling frequency to the reference time sequence scale. After downsampling and interpolation completion processing, all sensing parameters can be mapped to the same timestamp, and at the same moment, all monitoring information, including torso tilt angle, angular velocity, acceleration integral value, buckle status code, position data, and vibration amplitude, can be synchronously matched, ultimately generating time-aligned and dimensionally complete sensing time sequence data within the current time window.

[0030] In the above data acquisition steps, multiple types of sensor monitoring information are continuously collected through a fixed-duration sliding window, enabling real-time capture of raw data related to worker posture, protective equipment, and the work environment. To address the data timing misalignment caused by varying sampling frequencies of different sensors, a unified benchmark timing framework is constructed, defining a standardized sampling time axis as a unified reference standard for all data alignment. For low-frequency sensor data, a linear interpolation algorithm is used to extrapolate missing time data based on preceding and following measured values, filling in timing gaps and ensuring the continuity and completeness of the data's time dimension. High-frequency sensor data undergoes equal-interval downsampling processing, with valid data samples selected according to a predetermined step size, and redundant information removed, ensuring that the frequencies of both types of data are uniformly matched to the benchmark scale. After differential calibration processing, various... Monitoring parameters are precisely mapped to the same timestamp, enabling synchronous matching of multi-dimensional data such as torso tilt angle, angular velocity, acceleration integral, buckle status, position information, and vibration amplitude. This processing method effectively eliminates frequency domain differences in inter-frequency data and removes data distortion caused by timing deviations, resulting in well-ordered, dimensionally complete, and time-aligned window sensing time-series data. The aligned data can accurately and coherently reflect the real-time protective status and limb movement changes of workers, eliminating defects such as data mismatch, missing data, and redundancy. This provides a high-quality and highly consistent basic data source for subsequent hybrid neural networks to perform spatial and temporal feature extraction, scene action classification and recognition, and state machine safety threshold comparison and judgment, ensuring the accuracy and reliability of subsequent intelligent analysis and risk assessment results.

[0031] 102. Input the time series data of the sensor group within the current time window into the hybrid neural network model so that the hybrid neural network model outputs the current work scenario and the current action category; wherein, the hybrid neural network model uses the time series data of the sensor group as training samples, the work scenario and action category corresponding to the training samples as sample labels, and the predicted work scenario and predicted action category as output, and performs iterative training until the loss function converges.

[0032] The loss function is the cross-entropy loss.

[0033] Specifically, in the application phase, the time-series data of the sensor group within the current time window is input into the trained hybrid neural network model, enabling the model to output the current work scenario and action category. In the training phase, multiple sets of actual high-altitude work sample data are first collected. The time-series data of the sensor group, after time alignment and normalization, is used as the model's training samples. Simultaneously, based on actual working conditions, each set of sample data is labeled with corresponding tags, clearly matching three types of work scenario tags (towers, platforms, and scaffolding) and four types of action category tags (climbing, moving and changing points, standing still, and leaning backward). During training, sample data is fed into the hybrid neural network model in batches, and the model relies on convolution... The neural network extracts spatial features at a single time step, then encodes temporal features via a recurrent neural network, and finally outputs the prediction results of the work scene and action category through a dual-classification head. The system uses the cross-entropy loss function to calculate the deviation difference between the predicted category and the actual label in real time. Based on the deviation value, the internal weight parameters of the network are corrected layer by layer through the backpropagation mechanism. Multiple rounds of iterative training are carried out repeatedly to continuously reduce the error between the prediction result and the actual label. When the value of the cross-entropy loss function gradually stabilizes and no longer shows significant fluctuations, the model loss is determined to have converged. At this time, the network has fully learned the sensor data feature patterns corresponding to different scenes and different actions, the model training process ends, and it has the ability to formally recognize and classify.

[0034] Furthermore, the parameters collected by this invention include six categories: torso tilt angle, angular velocity, acceleration integral, buckle status code, position data, and vibration amplitude. The physical characterization characteristics of each type of parameter have clear distinguishability. At the operational scenario level: Pole tower operations involve narrow spaces, resulting in a large torso inclination angle during climbing. Angular velocity and acceleration exhibit periodic small fluctuations, while vibration amplitude remains stable. The locking status often switches according to the movement pattern. Platform operations have limited activity ranges, with personnel often remaining stationary or slightly leaning back. Inclination angle fluctuations are minimal, motion sensor parameters change smoothly, and the overall data pattern tends to be stable with clear boundaries for positional data changes. Scaffolding operations involve frequent changes in work points, resulting in greater body swaying, significantly wider fluctuations in inclination angle, angular velocity, and vibration amplitude, and a higher frequency of positional data changes. At the level of movement and behavior: climbing movements produce continuous and regular acceleration and changes in posture angle; moving and changing points are accompanied by positional shifts and limb swaying, with significant fluctuations in vibration parameters; in a static standing state, various motion sensor parameters remain basically constant; backward leaning movements will show tilt angle shifts within a fixed range, with the amplitude of motion fluctuations within a specific range. Different scenarios and action combinations will result in unique numerical distributions, change rhythms, and correlation ratios of multiple sensor parameters. Convolutional neural networks (CNNs) can capture the coupled correlation features of multiple parameters at the same time, while recurrent neural networks (RNNs) can learn the temporal patterns of parameter evolution over time. By relying on a large number of sample iterations, the model can summarize and distinguish the inherent patterns of data under various working conditions, thereby achieving accurate recognition and classification of scenarios and actions.

[0035] To illustrate, data augmentation and transfer learning techniques can be incorporated into the model training phase to adapt to the interference caused by complex on-site conditions and effectively improve the model's overall generalization performance. At the data augmentation level, diverse augmentation processing is performed on the collected sensor time-series sample data. Gaussian simulated noise is superimposed on time-series parameters such as torso tilt angle, angular velocity, acceleration integral, and vibration amplitude to replicate the real sensor noise conditions caused by electromagnetic interference and equipment vibration. A time-warp transformation is used to scale and adjust the original action time-series sequence and make local time-series fine-tuning to simulate individual behavioral differences among workers with varying paces and deviations in limb movement amplitude. After augmentation, the samples cover various action deformations, signal disturbances, and individual behavioral characteristics, allowing the model to fully learn diverse action forms and noisy data features, reducing... The limitations of single-sample mode in recognition can be addressed through transfer learning. First, basic features are pre-trained using a publicly available general high-altitude operation sensor dataset, allowing the network to initially learn common features related to posture, motion, and protective status. Then, the pre-trained network weight parameters are transferred to the proprietary model of this invention. Fine-tuning and optimization are performed using actual measured and labeled samples from poles, platforms, and scaffolding on-site. Leveraging existing mature features reduces reliance on samples specific to the scenario, while adapting to the behavioral differences of different work sites and personnel. This significantly enhances the model's ability to adapt to sudden changes in action, abnormal sensor signals, and variations in personnel behavior, avoiding recognition bias in practical applications. It ensures that the model consistently and accurately outputs work scenario and action category results under various real-world work conditions, laying a solid foundation for subsequent safety status assessment and accident early warning.

[0036] In the classification step, the hybrid neural network model, after offline training and convergence, can be formally deployed for actual recognition applications at high-altitude work sites. The on-site system continuously collects monitoring data from sensors worn by workers, and after time-series calibration and frequency unification, standard time-series samples are input into the model for intelligent analysis. Based on the data feature patterns learned during the training phase, the model first uses a convolutional neural network to capture the coupling relationship of parameters such as torso tilt angle, angular velocity, acceleration integral, buckle status, position, and vibration amplitude within a single time step, extracting the spatial features of instantaneous human posture and equipment state. Then, a recurrent neural network is used to perform time-series encoding on the continuous feature sequence, completely replicating the dynamic changes and rhythms of limb movements, and fusing to generate global spatiotemporal features that combine spatial morphology and temporal evolution information. Subsequently, dual classification heads simultaneously perform discrimination operations. The system calculates the probability distribution of work scenarios and action categories based on learned feature patterns, quickly determining whether the current working condition belongs to any scenario such as poles, platforms, or scaffolding. It also accurately distinguishes four types of work behaviors: climbing, moving to change points, standing still, and leaning back. Thanks to the inherent differences in numerical ranges, fluctuation amplitudes, and rhythms of sensor data under different working conditions, the model can stably distinguish the unique data patterns corresponding to various scenarios and actions, effectively avoiding misjudgments caused by single-parameter discrimination. The entire recognition process is automated, providing real-time feedback on the actual working status and outputting reliable scenario and action recognition results. This provides accurate and effective preliminary evidence for subsequent dynamic retrieval of safety judgment thresholds by the state machine, assessment of work safety levels, and triggering risk warnings, ensuring the efficient and reliable operation of high-altitude work safety monitoring and judgment.

[0037] In a preferred embodiment, the hybrid neural network model includes: a convolutional neural network, a recurrent neural network, a task scenario classification head, and an action category classification head; The step of inputting the time-series data of the sensor group within the current time window into the hybrid neural network model, so that the hybrid neural network model outputs the current work scenario and the current action category, includes: Based on a convolutional neural network, feature vectors at each time step are extracted from the time-series data of the sensor group to generate spatial features at each time step. Based on a recurrent neural network, spatial features are recursively encoded time-by-time to obtain the spatiotemporal features within the current time window. The task scenario classification head calculates the probability distribution of the task scenario based on the spatiotemporal characteristics within the current time window, and outputs the current task scenario based on the probability distribution; where the task scenario is a pole, platform, or scaffold. Using the action category classification head, the probability distribution of action categories is calculated based on the spatiotemporal characteristics within the current time window, and the current action category is output based on the action category probability distribution; where the action category is climbing operation, moving and changing point operation, standing still operation, or leaning backward operation.

[0038] Specifically, the time-aligned window sensing time-series data is fed into a hybrid neural network model. The model sequentially performs feature parsing and category determination through a convolutional neural network, a recurrent neural network, and a two-branch classification head. First, the convolutional neural network performs spatial feature extraction operations. For the multi-dimensional sensing data corresponding to each sampling moment within the time window, it mines the correlation between different monitoring parameters and the physical structure status information of the workers, generating spatial feature vectors for each time step, transforming the raw numerical information into spatial features that can represent the instantaneous work posture. Then, the recurrent neural network takes over the sequence features, recursively encoding the spatial features of all moments frame by frame according to the chronological order. Through the network memory mechanism, it connects the changes in actions before and after, integrating the entire time segment. The dynamic evolution patterns within the scope are integrated to obtain global spatiotemporal features encompassing spatial state and temporal change information. After feature aggregation, two independent classification branches are separated to perform discrimination operations. The work scenario classification head reads the global spatiotemporal features, calculates the probability distribution of various scenarios through feature mapping operations, compares the probability magnitudes, and finally determines whether the current work belongs to one of the following work scenarios: pole, platform, or scaffolding. The action category classification head simultaneously completes probability inference based on the same spatiotemporal features, and, combined with the posture patterns of the workers reflected by the features, distinguishes and identifies four types of behavioral states: climbing work, moving and changing points work, standing still work, and leaning back work. Finally, it simultaneously outputs two identification results: the current work scenario and the work action category, providing a basis for subsequent safety status assessment.

[0039] In the application phase of the model, the hierarchical feature parsing and dual-branch discrimination process can fully explore the state information hidden in multi-source sensor data, achieving efficient and accurate identification of work scenarios and actions. The convolutional neural network focuses on the correlation characteristics of multi-dimensional parameters at a single moment, breaking through the limitations of single-data-dimensional judgment. It transforms discrete raw monitoring values ​​such as tilt angle, angular velocity, and buckle status into abstract spatial features that can characterize body posture and equipment condition, accurately depicting instantaneous differences in work forms. The recurrent neural network, relying on temporal recursive encoding capabilities, connects the inherent correlations of features at different time steps, fully capturing dynamic patterns such as movement rhythm and posture change trends, integrating discrete instantaneous features into unified global spatiotemporal features, and completely reconstructing the continuous behavioral changes of workers over a period of time. After the dual-classification head is shared and fused... The system independently discriminates based on spatiotemporal characteristics, ensuring the integrity of feature information utilization while adapting to two different judgment logics: scene environment recognition and body movement recognition. By comparing probability distributions and selecting the optimal category result, it can effectively distinguish between three types of work site environments and four types of typical work behaviors, adapting to the ever-changing actual working conditions of high-altitude operations. The entire recognition process operates automatically and continuously without manual intervention, and can quickly output reliable scene and movement recognition conclusions. The accurate recognition results can truthfully reflect the actual working conditions on site, providing core basis for subsequent state machine retrieval and matching of safety judgment thresholds, comparison of sensor data parameters, and assessment of work safety levels. This ensures that subsequent safety status assessments and risk warnings are reasonable and reliable, improving the practicality and reliability of the entire high-altitude operation accident early warning scheme.

[0040] 103. Input the sensor group timing data, current work scenario, and current action category within the current time window into the state machine, so that the state machine can extract the current safety judgment threshold combination corresponding to the current work scenario and current action category, and output the current safety status judgment result based on the sensor group timing data and current safety judgment threshold combination within the current time window.

[0041] In a preferred embodiment, the current safety status determination result is output based on the combination of sensor group time-series data within the current time window and the current safety determination threshold, including: Extract the torso tilt angle, angular velocity, integral acceleration value, and buckle status code of the last moment within the current time window; If the torso tilt angle is less than the corresponding current torso tilt angle safety threshold, the angular velocity is less than the current angular velocity safety threshold, the integral value of acceleration is less than the current integral value of acceleration safety threshold, and the buckle status code is a closed code, then the current safety status determination result is determined to be safe operation; If the torso tilt angle is less than the corresponding current torso tilt angle safety threshold, the angular velocity is less than the current angular velocity safety threshold, the acceleration integral value is less than the current acceleration integral safety threshold, and the buckle status code is an open code, then the current safety status determination result is that there is a risk of illegal fastening. Otherwise, the current safety status is determined to be that there is a risk of a fall.

[0042] To illustrate, for the combination of tower climbing and other operations, the corresponding safety thresholds are: torso tilt angle 45°, angular velocity 18° / s, and acceleration integral 2.2 m / s². 2 When climbing the tower, the body posture tilts too much and the limb dynamics change significantly. The corresponding threshold needs to be adapted to the normal movement range of climbing. For the combination of tower work and stationary standing work, the corresponding safety thresholds are: torso tilt angle 20°, angular velocity 6° / s, and acceleration integral 0.6 m / s². 2 When the tower is stationary, the stationary posture tends to be stable, the range of motion parameter fluctuations is small, and the corresponding threshold is tightened to quickly identify abnormal swaying. For the combination of platform and backward tilting operation, the corresponding safe thresholds are: torso tilt angle 30°, angular velocity 12° / s, and acceleration integral 1.3 m / s². 2 When working on the platform with the body tilted back, there is a reasonable body offset, and the corresponding threshold takes into account both the working space and the risk of exceeding the posture limit. For the combination of scaffolding scenarios and mobile relocation operations, the corresponding safety thresholds are: torso tilt angle 38°, angular velocity 15° / s, and acceleration integral 1.8 m / s². 2 When moving and changing points in a scaffolding scenario, the frequency of limb swinging is high during the point movement process, which corresponds to the threshold matching of small walking motion characteristics; For the combination of tower operation and backward tilting, the corresponding safety thresholds are: trunk tilt angle 28°, angular velocity 11° / s, and acceleration integral 1.2 m / s². 2 When working backwards on a pole, the working space is narrow and cramped, and the backward tilting range is limited. Excessive offset can easily touch the components and break the body's balance. Therefore, the threshold is set relatively small, which can meet the needs of fine-tuning the posture during operation and can also identify the tendency to exceed the limit and become unstable in a timely manner. For the combination of platform scene and static standing operation, the corresponding torso tilt angle threshold is 18°, angular velocity threshold is 5° / s, and acceleration integral threshold is 0.5m / s². Platform standing operation requires the body to remain upright and stable, with no obvious limb movement and extremely small parameter fluctuation range. Tightening the threshold can sensitively capture dangerous signs such as sudden shaking and center of gravity shift. The combination of scaffolding scenarios and static standing work corresponds to a torso tilt angle threshold of 22°, an angular velocity threshold of 7° / s, and an acceleration integral threshold of 0.7m / s². Slight elastic swaying of the scaffolding assembly structure and natural small-amplitude swaying are normal phenomena. The thresholds are appropriately relaxed to avoid misjudgment caused by structural vibration and accurately identify human abnormal posture. For combinations of other work scenarios and action categories, they can be obtained based on high-altitude work experience data calibration or kinematic calculations, which will not be elaborated here; The buckle status determination code can be set as follows: 0 for closed and 1 for open. The state machine retrieves the corresponding threshold based on the identified combination of scene actions, compares it with real-time monitoring parameters, and completes the safety state transition determination according to preset rules.

[0043] Specifically, a finite state machine mechanism is used to automate the assessment of operational risks. The state machine pre-establishes multiple sets of corresponding safety threshold combinations based on three operational scenarios (towers, platforms, and scaffolding) and four action categories (climbing, moving and changing points, standing still, and leaning backward). These include trunk tilt angle safety thresholds, angular velocity safety thresholds, acceleration integral safety thresholds, and buckling status judgment rules, forming a complete state transition condition library. During the application phase, the state machine automatically matches and extracts the corresponding safety threshold combinations based on the current operational scenario and action category output by the hybrid neural network model, using this as the basis for determining the current state. Subsequently, the state machine extracts the trunk tilt angle, angular velocity, acceleration integral value, and buckling status code from the sensor group's time-series data within the current time window, and executes condition judgments sequentially according to preset logic. If the trunk tilt angle is less than the corresponding safety threshold, angular velocity... If the values ​​of the acceleration integral and the buckle status are both below the corresponding safety threshold, and the buckle status code is closed, then the safe operating conditions are met, and the state machine transitions and outputs a safe operating status. If the first three parameters meet the threshold requirements, but the buckle status code is open, the state machine determines that there is a risk of unauthorized attachment. If any motion parameter exceeds the threshold range, the state machine directly determines that there is a risk of a fall, regardless of the buckle status. These characteristics collectively indicate that the human body has entered a state of instability or a tendency to fall. Even if the buckle status sensor does not report that the hook has fallen off, it is still considered a high-risk state. This predictive mechanism can provide early warning at the moment of instability to prevent falls. The entire process is based on the state machine's condition matching, state transition, and rule reasoning mechanism to achieve dynamic threshold adaptation and multi-condition joint judgment, making the safety status judgment more consistent with the actual operating scenario and action pattern, and improving the accuracy, reliability, and real-time performance of the judgment results.

[0044] In the state determination step, a finite state machine is used to automatically identify the risks of high-altitude operations. Combined with differentiated configuration of judgment thresholds based on the work scenario and action type, a complete state transition rule system is constructed, effectively improving the scientific nature of safety assessment and the foresight of early warning. The state machine pre-defines corresponding posture parameter thresholds and buckle judgment criteria for three types of work environments (towers, platforms, and scaffolding) and four types of work behaviors (climbing, moving to different points, standing still, and leaning backward), forming a clearly categorized judgment condition library that can adapt to safety assessment standards under different working conditions. During actual operation, the system intelligently retrieves matching threshold combinations based on the scene and action results identified by the neural network, accurately assessing the current work state. By extracting key data such as tilt angle, angular velocity, acceleration integral, and buckle code from real-time sensor terminals, multi-parameter joint verification is completed according to the hierarchical judgment logic, and automatic work state transitions are achieved through condition comparison. The judgment system considers both protection compliance and limb stability, verifying both the wearing status of safety buckles and monitoring the range of human posture fluctuations. Once motion parameters exceed safety limits, a high risk of fall is immediately identified. Unaffected by the state of the restraints, it can quickly detect anomalies when a person's body shows signs of instability, enabling early warning of potential dangers. The entire judgment process operates automatically based on state machine rule-based reasoning logic, completing dynamic threshold adaptation and state changes without human intervention. It can output judgment conclusions that are consistent with the ever-changing work conditions on site. It avoids the misjudgment problems caused by single-parameter evaluation and shortens the early warning response time with its advanced prediction capabilities, effectively enhancing the accuracy, stability, and timeliness of work status judgment. This provides reliable judgment support for the issuance of graded warnings and on-site risk management, effectively reducing the probability of high-altitude fall accidents.

[0045] 104. Generate accident warnings based on the current safety assessment results.

[0046] In a preferred embodiment, generating an incident warning based on the current safety assessment result includes: When the current safety status is determined to be safe for operation, a first prompt message is generated to characterize the safe operation. When the current safety status assessment result indicates that there is a risk of illegal mooring, a second prompt message is generated to indicate the existence of the risk of illegal mooring. When the current safety status assessment result indicates that there is a risk of a fall accident, a third warning message is generated to indicate that there is a risk of a fall accident.

[0047] Specifically, based on the different safety judgment results output by the state machine, corresponding prompt information is generated in a differentiated manner, and a graded alarm triggering mechanism is matched to realize intuitive feedback on the operation status and risk graded warning; When the judgment result is a safe operation, the system generates the first prompt message, which indicates that the current personnel posture parameters and the status of the protective buckles meet the specifications and that the overall operation is stable and compliant. This type of prompt is only recorded in the background log and displayed on the interface. It does not activate any alarm devices and only presents the normal operation status in a visual way. The operator can carry out high-altitude operation normally. When the system determines that there is a risk of improper attachment, it simultaneously generates a second prompt message, visually indicating that the safety buckle is not closed or the protective equipment is not attached in accordance with the operating procedures. It immediately triggers a dedicated buckle attachment reminder, prompting the operator to check and tighten the protective buckle in time through terminal pop-up prompts or voice broadcasts, urging the operator to immediately rectify the violation and eliminate the hidden danger of protective lapses, so as to prevent further accidents. When the system determines that there is a risk of a fall, it immediately generates a third warning message, clearly alerting the personnel that their posture deviation and limb movement exceed the safety limits, indicating a risk of falling from a height. The system then immediately activates the audio-visual warning mechanism, with the warning indicator flashing at a high frequency and accompanied by a high-frequency alarm sound, sending a high-intensity danger warning signal in both directions. This quickly reminds the on-site workers to stop working and adjust their body posture, and also notifies the on-site supervisors to intervene in a timely manner to prevent a fall accident.

[0048] To illustrate, when the current safety status is determined to be a risk of unauthorized fastening, after triggering the second prompt message, the time when the latch opens is recorded, along with the allowable time for the latch to open under the current work scenario and current action category combination; if the latch opening time exceeds the allowable time, further audio and visual reminders can be triggered; otherwise, the operator's status is marked as moving, and only a prompt to check the latch status is output to avoid frequent triggering of sound alarms affecting high-altitude operations.

[0049] In the accident early warning process, a tiered alarm feedback mechanism is used to match differentiated prompts and warning strategies based on different safety assessment results. This enables intuitive presentation of the work status and tiered risk management, adapting to the safety management needs of different working conditions at heights. Three types of work statuses are differentiated and corresponding handling methods are adopted. In normal and compliant work scenarios, only work parameters and status information are logged and displayed on the operating interface, without triggering any alarm sounds or light prompts, avoiding additional signal interference with personnel's normal work and ensuring the orderly conduct of operations. When a violation of the safety buckle not being closed is detected, a violation prompt message is immediately generated and promptly notified to the workers through terminal pop-ups and voice broadcasts, reminding them to immediately check and tighten the safety buckle, correct improper wearing behavior as soon as possible, and expedite the process. Eliminating safety loopholes caused by inadequate protection prevents potential hazards from escalating and causing safety issues. If a person's posture or range of motion is determined to exceed safety thresholds, posing a risk of fall, the system quickly issues a hazard warning and simultaneously activates an audible and visual alarm mode. The high-frequency flashing warning lights combined with the urgent alarm sound create a dual, strong alert effect, prompting workers to stop their actions and adjust their posture to avoid instability, while also quickly alerting on-site supervisors to intervene and implement emergency response measures immediately. The overall hierarchical early warning logic is clear and reasonable, adapting different alert intensities according to the severity of the risk. This ensures that routine operations are not disturbed while accurately restraining violations and responding quickly to high-risk situations, comprehensively improving on-site safety management efficiency and effectively curbing the occurrence of falls from heights.

[0050] In another preferred embodiment, it further includes: After outputting the current safety status determination result, extract the location data of the last moment within the current time window as the real-time location; Based on the real-time location, obtain image information of the real-time location; Based on real-time location image information, the current security assessment result is reviewed.

[0051] The image information is obtained by the cloud platform by adjusting the on-site control ball or mobile camera equipment according to the location information; at the same time, the cloud platform is deployed with visual artificial intelligence algorithms, which are existing technologies and will not be described in detail in this article.

[0052] Specifically, after determining the safety status of the operation, the location data at the end of the current window is extracted to lock the real-time spatial location of the operator. This coordinate information is used as the basis for tracing and conducting cloud-based visual verification. Alarm-related data is simultaneously uploaded to the cloud platform. The platform parses the location information attached to the data packet, intelligently searches for deployed surveillance cameras and mobile cameras around the location, and selects the camera device closest to the operator. The platform issues control commands to drive the camera device to adjust its angle to the preset monitoring position, with the lens precisely aimed at the operator on site, and retrieves the real-time video footage. The cloud platform uses deep learning visual intelligence algorithms to call pedestrian detection and seat belt recognition models to analyze the real-time video stream frame by frame, automatically determining the personnel's posture parameters and the fastening status of the buckles, and drawing visual safety verification conclusions. The platform then cross-checks the visual recognition results with the safety assessment information obtained from the front-end sensors. If the two assessments match—for example, if the sensor detects a risk of an unclosed buckle and the visual image confirms that the seatbelt is not properly fastened—then the alarm is deemed valid, and a message is immediately sent to notify supervisory personnel to handle the situation. If the two assessments differ, and the sensor monitoring and visual recognition status do not correspond, the alarm is marked as a suspected false alarm. An automatic manual review process is initiated, allowing management personnel to review the details of the on-site monitoring video to further verify the actual operational status. This dual verification mode, combining sensor data analysis and visual intelligent recognition, effectively filters out false alarms caused by equipment interference and data fluctuations, significantly improving the accuracy of accident alarms and reducing the interference of invalid warnings on safety management.

[0053] To illustrate, reverse verification can also be added. The cloud platform routinely runs a video intelligent monitoring program, continuously analyzing real-time footage transmitted from various on-site cameras. Based on a built-in deep learning visual recognition model, it determines the height, body posture, and actual wearing and fastening status of workers at height in real time, accurately capturing violations such as drooping safety belts, loose buckles, and protective equipment being out of compliance with regulations. Once the visual algorithm determines that a person is in a high-altitude work area and detects an abnormality such as an improperly fastened safety belt, the platform immediately starts timed monitoring, using a preset detection frequency as the preset judgment duration, continuously waiting to receive alarm messages from the front-end intelligent safety belt device. If within the specified time period... If the cloud fails to receive risk alarm signals reported by the front-end sensors, it can be determined that there is a high probability of sensor component failure, abnormal data acquisition, or wireless communication link interruption, and the front-end monitoring system has failed to properly identify and report safety hazards. At this time, the cloud automatically triggers a reverse alarm mechanism, independently generates corresponding risk warning information, and immediately pushes the alarm content to the on-site monitoring terminal. With the help of this reverse verification logic, the deficiencies of single-point front-end monitoring can be made up for, forming a redundant monitoring backup capability for sudden problems such as sensor failure and abnormal signal transmission. It can complement each other in two directions to investigate various safety hazards, comprehensively cover the operation supervision scenario, prevent the omission of risk issues due to equipment or communication failures, and ensure that there are no blind spots in the safety supervision of high-altitude operations.

[0054] In the review of the judgment results, this invention establishes a linkage verification mechanism that combines sensor monitoring and image recognition. Through multi-dimensional cross-verification, it further enhances the reliability of safety judgment conclusions and alarm information, and makes up for the limitations of single monitoring methods. After the front-end equipment completes the preliminary judgment of the on-site operation safety status, the system immediately extracts the real-time location data within the corresponding time window, accurately locates the spatial position of the high-altitude workers, and uploads the complete alarm data, sensor judgment results and positioning information to the cloud platform to formally start the secondary review and verification process. After receiving the relevant data, the cloud platform first analyzes the location coordinates and automatically searches for deployed surveillance cameras and mobile cameras around the work site. It prioritizes the nearest camera with a good signal as the evidence collection device. Then, the platform issues remote control commands to drive the camera to adjust the pan-tilt angle and shooting direction, accurately aiming at the target workers and stably retrieving real-time video streams from the site, providing a clear and effective image data source for subsequent intelligent visual analysis. Leveraging the powerful computing capabilities of cloud servers, the platform utilizes pre-deployed deep learning vision algorithms, combined with human posture detection and safety belt-specific recognition models, to analyze and judge real-time video frame by frame. The algorithms can accurately capture changes in the worker's limb posture and body tilt angle, while also identifying the overall wearing status of the safety belt and the closing status of the safety buckle, independently generating objective and intuitive visual verification conclusions. After completing the visual analysis, the system compares and cross-verifies the conclusion with the safety judgment results output by the front-end sensors, achieving dual verification of the perceived data and image data. If the two judgment results are completely consistent, for example, if the sensor detects that the buckle is not closed and the visual recognition also confirms that the seat belt is not properly fastened, then the alarm is determined to be genuine and valid. The platform will immediately push an alarm notification to the monitoring terminal to remind the management personnel to deal with the hidden danger in a timely manner. If there is a significant discrepancy between the two judgment results, the system will automatically mark the alarm as a suspected false alarm and actively initiate a manual review process, whereby the management personnel will review and view the complete video footage to verify the actual on-site operation. This dual verification system can effectively resist misjudgments caused by on-site environmental interference, sensor signal fluctuations, hardware equipment failures, etc., and efficiently filter out various invalid alarm information. The working mode of using intelligent technology comparison as the main method and manual verification as a supplement significantly improves the overall accuracy of high-altitude operation risk alarms, reduces the waste of human and material resources caused by false warnings, helps regulatory personnel to grasp the on-site safety situation in real time and accurately, and comprehensively improves the level of safety management and emergency response efficiency of high-altitude operations.

[0055] In an optional embodiment, low-power control is also included.

[0056] Specifically, in the high-altitude operation monitoring system of this invention, the human vibration detection sensor mounted on the sensor group collects vibration signals generated by human limb movements in real time throughout the entire process to determine the activity status of the workers. During the monitoring process, the sensor continuously captures vibration waveform data corresponding to effective actions such as limb swinging, body movement, and climbing displacement, and the system continuously counts the existence of effective action signals. If no effective action signal is captured for 5 consecutive minutes, that is, the vibration amplitude is less than the preset effective action threshold for 5 minutes, it is determined that the on-site personnel have left the work area or stopped working and entered a resting state. Then, a control command is issued to switch the system to a hibernation mode, temporarily suspending high-frequency data acquisition, signal transmission, and data processing, and retaining only the low-frequency acquisition function of the vibration sensor. Once the vibration sensor senses vibration feedback generated by human activity again, a wake-up command is immediately issued to quickly activate the entire monitoring module and resume the normal data acquisition, timing calibration, feature recognition, and safety judgment process, and carry out normal operation status monitoring. If no effective action signal is detected for a long time after wake-up, the system will control the power supply step by step and finally perform an automatic shutdown operation.

[0057] This dynamic and static power consumption control mechanism perfectly matches the entire work process of high-altitude workers leaving their posts, resting, and resuming work. Relying on signals collected by vibration sensors, it accurately determines the worker's activity status and dynamically switches the operating mode of the intelligent safety belt monitoring module to achieve a balance between energy consumption and monitoring efficiency. When workers leave their posts or rest in place and do not generate effective limb vibration for a continuous period, the equipment automatically enters a sleep state, suspending energy-consuming tasks such as high-frequency data acquisition, transmission, and processing, significantly reducing overall power consumption, effectively extending the battery life of wearable devices, reducing the frequency of repeated charging on-site, and significantly improving the equipment's continuous operating capability in complex field conditions. When workers resume work and the sensors detect vibration signals, the system can quickly wake up and immediately... The system restores the entire monitoring process, including data acquisition, time-series alignment, feature recognition, and safety assessment, seamlessly connecting monitoring work to ensure complete acquisition of key data such as attitude, latch status, and position, without any monitoring gaps. For long-term unattended operation conditions, the equipment automatically shuts down, completely eliminating unnecessary power consumption from idle operation and reducing wear and tear on hardware during prolonged standby. The entire state switching logic operates autonomously, requiring no manual power on / off switching or adjustment of operating modes, resulting in a high degree of automation. This mechanism ensures comprehensive and uninterrupted high-altitude operation safety monitoring and risk warnings while continuously optimizing equipment energy consumption, meeting the core needs of on-site safety supervision while improving the environmental adaptability and long-term economic efficiency of the monitoring device.

[0058] By implementing the above embodiments, the following effects are achieved: the current work scenario and current action category can be obtained based on the time-series data of the sensor group through a trained hybrid neural network model; based on the accurate classification results of the current work scenario and current action category, the subsequent state machine loads the corresponding safety judgment threshold group according to the current work scenario and current action category, performs safety status judgment based on the current time-series data and the safety judgment threshold set, and finally generates an accident warning based on the safety judgment result; the judgment threshold can be adaptively selected according to the identified work scenario and action category, reducing the occurrence of misjudgments, improving the accuracy of accident warnings, and thus significantly improving the safety of high-altitude operations.

[0059] like Figure 2 As shown, based on the above method embodiments, corresponding apparatus embodiments are provided; One embodiment of the present invention provides a high-altitude operation accident early warning device based on a hybrid neural network model, comprising: a data acquisition module, a classification module, a status determination module, and an accident early warning module; The data acquisition module is used to acquire the time series data of the sensor group within the current time window; The classification module is used to input the time-series data of the sensor group within the current time window into the hybrid neural network model, so that the hybrid neural network model outputs the current work scenario and the current action category; wherein, the hybrid neural network model uses the time-series data of the sensor group as training samples, uses the work scenario and action category corresponding to the training samples as sample labels, and uses the predicted work scenario and predicted action category as outputs, and performs iterative training until the loss function converges; The state determination module is used to input the sensor group time-series data, the current work scenario, and the current action category within the current time window into the state machine, so that the state machine can extract the current safety determination threshold combination corresponding to the current work scenario and the current action category, and output the current safety state determination result based on the sensor group time-series data and the current safety determination threshold combination within the current time window. The accident warning module is used to generate accident warnings based on the current safety assessment results.

[0060] In a preferred embodiment, the high-altitude operation accident early warning device based on a hybrid neural network model further includes: a judgment verification module; The judgment and verification module is used to extract the location data of the last moment in the current time window as the real-time location after outputting the current security status judgment result; Based on the real-time location, obtain image information of the real-time location; Based on real-time location image information, the current security assessment result is reviewed.

[0061] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can realize the high-altitude operation accident early warning method based on a hybrid neural network model provided by any of the above-described method embodiments of the present invention.

[0062] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0063] Based on the above embodiments of the high-altitude operation accident early warning method based on a hybrid neural network model, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the high-altitude operation accident early warning method based on a hybrid neural network model of any embodiment of the present invention.

[0064] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0065] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0066] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0067] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the high-altitude operation accident early warning method based on a hybrid neural network model as described in any of the above-described method embodiments of the present invention.

[0068] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0069] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for early warning of high-altitude operation accidents based on a hybrid neural network model, characterized in that, include: Obtain the time series data of the sensor group within the current time window; The time series data of the sensor group within the current time window is input into the hybrid neural network model so that the hybrid neural network model outputs the current work scenario and the current action category. The hybrid neural network model uses the time series data of the sensor group as training samples, the work scenario and action category corresponding to the training samples as sample labels, and the predicted work scenario and predicted action category as outputs. It is trained iteratively until the loss function converges. Input the sensor group timing data, current work scenario, and current action category within the current time window into the state machine, so that the state machine can extract the current safety judgment threshold combination corresponding to the current work scenario and current action category, and output the current safety status judgment result based on the sensor group timing data and current safety judgment threshold combination within the current time window. An accident warning is generated based on the current safety assessment results.

2. The high-altitude operation accident early warning method based on a hybrid neural network model as described in claim 1, characterized in that, The acquisition of sensor group time-series data within the current time window includes: Data from the sensor group is continuously collected according to a preset time sliding window; Based on a preset reference timing sequence, the data of the sensor group is calibrated to obtain the timing data of the sensor group within the current time window; The data from the sensor array includes: torso tilt angle, angular velocity, integral acceleration value, buckle status code, position data, and vibration amplitude.

3. The high-altitude operation accident early warning method based on a hybrid neural network model as described in claim 2, characterized in that, The hybrid neural network model includes: a convolutional neural network, a recurrent neural network, a task scene classification head, and an action category classification head; The step of inputting the time-series data of the sensor group within the current time window into the hybrid neural network model, so that the hybrid neural network model outputs the current work scenario and the current action category, includes: Based on a convolutional neural network, feature vectors at each time step are extracted from the time-series data of the sensor group to generate spatial features at each time step. Based on a recurrent neural network, spatial features are recursively encoded time-by-time to obtain the spatiotemporal features within the current time window. The task scenario classification head calculates the probability distribution of the task scenario based on the spatiotemporal characteristics within the current time window, and outputs the current task scenario based on the probability distribution; where the task scenario is a pole, platform, or scaffold. Using the action category classification head, the probability distribution of action categories is calculated based on the spatiotemporal characteristics within the current time window, and the current action category is output based on the probability distribution of action categories; where the action category is climbing operation, moving and changing point operation, standing still operation, or leaning backward operation.

4. The high-altitude operation accident early warning method based on a hybrid neural network model as described in claim 3, characterized in that, Based on the combination of sensor group time-series data within the current time window and the current safety judgment threshold, the current safety status judgment result is output, including: Extract the torso tilt angle, angular velocity, integral acceleration value, and buckle status code of the last moment within the current time window; If the torso tilt angle is less than the corresponding current torso tilt angle safety threshold, the angular velocity is less than the current angular velocity safety threshold, the integral value of acceleration is less than the current integral value of acceleration safety threshold, and the buckle status code is a closed code, then the current safety status determination result is determined to be safe operation; If the torso tilt angle is less than the corresponding current torso tilt angle safety threshold, the angular velocity is less than the current angular velocity safety threshold, the acceleration integral value is less than the current acceleration integral safety threshold, and the buckle status code is an open code, then the current safety status determination result is that there is a risk of illegal fastening. Otherwise, the current safety status is determined to be that there is a risk of a fall.

5. The high-altitude operation accident early warning method based on a hybrid neural network model as described in claim 4, characterized in that, An incident warning is generated based on the current safety assessment results, including: When the current safety status is determined to be safe for operation, a first prompt message is generated to characterize the safe operation. When the current safety status assessment result indicates that there is a risk of illegal mooring, a second prompt message is generated to indicate the existence of the risk of illegal mooring. When the current safety status assessment result indicates that there is a risk of a fall accident, a third warning message is generated to indicate that there is a risk of a fall accident.

6. The high-altitude operation accident early warning method based on a hybrid neural network model as described in claim 5, characterized in that, Also includes: After outputting the current safety status determination result, extract the location data of the last moment within the current time window as the real-time location; Based on the real-time location, obtain image information of the real-time location; Based on real-time location image information, the current security assessment result is reviewed.

7. A high-altitude operation accident early warning device based on a hybrid neural network model, characterized in that, include: The module includes a data acquisition module, a classification module, a status determination module, and an accident early warning module. The data acquisition module is used to acquire the time series data of the sensor group within the current time window; The classification module is used to input the time-series data of the sensor group within the current time window into the hybrid neural network model, so that the hybrid neural network model outputs the current work scenario and the current action category; wherein, the hybrid neural network model uses the time-series data of the sensor group as training samples, uses the work scenario and action category corresponding to the training samples as sample labels, and uses the predicted work scenario and predicted action category as outputs, and performs iterative training until the loss function converges; The state determination module is used to input the sensor group time-series data, the current work scenario, and the current action category within the current time window into the state machine, so that the state machine can extract the current safety determination threshold combination corresponding to the current work scenario and the current action category, and output the current safety state determination result based on the sensor group time-series data and the current safety determination threshold combination within the current time window. The accident warning module is used to generate accident warnings based on the current safety assessment results.

8. The high-altitude operation accident early warning device based on a hybrid neural network model as described in claim 7, characterized in that, Also includes: Judgment review module; The judgment and verification module is used to extract the location data of the last moment in the current time window as the real-time location after outputting the current security status judgment result; Based on the real-time location, obtain image information of the real-time location; Based on real-time location image information, the current security assessment result is reviewed.

9. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the high-altitude operation accident early warning method based on a hybrid neural network model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the high-altitude operation accident early warning method based on a hybrid neural network model as described in any one of claims 1-7.

Images