A hoisting state intelligent perception early warning method and system

Abstract

The application discloses a hoisting state intelligent perception early warning method and system, relates to the hoisting state monitoring technical field, deploys a multimodal sensor network, and collects four types of data of equipment operation, hoisting object posture, operation environment and on-site vision in real time; after pre-processing of multi-source heterogeneous data, time and space alignment is completed based on a satellite timing unified reference to generate a standardized data set; multi-dimensional features are extracted and fused through a deep learning model with double branches and cross-modal attention to identify operation condition types and real-time state parameters; a comprehensive risk value is calculated by combining multi-level safety threshold values, condition weight coefficients and historical fault matching degrees to map corresponding risk levels; finally, a graded sound and light early warning is triggered according to the risk level, and corresponding safety constraint instructions are output to a device control system. The application realizes full-dimensional perception, intelligent evaluation and closed-loop management and control of the hoisting state, and effectively improves the risk early warning precision and active safety protection capability.

Description

Technical Field

[0001] The present invention relates to the technical field of hoisting condition monitoring, and more specifically, to an intelligent sensing and early warning method and system for hoisting conditions. Background Art

[0002] At present, hoisting operation is a core operation link in engineering construction, port logistics, heavy equipment manufacturing and other fields, which is characterized by large load, complex working environment and high coupling degree of risk factors. Once accidents such as heavy object falling, equipment overturning, and personnel collision occur, it will cause serious casualties and property losses. Therefore, realizing real-time sensing and risk early warning of hoisting operation status is the core requirement to guarantee hoisting safety.

[0003] Existing hoisting safety monitoring technologies mostly adopt single-point monitoring schemes with a single load sensor or wind speed sensor, supplemented by the management mode of manual inspection, and have the following significant defects: First, the sensing dimension is single, which can only monitor a few core parameters, cannot fully cover equipment status, lifted object posture, environmental risks and on-site violations, resulting in a high risk missing report rate; Second, the multi-source data fusion capability is insufficient. Although some schemes deploy multiple types of sensors, they do not solve the problems of inconsistent sampling frequencies and time asynchronization of different sensors. The spatiotemporal dislocation of data leads to low precision of fusion analysis and it is difficult to reflect the real operation status; Third, the risk assessment method is simple and extensive, mostly adopts the single-parameter comparison method with fixed threshold, does not consider the difference of risk weight of each parameter under different operation conditions, nor does it combine historical fault data for risk matching, resulting in high false alarm rate and rough risk classification, which cannot adapt to complex and changeable operation scenarios; Fourth, early warning is disconnected from control. Most systems only realize sound and light alarm prompts, do not link with the control system of hoisting equipment, cannot actively intervene in risk evolution, and there is obvious lag in safety protection.

[0004] In summary, the existing technologies are difficult to meet the requirements of accurate sensing, accurate assessment and active protection for hoisting operations in complex scenarios, and there is an urgent need for a hoisting condition sensing and early warning scheme with multi-modal fusion, intelligent assessment and closed-loop management and control. Summary of Invention

[0005] In view of this, the present invention provides an intelligent sensing and early warning method and system for hoisting conditions to solve the problems existing in the background art.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solution: An intelligent sensing and early warning method for hoisting conditions, comprising: S1, deploying a multi-modal sensor network on hoisting equipment and an operation site, and collecting operation parameters of the hoisting equipment, posture parameters of a lifted object, operation environment parameters and on-site visual image data in real time; S2. The collected multi-source heterogeneous data are preprocessed sequentially, and the spatiotemporal alignment of multi-channel data is completed based on a unified time base to generate a standardized hoisting status dataset. S3. Input the standardized hoisting status dataset into the pre-trained multi-feature fusion deep learning model, extract multi-dimensional features and perform fusion calculations to identify the working condition type and real-time status parameters of the current hoisting operation. S4. Match and compare the identified real-time status parameters with the preset multi-level safety thresholds, and calculate the comprehensive risk value of the current operation by combining the historical fault sample library and the working condition weight coefficient, and map the corresponding risk level. S5. Based on the risk level obtained from the assessment, trigger the corresponding level of audible and visual early warning signal and push it to the on-site terminal and monitoring platform, while simultaneously outputting the corresponding level of safety constraint command to the hoisting equipment control system.

[0007] Optionally, the deployment and data acquisition of the multimodal sensor network specifically includes: Equipment operation sensors are deployed at the boom, hoisting mechanism, slewing mechanism, and luffing mechanism of the lifting equipment to collect operating parameters including lifting capacity, hoisting speed, wire rope tension, slewing angle, slewing speed, luffing angle, and luffing speed. A six-axis attitude sensor is deployed at the connection between the hook and the load to collect load attitude parameters including load tilt angle, swing amplitude, swing angular velocity, and torsion angle. Environmental sensing units are deployed around the work site and at the top of the boom to collect environmental parameters including wind speed, wind direction, ambient temperature, and atmospheric visibility. Binocular vision acquisition units are deployed at diagonal positions at the work site to collect visual image data including panoramic images of the work area, close-up images of the load, and images of the hook in operation. All sensing units are connected to a wired and wireless hybrid transmission network through an industrial IoT gateway to achieve real-time data aggregation and uploading.

[0008] Optionally, the data preprocessing and spatiotemporal alignment specifically include: For time-series sensor data, wavelet threshold denoising is used to remove high-frequency acquisition noise, and the 3σ criterion is used to remove outliers exceeding the normal value range. For visual image data, Gaussian filtering is used to remove image noise, and contrast-limited adaptive histogram equalization is used to improve image contrast in low-light environments. UTC time, synchronized by the BeiDou satellite, is used as the unified time base, and millisecond-level timestamps are applied to all channel data. For heterogeneous data with different sampling frequencies, linear interpolation is used to align all data to a unified sampling time series; the alignment formula is as follows:

[0009] In the formula, t is the target alignment time. t 1. t2 represents the consecutive sampling times of the sensor to be interpolated. x 1. x 2 represents the collected value at the corresponding time. x ( t The values ​​are aligned; all modal data at the same time are combined into data samples according to the timestamp order to generate a time-series continuous standardized hoisting state dataset. Optionally, the multi-feature fusion deep learning model adopts a dual-branch extraction and cross-modal attention fusion architecture. The feature extraction and fusion process specifically includes: Visual Feature Extraction Branch: Using a lightweight convolutional neural network as the backbone, combined with a spatial attention module, it extracts spatial visual features such as hook position, object outline, and personnel intrusion into the work area from on-site visual image data. F v ; Temporal Feature Extraction Branch: Using a bidirectional long short-term memory network (Bi-LSTM) as the backbone, dynamic temporal features of equipment operation trends and load swing patterns are extracted from temporal parameters of equipment operation, load posture, and working environment. F t ; Cross-modal attention fusion module: First, a fully connected layer maps the two types of features to the same feature dimension, then calculates the attention weights of each modality, and finally weights them to obtain the fused features. F f The fusion calculation formula is as follows:

[0010] In the formula, W v , W t , W a This is the weight matrix. b v , b t , b a For bias terms, This indicates element-wise multiplication. These are the attention weights for visual features and temporal features, respectively; Fusion features F f Input the classification output header and the regression output header respectively. The classification header outputs the current operation condition type, and the regression header outputs the quantified real-time status parameters. Optionally, the calculation and mapping of the comprehensive risk value and the risk level specifically includes: A four-level safety threshold system is preset, corresponding to four levels: low risk, medium risk, high risk, and extremely high risk. Each real-time status parameter corresponds to a threshold range for each of the four levels. Based on the deviation of real-time status parameters from corresponding level thresholds, a risk score for each parameter is calculated. k i The closer the parameter is to or exceeds the higher-level threshold, the higher the risk score. Based on the identified working condition type, dynamically match the corresponding working condition weight coefficient. w i The weight ratio of each parameter is set differently under different working conditions, and the sum of all weight coefficients is 1. The cosine similarity between the current state features and the fault features in the historical fault sample library is calculated to obtain the historical fault matching coefficient. s i ; Calculate the overall risk value R:

[0011] In the formula, n is the total number of state parameters participating in the evaluation; Three preset risk thresholds T 1. T 2. T 3. Map the risk level based on the range of the comprehensive risk value: when R < T At 1 o'clock, it was classified as a level 1 low-risk area. At that time, it was a level 2 medium risk. At that time, it was a level three high-risk area. The risk level is Level 4, extremely high. Optionally, the execution logic of S5 is as follows: Level 1 Low Risk: The monitoring platform displays normal operating status, does not trigger on-site audible and visual warnings, and does not output safety constraint commands. Level 2 medium risk triggers on-site low-frequency audible and visual warnings, pushes warning information to on-site handheld terminals and monitoring platforms, and outputs deceleration constraint commands to the hoisting equipment control system; Level 3 high risk triggers on-site medium-frequency audible and visual warning, pushes alarm information to on-site terminals and monitoring platforms, outputs limit constraint commands to the hoisting equipment control system, and locks the operation authority of the mechanism in the dangerous direction; Level 4 extremely high risk triggers on-site high-frequency audible and visual warnings, pushes emergency alarm information to all terminals, outputs emergency stop commands to the hoisting equipment control system, and triggers the equipment safety braking mechanism; All early warning events and control commands are synchronously stored in the safety log database for post-event traceability and model iteration optimization. Optionally, the multi-feature fusion deep learning model also includes an online incremental update mechanism: at fixed times each day, newly added normal operation samples and risk event samples are added to the training set, and the top-level parameters of the model are fine-tuned using the learning rate to update the feature extraction and fusion weights, continuously improving the model's adaptability to the on-site environment. A hoisting status intelligent perception and early warning system includes: The data acquisition module deploys a multimodal sensor network on the hoisting equipment and at the work site to collect real-time operating parameters of the hoisting equipment, attitude parameters of the hoisted object, parameters of the working environment, and on-site visual image data. The data preprocessing module preprocesses the collected multi-source heterogeneous data sequentially and completes the spatiotemporal alignment of multi-channel data based on a unified time base to generate a standardized hoisting status dataset. The model recognition module inputs a standardized hoisting status dataset into a pre-trained multi-feature fusion deep learning model, extracts multi-dimensional features and performs fusion calculations to identify the current hoisting operation's working condition type and real-time status parameters. The risk level mapping module matches and compares the identified real-time status parameters with preset multi-level safety thresholds, and calculates the comprehensive risk value of the current operation by combining the historical fault sample library and the working condition weight coefficient, and maps the corresponding risk level. The early warning module triggers corresponding audible and visual early warning signals based on the assessed risk level and pushes them to the on-site terminal and monitoring platform. At the same time, it outputs corresponding safety constraint commands to the hoisting equipment control system.

[0012] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for intelligent sensing and early warning of hoisting status, which has the following beneficial effects: 1. This invention breaks through the perception limitations of a single sensor by deploying four types of sensing units covering equipment operation, hoisted object posture, working environment, and on-site vision. It can capture risk factors throughout the entire hoisting operation chain and comprehensively cover multiple risk scenarios such as equipment failure, posture instability, environmental degradation, and personnel violations.

[0013] 2. This invention, based on a unified time reference provided by satellite time synchronization and combined with a linear interpolation alignment method, solves the problem of asynchronous data from sensors of different types and sampling frequencies. The generated standardized dataset eliminates spatiotemporal misalignment and provides reliable data support for the accurate identification of subsequent deep learning models.

[0014] 3. This invention employs a dual-branch deep learning architecture to extract visual spatial features and temporal dynamic features respectively, and combines an attention mechanism to achieve adaptive weighted fusion. This not only preserves the scene perception capability of the image, but also takes into account the temporal change law of the parameters. The accuracy of working condition recognition and state parameter measurement is significantly better than that of the single-modal model.

[0015] 4. This invention comprehensively considers parameter threshold deviation, operating condition weight differences, and historical fault similarity to calculate a comprehensive risk value. Compared with the traditional fixed threshold method, it can effectively distinguish between normal parameter fluctuations and real risks under different operating conditions. The risk assessment results are more in line with actual operating scenarios, and the false alarm rate is significantly reduced.

[0016] 5. The hierarchical early warning mechanism of this invention is linked with equipment safety constraint commands, realizing a closed loop of the entire process from risk identification to proactive intervention; different risk levels correspond to differentiated control strategies, which can curb risks in a timely manner while avoiding excessive braking that affects operational efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the method flow provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention discloses an intelligent sensing and early warning method for hoisting status, such as... Figure 1 As shown, it includes: S1. Deploy a multimodal sensor network on the hoisting equipment and at the work site to collect real-time operating parameters of the hoisting equipment, attitude parameters of the hoisted object, parameters of the working environment, and on-site visual image data; Specifically, sensor networks are divided into four types of acquisition units: Equipment operation data acquisition unit: Deployed at the core mechanism of the hoisting equipment. Tension and compression sensors are installed on the hook pulley block to collect data on the lifting weight and wire rope tension; rotary encoders and angular velocity sensors are installed on the hoisting mechanism, slewing mechanism, and luffing mechanism respectively to collect data on hoisting speed, slewing angle, slewing speed, luffing angle, and luffing speed; all equipment parameters are simultaneously acquired through connection to the crane's PLC control system, forming dual-source redundant acquisition to improve data reliability.

[0021] The load attitude acquisition unit adopts a six-axis IMU attitude sensor, which is installed at the connection position between the lifting device and the load. The sampling frequency is set to 50Hz. It collects the X, Y, and Z axis tilt angle, swing amplitude, swing angular velocity and torsion angle of the load in real time, and accurately captures dangerous attitudes such as swaying and torsion of the load.

[0022] Operational environment data acquisition unit: An ultrasonic anemometer is installed at the top of the crane boom to collect real-time wind speed and direction at the operating height; temperature, humidity and visibility sensors are installed in an open, unobstructed location at the operation site to collect environmental parameters such as ambient temperature and atmospheric visibility to assess the impact of extreme weather on the operation.

[0023] Visual image acquisition unit: Two high-definition binocular cameras are deployed diagonally at the work site to cover the entire work area. The sampling frame rate is 10fps, and the cameras acquire panoramic images of the work area, close-up images of the suspended objects, and images of the hook operation. These images are used to identify visual risks such as personnel intrusion, deformation of the suspended objects, and hook detachment.

[0024] All sensing units are connected to the field wired and 5G hybrid transmission network through an industrial IoT gateway to control data upload latency and ensure real-time monitoring. The gateway also has edge caching capabilities, storing data locally when the network is interrupted and automatically retransmitting it after recovery.

[0025] S2. The collected multi-source heterogeneous data are preprocessed sequentially, and the spatiotemporal alignment of multi-channel data is completed based on a unified time base to generate a standardized hoisting status dataset. Specifically, multi-source heterogeneous data is cleaned and synchronized to generate standardized datasets, providing qualified input for subsequent intelligent recognition.

[0026] Data preprocessing

[0027] Differentiated preprocessing strategies are used for different types of data: For time-series numerical data such as equipment operation, load posture, and working environment: First, wavelet threshold denoising based on db4 wavelet basis is used, with the number of decomposition layers set to 5, to remove high-frequency electromagnetic noise during sensor acquisition; then, the 3σ criterion is used to identify and remove outliers exceeding three standard deviations based on the historical normal range of this parameter, so as to avoid abrupt abnormal data interfering with subsequent evaluation.

[0028] For visual image data: First, a 5×5 Gaussian filter kernel is used to remove image noise; then, contrast-limited adaptive histogram equalization (CLAHE) is used to improve the image contrast under low light and hazy weather conditions, with clipLimit set to 2.0 to enhance the detail of key areas of the image.

[0029] Spatiotemporal alignment

[0030] Because different sensors have different sampling frequencies (e.g., IMU 50Hz, anemometer 1Hz, camera 10fps), time alignment is required. Specifically, UTC time synchronized by the BeiDou satellite is used as the unified time reference, and each piece of collected data is timestamped at the millisecond level. The system's unified sampling frequency is set to 10Hz. For sensors with sampling frequencies higher than the reference, an equal-interval sampling method is used to downsample; for sensors with sampling frequencies lower than the reference, linear interpolation is used to complete the data. The interpolation formula is:

[0031] In the formula, t is the target alignment time. t 1. t 2 represents the consecutive sampling times of the sensor to be interpolated. x 1. x 2 represents the collected value at the corresponding time. x ( t () represents the aligned numerical value; After time alignment is completed, all modal data at the same time are combined into a data sample according to the timestamp order to form a time-series continuous standardized hoisting status dataset. Each sample contains all sensing parameters and corresponding image frames at the corresponding time.

[0032] S3. Input the standardized hoisting status dataset into the pre-trained multi-feature fusion deep learning model, extract multi-dimensional features and perform fusion calculations to identify the working condition type and real-time status parameters of the current hoisting operation. Specifically, a pre-trained multi-feature fusion deep learning model is used to extract and fuse multi-dimensional features from standardized data, and output the job condition type and real-time status parameters.

[0033] The model adopts an architecture of dual-branch feature extraction, cross-modal attention fusion, and dual output heads. The specific processing steps are as follows: Visual feature extraction branch: The input is aligned visual image frames. The backbone uses a lightweight MobileNetV3 network, combined with a spatial attention module, to focus on key areas such as hooks, suspended objects, and workers, suppressing redundant background information. The final output is a 256-dimensional spatial visual feature. F v It includes visual information such as the position of the hook, the outline of the suspended object, and the status of personnel intrusion.

[0034] Temporal feature extraction branch: The input is an aligned temporal parameter sequence (taking a temporal window of 10 sampling points from 1 second before the current time). The backbone adopts a two-layer bidirectional long short-term memory network (Bi-LSTM) with a hidden layer dimension of 128, simultaneously capturing the positive and negative trends of parameter changes, and outputting dynamic temporal features with a dimension of 256. Ft It includes dynamic information such as equipment operation trends and the swing patterns of suspended objects.

[0035] Cross-modal attention fusion module: First, visual features and temporal features are mapped to the same 128-dimensional dimension through two fully connected layers, respectively, to obtain... F v' and F t' Then, the attention weights for the two types of features are calculated using a shared attention weight matrix. and A higher weight value indicates a greater contribution of that modality feature to the current state recognition; finally, the fused feature is obtained by element-wise weighted summation. F f The fusion calculation formula is as follows:

[0036] In the formula, W v , W t , W a This is the weight matrix. b v , b t , b a For bias terms, This indicates element-wise multiplication. These are the attention weights for visual features and temporal features, respectively; Output layer: Feature fusion F f Input the classification output head and the regression output head separately. The classification head uses the Softmax activation function to output the current working condition type, including six categories: idle standby, normal lifting, slewing operation, luffing operation, loaded travel, and hook lowering and positioning. The regression head uses the linear activation function to output quantified real-time status parameters, including core evaluation parameters such as load factor, maximum swing angle of the suspended load, wind speed margin, and operating speed margin.

[0037] During the model pre-training phase, historical datasets from multiple hoisting sites were collected for training. The datasets included normal working condition samples and various risk samples (overload, swing amplitude exceeding limits, operation in strong winds, personnel intrusion, etc.). The training adopted a joint loss function with cross-entropy loss (classification task) and mean squared error loss (regression task) weighted at a ratio of 1:2. The model parameters were iteratively optimized through the Adam optimizer, and the initial learning rate was set to 0.001.

[0038] As a further optimization, the model has an online incremental update mechanism: at a fixed time in the early morning every day, newly added normal operation samples and confirmed risk event samples are added to the training set, and the top fully connected layer and attention module of the model are fine-tuned with a small learning rate of 1e-5, so that the model can continuously adapt to the on-site operation environment and equipment characteristics and maintain long-term recognition accuracy.

[0039] S4. Match and compare the identified real-time status parameters with the preset multi-level safety thresholds, and calculate the comprehensive risk value of the current operation by combining the historical fault sample library and the working condition weight coefficient, and map the corresponding risk level. The specific implementation process is as follows: Multi-level safety threshold presets: A four-level safety threshold system is established, corresponding to Level 1 (low risk), Level 2 (medium risk), Level 3 (high risk), and Level 4 (extremely high risk). Each state parameter has four threshold ranges. For example, for the swing angle of the suspended object, the Level 1 threshold is ≤2°, Level 2 is 2°~5°, Level 3 is 5°~8°, and Level 4 is >8°; for the load factor, the Level 1 threshold is ≤80%, Level 2 is 80%~90%, Level 3 is 90%~100%, and Level 4 is >100%.

[0040] Individual risk score calculation: The identified real-time state parameters are compared with the corresponding threshold intervals to calculate the risk score for each parameter. k i The score range is 0 to 100. When the parameter is in the first-level range, the score is 0 to 25; in the second-level range, the score is 25 to 50; in the third-level range, the score is 50 to 75; and in the fourth-level range, the score is 75 to 100. The closer the parameter value is to the upper limit of the range, the higher the score.

[0041] Dynamic matching of operating condition weights: Based on the identified operating condition type, a preset operating condition weight table is called to obtain the operating condition weight coefficient corresponding to each parameter. w i The sum of all weights is 1. For example, in the slewing operation condition, the weight of the load swing angle is set to 0.3 and the weight of the load is set to 0.2; in the lifting operation condition, the weight of the load is set to 0.35 and the weight of the load swing angle is set to 0.15, so that the risk weights can be dynamically adjusted according to the operation condition to match the risk characteristics of different operations.

[0042] Historical fault matching degree calculation: merging features at the current moment F f The cosine similarity is calculated between the fault feature vectors and the historical fault sample database, and the maximum similarity value is taken as the historical fault matching coefficient. s i The value ranges from 0 to 1. The higher the similarity, the closer the current state is to the historical failure scenario, and the higher the probability of risk outbreak, thereby improving the ability to identify hidden risks.

[0043] Overall Risk Value Calculation: The overall risk value R of the current operation is obtained by weighted summation. The calculation formula is as follows:

[0044] In the formula, n is the total number of state parameters participating in the evaluation. In this embodiment, n=6, which are load rate, hanging object swing angle, wind speed, operating speed, personnel intrusion status, and equipment tilt angle, respectively.

[0045] Risk level mapping: Preset three risk thresholds T 1=25、 T 2=50、 T 3=75, risk levels are determined based on the range of the comprehensive risk value: When R < 25, it is classified as Level 1 low risk; when At that time, it was determined to be a level 2 medium risk; when At that time, it was determined to be a level three high-risk area; when At that time, it was determined to be a level four extremely high risk.

[0046] S5. Based on the risk level obtained from the assessment, trigger the corresponding level of audible and visual early warning signal and push it to the on-site terminal and monitoring platform, while simultaneously outputting the corresponding level of safety constraint command to the hoisting equipment control system.

[0047] The specific execution process is as follows: Level 1 Low Risk: The monitoring platform displays a normal operating status, does not trigger on-site audible and visual warnings, and does not output any constraint commands to the equipment control system, ensuring the normal and efficient operation.

[0048] Level 2 Medium Risk: Triggers the low-frequency audible and visual alarm (flashing yellow light and intermittent buzzing) installed on the crane boom and on-site column; simultaneously pushes early warning information to the on-site driver's handheld terminal, safety management personnel's terminal and remote monitoring platform, indicating the risk type and suggested handling measures; outputs deceleration constraint commands to the lifting equipment PLC control system to reduce the speed of risk evolution.

[0049] Level 3 High Risk: Triggers the medium-frequency audible and visual alarm (flashing red light, continuous buzzing); pushes alarm information to all relevant terminals, prompting immediate action; outputs limit constraint commands to the equipment control system, locking the authority of mechanisms operating in the dangerous direction, such as locking the acceleration operation of the slewing mechanism when the swing angle of the suspended object exceeds the limit, prohibiting further increase in swing amplitude, and curbing the escalation of risk.

[0050] Level 4 Extremely High Risk: Triggers a high-frequency audible and visual alarm (high-frequency flashing red light and continuous high-decibel alarm); pushes emergency alarm information to all terminals, prompting immediate evacuation from the danger zone; outputs an emergency stop command to the equipment control system, triggering the equipment's safety braking mechanism, stopping the operation of all mechanisms, and minimizing the occurrence of accidents.

[0051] All early warning events, risk parameters, and control commands are synchronously stored in the safety log database, supporting post-event traceability, accident analysis, and model iterative optimization, forming a continuously optimized safety management closed loop.

[0052] A hoisting status intelligent sensing and early warning system, comprising: The data acquisition module deploys a multimodal sensor network on the hoisting equipment and at the work site to collect real-time operating parameters of the hoisting equipment, attitude parameters of the hoisted object, parameters of the working environment, and on-site visual image data. The data preprocessing module preprocesses the collected multi-source heterogeneous data sequentially and completes the spatiotemporal alignment of multi-channel data based on a unified time base to generate a standardized hoisting status dataset. The model recognition module inputs a standardized hoisting status dataset into a pre-trained multi-feature fusion deep learning model, extracts multi-dimensional features and performs fusion calculations to identify the current hoisting operation's working condition type and real-time status parameters. The risk level mapping module matches and compares the identified real-time status parameters with preset multi-level safety thresholds, and calculates the comprehensive risk value of the current operation by combining the historical fault sample library and the working condition weight coefficient, and maps the corresponding risk level. The early warning module triggers corresponding audible and visual early warning signals based on the assessed risk level and pushes them to the on-site terminal and monitoring platform. At the same time, it outputs corresponding safety constraint commands to the hoisting equipment control system.

[0053] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0054] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent sensing and early warning of hoisting status, characterized in that, include: S1. Deploy a multimodal sensor network on the hoisting equipment and at the work site to collect real-time operating parameters of the hoisting equipment, attitude parameters of the hoisted object, parameters of the working environment, and on-site visual image data; S2. The collected multi-source heterogeneous data are preprocessed sequentially, and the spatiotemporal alignment of multi-channel data is completed based on a unified time base to generate a standardized hoisting status dataset. S3. Input the standardized hoisting status dataset into the pre-trained multi-feature fusion deep learning model, extract multi-dimensional features and perform fusion calculations to identify the working condition type and real-time status parameters of the current hoisting operation. S4. Match and compare the identified real-time status parameters with the preset multi-level safety thresholds, and calculate the comprehensive risk value of the current operation by combining the historical fault sample library and the working condition weight coefficient, and map the corresponding risk level. S5. Based on the risk level obtained from the assessment, trigger the corresponding level of audible and visual early warning signal and push it to the on-site terminal and monitoring platform, while simultaneously outputting the corresponding level of safety constraint command to the hoisting equipment control system.

2. The intelligent sensing and early warning method for hoisting status according to claim 1, characterized in that, The deployment and data acquisition of the multimodal sensor network specifically includes: Equipment operation sensors are deployed at the boom, hoisting mechanism, slewing mechanism, and luffing mechanism of the lifting equipment to collect operating parameters including lifting capacity, hoisting speed, wire rope tension, slewing angle, slewing speed, luffing angle, and luffing speed. A six-axis attitude sensor is deployed at the connection between the hook and the load to collect load attitude parameters including load tilt angle, swing amplitude, swing angular velocity, and torsion angle. Environmental sensing units are deployed around the work site and at the top of the boom to collect environmental parameters including wind speed, wind direction, ambient temperature, and atmospheric visibility. Binocular vision acquisition units are deployed at diagonal positions at the work site to collect visual image data including panoramic images of the work area, close-up images of the load, and images of the hook in operation. All sensing units are connected to a wired and wireless hybrid transmission network through an industrial IoT gateway to achieve real-time data aggregation and uploading.

3. The intelligent sensing and early warning method for hoisting status according to claim 1, characterized in that, The data preprocessing and spatiotemporal alignment specifically include: For time-series sensor data, wavelet threshold denoising is used to remove high-frequency acquisition noise, and the 3σ criterion is used to remove outliers exceeding the normal value range. For visual image data, Gaussian filtering is used to remove image noise, and contrast-limited adaptive histogram equalization is used to improve image contrast in low-light environments. UTC time, synchronized by the BeiDou satellite, is used as the unified time base, and millisecond-level timestamps are applied to all channel data. For heterogeneous data with different sampling frequencies, linear interpolation is used to align all data to a unified sampling time series; the alignment formula is as follows: In the formula, t is the target alignment time. t 1. t 2 represents the consecutive sampling times of the sensor to be interpolated. x 1. x 2 represents the collected value at the corresponding time. x ( t The values ​​are aligned; all modal data at the same time are combined into data samples according to the timestamp order to generate a time-series continuous standardized hoisting status dataset.

4. The intelligent sensing and early warning method for hoisting status according to claim 1, characterized in that, The multi-feature fusion deep learning model adopts an architecture of dual-branch extraction and cross-modal attention fusion. The feature extraction and fusion process specifically includes: Visual Feature Extraction Branch: Using a lightweight convolutional neural network as the backbone, combined with a spatial attention module, it extracts spatial visual features such as hook position, object outline, and personnel intrusion into the work area from on-site visual image data. F v ; Temporal Feature Extraction Branch: Using a bidirectional long short-term memory network (Bi-LSTM) as the backbone, dynamic temporal features of equipment operation trends and load swing patterns are extracted from temporal parameters of equipment operation, load posture, and working environment. F t ; Cross-modal attention fusion module: First, a fully connected layer maps the two types of features to the same feature dimension, then calculates the attention weights of each modality, and finally weights them to obtain the fused features. F f The fusion calculation formula is as follows: In the formula, W v , W t , W a This is the weight matrix. b v , b t , b a For bias terms, This indicates element-wise multiplication. These are the attention weights for visual features and temporal features, respectively; Fusion features F f Input the classification output header and the regression output header respectively. The classification header outputs the current operation condition type, and the regression header outputs the quantified real-time status parameters.

5. The intelligent sensing and early warning method for hoisting status according to claim 1, characterized in that, The calculation and mapping of the comprehensive risk value and the risk level specifically includes: A four-level safety threshold system is preset, corresponding to four levels: low risk, medium risk, high risk, and extremely high risk. Each real-time status parameter corresponds to a threshold range for each of the four levels. Based on the deviation of real-time status parameters from corresponding level thresholds, a risk score for each parameter is calculated. k i The closer the parameter is to or exceeds the higher-level threshold, the higher the risk score. Based on the identified working condition type, dynamically match the corresponding working condition weight coefficient. w i The weight ratio of each parameter is set differently under different working conditions, and the sum of all weight coefficients is 1. The cosine similarity between the current state features and the fault features in the historical fault sample library is calculated to obtain the historical fault matching coefficient. s i ; Calculate the overall risk value R: In the formula, n is the total number of state parameters participating in the evaluation; Three preset risk thresholds T 1. T 2. T 3. Map the risk level based on the range of the comprehensive risk value: when R < T At 1 o'clock, it was classified as a level 1 low-risk area. At that time, it was a level 2 medium risk. At that time, it was a level three high-risk area. It was at level four, extremely high risk.

6. The intelligent sensing and early warning method for hoisting status according to claim 1, characterized in that, The execution logic of S5 is as follows: Level 1 Low Risk: The monitoring platform displays normal operating status, does not trigger on-site audible and visual warnings, and does not output safety constraint commands. Level 2 medium risk triggers on-site low-frequency audible and visual warnings, pushes warning information to on-site handheld terminals and monitoring platforms, and outputs deceleration constraint commands to the hoisting equipment control system; Level 3 high risk triggers on-site medium-frequency audible and visual warning, pushes alarm information to on-site terminals and monitoring platforms, outputs limit constraint commands to the hoisting equipment control system, and locks the operation authority of the mechanism in the dangerous direction; Level 4 extremely high risk triggers on-site high-frequency audible and visual warnings, pushes emergency alarm information to all terminals, outputs emergency stop commands to the hoisting equipment control system, and triggers the equipment safety braking mechanism; All warning events and control commands are synchronously stored in the safety log database for post-event traceability and model iteration optimization.

7. The intelligent sensing and early warning method for hoisting status according to claim 1, characterized in that, The multi-feature fusion deep learning model also includes an online incremental update mechanism: at fixed times each day, newly added normal operation samples and risk event samples are added to the training set, and the learning rate is used to fine-tune the top-level parameters of the model, update the feature extraction and fusion weights, and continuously improve the model's adaptability to the on-site environment.

8. A hoisting status intelligent sensing and early warning system, characterized in that, The intelligent sensing and early warning method for hoisting status according to any one of claims 1-7 includes: The data acquisition module deploys a multimodal sensor network on the hoisting equipment and at the work site to collect real-time operating parameters of the hoisting equipment, attitude parameters of the hoisted object, parameters of the working environment, and on-site visual image data. The data preprocessing module preprocesses the collected multi-source heterogeneous data sequentially and completes the spatiotemporal alignment of multi-channel data based on a unified time base to generate a standardized hoisting status dataset. The model recognition module inputs a standardized hoisting status dataset into a pre-trained multi-feature fusion deep learning model, extracts multi-dimensional features and performs fusion calculations to identify the current hoisting operation's working condition type and real-time status parameters. The risk level mapping module matches and compares the identified real-time status parameters with preset multi-level safety thresholds, and calculates the comprehensive risk value of the current operation by combining the historical fault sample library and the working condition weight coefficient, and maps the corresponding risk level. The early warning module triggers corresponding audible and visual early warning signals based on the assessed risk level and pushes them to the on-site terminal and monitoring platform. At the same time, it outputs corresponding safety constraint commands to the hoisting equipment control system.

Images