Unsafe hoisting behavior identification method based on multi-sensor data fusion

Abstract

The invention discloses an unsafe hoisting behavior identification method based on multi-sensor data fusion, which comprises the following steps of: 1, acquiring multi-dimensional monitoring data in real time through multi-sensor data acquisition; 2, data preprocessing and data fusion are carried out, data synchronization and alignment, data cleaning and abnormal value processing, filtering and denoising, feature extraction and data standardization normalization processing are carried out on the multi-dimensional monitoring data in sequence, and then a multi-source fusion feature matrix is constructed by adopting a weighted fusion algorithm; 3, screening core features associated with the unsafe hoisting behaviors from the multi-source fusion feature matrix; 4, establishing a quantitative safety rule base based on deterministic identification of the safety rule base, matching the core features with rule base judgment logic, and outputting a preliminary identification result; and 5, identification result verification and early warning: correcting the preliminary identification result through multi-source data consistency verification and time sequence continuity verification, and triggering graded early warning according to the correction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of construction safety monitoring, and particularly relates to an unsafe hoisting behavior identification method based on multi-sensor data fusion. BACKGROUND

[0002] As the core heavy equipment in construction, the safety of the hoisting operation of the tower crane is directly related to the safety of the construction personnel and the safety of the project property. According to relevant accident statistics, most tower crane accidents are closely related to unsafe hoisting behaviors, such as oblique lifting of heavy objects, overloading, sudden braking, sudden unloading and the like. Once these unsafe behaviors occur, they can easily lead to serious consequences such as swinging of the hoisted object, breaking of the steel wire rope, deformation of the hoisting arm and even overall collapse.

[0003] The existing unsafe behavior identification method mainly relies on a single sensor and threshold judgment, and has the following shortcomings:

[0004] Firstly, the data dimension is one-sided. A single sensor cannot comprehensively reflect complex working conditions. For example, only relying on the tension sensor cannot accurately identify the oblique lifting of heavy objects, and the reliable conclusion can be obtained only by combining the hook horizontal deviation rate or the rope angle data;

[0005] Secondly, the environmental interference is significant. The sensor is easily disturbed by factors such as vibration, wind and temperature in the construction site. For example, the instantaneous abnormal value generated by the tension sensor when the hoisted object swings will cause false positives, and the incomplete data collected by the camera in foggy or strong light will also cause recognition blind spots;

[0006] Thirdly, there is a lack of fusion mechanism. Some existing multi-sensor solutions simply superimpose the threshold results of different sensors, and fail to form an effective fusion and judgment mechanism, resulting in insufficient recognition accuracy and robustness of complex unsafe behaviors, and it is difficult to meet the high reliability requirements of safety monitoring in actual engineering.

[0007] Although some researches try to introduce multi-sensor, they are still limited to simple superposition or independent judgment of data, and fail to establish an effective cross-modal information fusion and collaborative decision mechanism, resulting in that the overall recognition rate and system robustness of diversified and complex unsafe behaviors are still difficult to meet the actual engineering needs. Therefore, it is urgent to build a multi-sensor fusion identification method that can cover typical and extended unsafe hoisting behaviors and has high accuracy and high reliability. SUMMARY

[0008] The technical problem to be solved by the present application is to provide an unsafe hoisting behavior identification method based on multi-sensor data fusion to solve the problems of the prior art.

[0009] In order to solve the above technical problems, the application discloses an unsafe hoisting behavior recognition method based on multi-sensor data fusion, comprising the following steps:

[0010] Step 1: multi-sensor data acquisition, deploying sensor arrays on the tower crane hook, jib, luffing trolley, braking system and cab, and collecting multi-dimensional monitoring data in real time;

[0011] Step 2: data preprocessing and data fusion, sequentially performing data synchronization and alignment, data cleaning and outlier processing, filtering and denoising, feature extraction and data standardization and normalization processing on the multi-dimensional monitoring data, and then adopting a weighted fusion algorithm to construct a multi-source fusion feature matrix;

[0012] Step 3: core feature extraction, screening core features associated with unsafe hoisting behaviors such as cable-stayed hoisting, overload hoisting, sudden braking and sudden unloading from the multi-source fusion feature matrix;

[0013] Step 4: deterministic recognition based on a safety rule base, establishing a quantitative safety rule base, matching the core features with the rule base judgment logic, and outputting a preliminary recognition result;

[0014] Step 5: verification and early warning of the preliminary recognition result, correcting the preliminary recognition result through multi-source data consistency verification and time series continuity verification, and triggering a graded early warning according to the correction result.

[0015] The multi-dimensional monitoring data includes hoisting load tension data of an axle pin tension sensor, jib / tower body inclination data of a dual-axis inclination sensor, hoisting object acceleration data of a three-dimensional acceleration sensor, lifting / luffing / rotary motor speed of an incremental rotary encoder, luffing trolley displacement data of a pull wire displacement sensor, environmental wind speed data of a wind speed sensor and hoisting environment image data of an industrial camera.

[0016] The sensor array deployment and parameters in step 1 are as follows:

[0017] The axle pin tension sensor is installed on the hook pulley block, with a range of 0-120t, an accuracy of ±0.1% FS and a collection frequency of 10Hz;

[0018] The dual-axis inclination sensor is installed at the head of the jib and the middle of the tower body, respectively, with a range of ±30°, an accuracy of 0.1° and a collection frequency of 50Hz;

[0019] The three-dimensional acceleration sensor is installed at the top of the hoisting rope, with a range of ±10g and a sampling rate of 100Hz;

[0020] The pull wire displacement sensor is installed at the non-moving end of the jib root and the luffing trolley, with a range of 0-60m, an accuracy of ±0.5cm and a sampling rate of 10Hz;

[0021] Incremental rotary encoder is installed on the winding shaft or slewing drive shaft of the lifting / variable amplitude / slewing mechanism, with a resolution of 1024 ppr;

[0022] Wind speed sensor is installed on the top of the boom or the upper part of the tower, with a range of 0-60 m / s, an accuracy of ±0.5 m / s FS, and a collection frequency of 10 Hz;

[0023] Industrial camera is installed below the cab, with a resolution of 1920×1080, a frame rate of 30 fps, and support for H.265 encoding;

[0024] Each sensor is connected to the edge computing gateway through the RS485 bus or Ethernet, and Kalman filtering is used to correct data and compensate for drift error, realizing real-time data transmission and timestamp synchronization, with a synchronization error of less than 10 ms.

[0025] Each sensor is connected to the edge computing gateway according to the difference in data type: six types of numerical sensors such as shaft pin tension sensors and double-axis inclination sensors are connected through the "hand-in-hand" topology RS485 bus (with 120Ω terminal resistance at both ends of the bus, shielded twisted pair anti-interference, Modbus-RTU protocol with 9600 bps baud rate, 8-bit data bit, and 1-bit stop bit, and gateway 100 ms interval cycle collection and analysis); industrial camera is connected through gigabit Ethernet, and image stream is pushed through static IP configuration combined with H.265 encoded RTSP protocol.

[0026] Timestamp synchronization and data correction are based on the gateway industrial RTC clock, which generates a hardware synchronization trigger signal every 100 ms to trigger sensor collection; numerical sensor data is corrected in real time through Kalman filtering algorithm to compensate for drift error; for the slight time offset of industrial camera and numerical sensor, dynamic time warping (DTW) algorithm is used to calibrate the time axis, combined with sensor hardware delay calibration and software interpolation compensation, to ensure that the data synchronization error is less than or equal to 10 ms, laying a foundation for time consistency for subsequent fusion.

[0027] The data preprocessing and fusion process described in step 2 is as follows:

[0028] Step 2-1: Data synchronization and alignment, hardware trigger sensor access to the same collection card, with internal clock providing sampling reference, realizing hardware level time synchronization; for heterogeneous sensors that cannot be directly synchronized, dynamic time warping (DTW) algorithm is used to calibrate the time axis of heterogeneous sensor data, ensuring data alignment;

[0029] Step 2-2: Data cleaning and outlier processing, using improved 3σ criterion combined with Isolation Forest algorithm: take n50 sampling points as a sliding window to calculate the mean μ and standard deviation σ, preliminarily screen suspected outliers | x - μ | > 3σ, where x represents a single sensor sampling data point to be determined in the sliding window, input the Isolation Forest model (abnormal score threshold 0.6) to determine and remove outliers, and linear interpolation to fill in the gaps;

[0030] Step 2-3: Filter denoising, forward-backward filtering with Butterworth low-pass filter (cutoff frequency 5Hz, sampling rate 50Hz, order 4), realizing zero-phase delay noise smoothing, and the cutoff frequency can be adjusted as needed;

[0031] Step 2-4: Feature extraction, extract time domain features (mean, variance, peak) and frequency domain features (FFT main frequency, spectral energy) in the 1 second (50 sampling points) sliding window corresponding to n sampling points, reduce dimensionality by principal component analysis (PCA), retain principal components with cumulative contribution rate greater than threshold ≥ 90%, form optimized feature vector; The optimized feature vector is the basic input for constructing the multi-source fusion feature matrix in step 2-6, and further supports the feature input of step 3 core feature selection, step 4 safety rule base matching and step 5 verification model, providing standardized feature data for unsafe behavior identification.

[0032] Step 2-5: Standardize / normalize the filtered and denoised data, Min-Max standardization for numerical data: Mapping to [0, 1] interval, where x min and x max are the minimum and maximum values of the feature in the 10-minute sliding window, and x norm is the normalized feature value, and image data is enhanced in contrast using CLAHE algorithm.

[0033] Step 2-6: Weighted fusion of feature groups contained in the optimized feature vector extracted in step 2-4, dynamically adjust feature group weights according to task relevance and information entropy of identification task (mechanics: 0.3~0.6, posture: 0.2~0.5, motion: 0.1~0.3, image: 0~0.2, image data belongs to visual supplementary data in environmental perception dimension); Finally, construct a multi-source optimized fusion feature matrix.

[0034] Specifically: based on the task correlation and information entropy distribution weight, form an optimized fusion feature matrix: identify overloaded tasks to improve mechanical feature weight (0.5-0.6), identify cable-stayed tasks to improve attitude features (including hook horizontal offset rate) weight (0.4-0.5); Intra-group features are assigned weights based on information entropy (the more information, the higher the weight); Image data is first converted into probability features by a lightweight CNN or SVM, and then fused with other modalities at the decision level to ensure that the fusion matrix can focus on the current task and maximize the use of multi-source information.

[0035] The feature group described in step 2-6 refers to dividing the optimized feature vector extracted in step 2-4 into four categories of mechanical, attitude, motion, and environment (including image visualization data) according to physical meaning and data source; Within each feature group, information entropy (used to quantify the effective decision information contained in the feature, the higher the entropy value, the stronger the feature's discrimination for unsafe behavior recognition) is used to calculate the information amount of each feature, and the larger the information amount, the higher the weight, achieving adaptive weight distribution within the group; At the decision level, the class probability distribution output by the image data after being processed by the classification model (such as SVM), and the decision results of other feature groups are probabilistically fused (such as the product rule based on evidence theory, used to solve the conflict of multi-source decision results, and enhance the reliability of the fusion conclusion),

[0036] Step 3 core feature extraction process:

[0037] Step 3-1, construct a multi-modal perception "unsafe behavior type-core feature" mapping system, extract the core features according to the mechanical perception dimension, attitude perception dimension, motion perception dimension, and environmental perception dimension;

[0038] Step 3-2, evaluate the importance of core features through Gradient Boosting Decision Tree (GBDT), and select core features with importance score ≥0.05; Support the expansion of new behavior feature dimension, and realize the compatible fusion of new and old features through domain adaptive algorithm.

[0039] The unsafe behavior types described in step 2-3 include: cable-stayed lifting, overloaded lifting, non-stable operation, turning back when turning, sudden speed change when turning, speed and load mismatch, unstable lifting with binding, blind area rapid operation, lifting embedded objects, lifting weight beyond the rated lifting capacity, resetting operation immediately after emergency stop, variable speed motion exacerbating vibration, over-range operation during operation, not using the minimum lifting speed before the lifting wire is tightened, not using the minimum lowering speed when lowering the heavy object into place, etc.

[0040] Mechanical perception dimension: mainly corresponding to overloading hoisting, unstable hoisting, lifting embedded objects, and hoisting at the far end of the rated load. The corresponding extracted core features include basic features such as hoisting tension, hook two-side tension difference, and optional derived features such as hoisting torque, brake pressure change rate, and tension sudden change value, which are used to reflect the dynamic changes of load and braking process.

[0041] Posture perception dimension: mainly corresponding to oblique lifting of heavy objects, and assisting in judging unstable bundling and overloading behaviors; the corresponding extracted core features include basic features such as hook horizontal offset rate (derived from three-dimensional acceleration integration + working amplitude), tower body verticality, and working amplitude; further calculation of rope inclination angle (derived from three-dimensional acceleration) and amplitude overload rate can be used to depict posture abnormalities and offset degree; among them, the hook horizontal offset rate directly reflects the horizontal offset proportion of the hook relative to the center of rotation, which is the core index for judging oblique lifting of heavy objects.

[0042] Motion perception dimension: mainly corresponding to non-stationary operation, turning back when turning, sudden speed change when turning, speed and load mismatch, immediate reset operation after emergency stop, increased oscillation caused by variable speed motion, overgear operation, not using the lowest speed before lifting, and not using the lowest speed when lowering into position. The corresponding extracted core features include basic features such as motor speed and hoisted object horizontal / vertical acceleration; the derived hoisting / variable amplitude / rotation speed, rotation angle acceleration, and hoisted object bounce height are preferred to depict motion trends and instability;

[0043] Environmental perception dimension: mainly corresponding to blind area fast operation, and providing environmental correction in oblique lifting and lowering into position judgments. The corresponding extracted core features include basic features such as wind speed, wind direction, light intensity, and hoisted object and obstacle distance; optional extraction of wind speed overload rate, hoisted object and obstacle overlap area, environmental adaptation degree, and wind load influence coefficient can be used to evaluate the interference degree of environment on operation safety, such as wind speed > 8 m / s, which can modify the hook horizontal offset rate judgment threshold.

[0044] Step 4 safety rule base quantitative judgment logic is:

[0045] The rule base is structured and stored according to the six elements of "unsafe behavior type - judgment index group - threshold interval matrix - duration threshold - associated feature weight - priority level", supporting dynamic addition of rules; the judgment logic of unsafe behavior is as follows:

[0046] Oblique lifting of heavy objects: meet any two of the following conditions and last for ≥2s, ① hook horizontal offset rate > 8% (relaxed to 10% when wind speed > 8m / s), ② rope inclination angle > 5°, ③ hook two-side tension difference > 10%;

[0047] Overload lifting: ① Actual lifting weight / Current amplitude rated lifting weight ratio≥1.0 and lasts≥2s, ② The ratio 1.0-1.1 is slight overload, 1.1-1.2 is moderate overload, ≥1.2 is serious overload;

[0048] Sudden braking: ① Brake pressure rate of change >10MPa / s, ② Lifting object horizontal acceleration >2g, ③ Lifting arm swing angle >3°, all three are met and last≥100ms;

[0049] Introducing support vector machine (SVM) auxiliary optimization under complex scenarios: input the core features into the pre-trained SVM model to get the behavior recognition confidence, i.e. SVM confidence (range [0,1], representing the credibility of the model determining this unsafe behavior); If the consistency with rule determination is≥90%, directly output the result, if it is<90%, determine through evidence theory fusion;

[0050] The "complex scenario" is defined as a working condition that meets any of the following conditions: ① Single unsafe behavior triggers≥3 judgment indexes in the rule library (such as cable-stayed lifting simultaneously triggering hook offset, lifting rope inclination, and tension difference indexes) ② Environmental interference parameters exceed the normal threshold (such as wind speed>8m / s, illumination intensity<200lux) ③ Multiple unsafe behavior characteristics appear (such as overload accompanied by lifting object swinging); If the complex scenario is not defined separately, the SVM auxiliary optimization can be introduced to all rule determination results to ensure the recognition reliability under all working conditions;

[0051] The "consistency with rule determination" is defined as the matching degree of the behavior recognition confidence obtained by the SVM model and the rule library determination result, which is calculated as follows: if the rule library determines that "there is an unsafe behavior", the consistency=SVM confidence; if the rule library determines that "there is no unsafe behavior", the consistency=1-SVM confidence; When the consistency≥90%, directly output the result, when the consistency<90%, determine through evidence theory fusion: Specifically, the SVM confidence and the rule library determination result are used as independent evidence sources, the historical verification accuracy of the two is combined to assign weights, and the comprehensive confidence is calculated through the evidence combination rule. If the comprehensive confidence≥0.8, it is determined that "there is an unsafe behavior", if the comprehensive confidence<0.8, it is determined that "there is no unsafe behavior";

[0052] The hook horizontal offset rate is calculated by multi-sensor data fusion: using a three-dimensional acceleration sensor installed on the hook, the horizontal displacement D is obtained by double integration; combined with the current working amplitude R obtained by the tension line displacement sensor or the amplitude encoder, the formula D / R*100% is used to calculate it.

[0053] The actual lifting load / current amplitude rated lifting load ratio is obtained by dividing the actual lifting load measured by the shaft pin tension sensor at the head of the lifting arm by the rated lifting load obtained from the amplitude measured by the luffing encoder and the rated load characteristic table pre-stored in the tower crane.

[0054] The brake pressure change rate is obtained by collecting the pressure signal P(t) in the hydraulic brake circuit at a fixed frequency, calculating the pressure difference between adjacent two sampling points, and dividing the sampling interval Δt, i.e. (P(t)-P(t-1)) / Δt.

[0055] Step 5 specifically comprises:

[0056] Step 5-1: Time series continuity verification: using long short-term memory network (LSTM) to predict feature trend and short-time Fourier transform (STFT) energy peak analysis, inputting the core features of the current time window T (1 second) to predict the trend of the next time window (100 ms);

[0057] Step 5-2: Based on the analysis result of step 5-1, the preliminary identification result obtained in step 4 is screened: first, it is judged whether it is “instantaneous impact” (the feature suddenly rises to trigger the step 4 rule, but the LSTM predicts that the next time window will fall), if so, the preliminary result is excluded; if not, further verification is carried out: if the STFT energy peak (the frequency energy aggregation formed by the fluctuation of the feature) is detected for more than 5 consecutive time windows (≥500 ms) and the LSTM predicts that the feature is stable and exceeds the limit, the preliminary result is retained and output to step 5-3 for multi-source data consistency verification.

[0058] The “instantaneous impact” determination condition is: if the “feature suddenly rises” but the LSTM predicts that the next time window will fall, it is determined as instantaneous impact;

[0059] The “feature suddenly rises” refers to the change of the core feature in the time period T exceeding the pre-set threshold (such as lifting force > 5 kN / 100 ms, lifting rope inclination angle > 2° / 100 ms);

[0060] Step 5-3: Bayesian inference algorithm is introduced for multi-source data consistency verification, and the “continuous over-limit preliminary result” (the result is excluded from the instantaneous interference by the time series continuity verification in step 5-1, and meets the condition of “detecting the STFT energy peak for more than 5 consecutive time windows and the LSTM predicting that the feature is stable and exceeds the limit”) reserved after step 5-2 screening, is assigned a sensor credibility weight to construct a probability model, and the “real unsafe behavior” posterior probability and weighted consistency index are calculated When the posterior probability ≥ 0.8 and the WCI ≥ 0.7, it is determined that the identification is effective; when the WCI < 0.7 or the posterior probability < 0.8 (such as the shaft pin type tension sensor shows overload, but the industrial camera recognizes the volume of the hoisted object and the derived weight is normal), the system automatically triggers self-diagnosis (checks the sensor communication log, zero drift value), shields the abnormal data source and corrects the result, and after correction, if the posterior probability ≥ 0.8 and the WCI ≥ 0.7, it is confirmed that the identification is effective, and if it still does not meet the requirements, the sensor is reviewed to exclude faults;

[0061] The behavior occurrence probability corresponding to each sensor data is taken as the prior probability, and the posterior probability is the probability of "real unsafe behavior" calculated by the Bayes formula, with a value range of 0-1;

[0062] The posterior probability of "real unsafe behavior" is calculated by the Bayes formula; the weighted consistency index WCI is calculated by the formula:

[0063]

[0064] That is, the weighted sum of sensor weight and matching degree, with a range of 0-1, wherein, is the sensor reliability weight, is the matching degree of the sensor and the reference feature (the "reference feature" refers to the standard feature threshold of the corresponding unsafe behavior in the preliminary identification result output by step 4 based on the safety rule library, such as identifying "overload hoisting", the reference feature is "actual hoisting weight / rated hoisting weight ratio ≥ 1.0"); the "matching degree" is calculated by the closeness of the feature value collected by the sensor in real time and the reference feature threshold, the closer to the threshold, the higher the matching degree, and the result is mapped to 0-1;

[0065] The step 5 described above triggers a hierarchical warning according to the correction result, which specifically includes:

[0066] Fuzzy logic is used to evaluate the warning level, with the core features as input, and the membership function is used to determine the final level:

[0067] The first level warning (slight overload, slight inclined hoisting) triggers an audible and light prompt; the second level warning (sudden braking, sudden unloading) limits the operation speed; the third level warning (serious overload, serious inclined hoisting) triggers an emergency shutdown and an alarm.

[0068] The "unsafe behavior type-core feature" mapping system is used to add new behaviors according to the four-element supplement rule. The transfer learning adaptation model is realized by "pre-training model parameter fine-tuning + a small amount of new behavior sample training". It does not need to build a model from scratch, quickly adapts to the recognition needs of new unsafe behaviors, and the transfer learning adaptation model refers to the support vector machine (SVM) model in claim 6 for auxiliary optimization in complex scenes, and the long short-term memory network (LSTM) model in claim 9 for time series continuity verification.

[0069] Specifically, the rule base supports dynamic management throughout the life cycle: when a new unsafe behavior type is added, the rule is entered according to the six elements through the visual configuration interface (such as adding "swing amplitude of hoisted objects", configuring the index as "swing angle of hoisted objects > 15°" and "swing frequency > 0.5Hz", and the threshold interval is divided into three levels, and the duration is ≥1.5s), the system automatically associates the corresponding features in the multi-source fusion feature matrix (such as calling the swing angle feature derived from the three-dimensional acceleration sensor), without modifying the underlying recognition algorithm; At the same time, it supports rule validity verification (new rule conflict detection with existing rules), priority calculation and gray release (first run on a single tower crane, and when the accuracy is ≥95%, deploy it in full), and combine with the cloud database to realize threshold optimization and rule recommendation, to ensure the safety, adaptability and reliability of rule base iteration.

[0070] Beneficial effects:

[0071] 1. Comprehensive recognition dimension, wide scene coverage: Deploy multi-modal sensors to build a comprehensive perception system for mechanics, posture, motion, and environment, combined with gradient boosting decision tree to filter core features, which can not only accurately identify traditional unsafe behaviors such as overload and inclined pulling, but also can be flexibly incorporated into new behaviors such as swing amplitude exceeding limits through feature extension interface and domain adaptive algorithm, overcoming the limitations of single sensor, and adapting to high-rise buildings and complex weather conditions.

[0072] 2. Strong anti-interference ability, significantly reduced false positive rate: Data preprocessing uses "improved 3σ rule + isolation forest" to remove outliers, and adaptive Butterworth filter to suppress noise; Time synchronization uses Kalman filter to compensate for sensor drift and dynamic time warping (DTW) to correct data drift; In the recognition stage, Bayesian inference and LSTM are used for multi-source verification to effectively avoid interference and false positives, reducing the false positive rate by more than 40% compared to traditional methods.

[0073] 3. Good environmental adaptability and high robustness: Dynamic adaptive weighted fusion is used, combined with attention mechanism to adjust sensor weights in different environments (such as reducing image feature weights in foggy weather); When new sensors or dimensions are added, domain adaptive algorithm is used for quick adaptation without the need to restructure the system, ensuring stable operation in complex weather conditions such as strong winds and night, with a continuous operation reliability of ≥99%.

[0074] 4. Decision transparency and traceability, flexible extension: the core decision is based on the quantified rule base, the decision process is structured and presented, which is convenient for audit traceability; the hybrid mode of "rule base + SVM / migration learning" is introduced to optimize the decision of complex scenarios; the behavior of new extension only needs to configure the rules according to six elements, and the model adaptation is completed by combining migration learning, without modifying the underlying algorithm, and the extension efficiency is improved by 60%.

[0075] 5. Real-time and security, rapid response: the whole process is optimized and designed, the sensor synchronization error is less than or equal to 10ms, the end-to-end delay is less than or equal to 500ms, which meets the real-time monitoring; the risk is dynamically evaluated by combining the hierarchical early warning and fuzzy logic, which supports early warning upgrade / downgrade and emergency shutdown, and is matched with closed-loop management to improve the timeliness of safety control. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 The whole method flowchart of the application.

[0077] Figure 2 The data processing flowchart of the application.

[0078] Figure 3 The schematic diagram of identification and decision of the application. DETAILED DESCRIPTION

[0079] In this embodiment, combined with a QTZ125 type tower crane (rated maximum lifting weight 12t, maximum working amplitude 60m, tower body height 220m) used in a super high-rise building project, in the "core tube steel structure hoisting" scene, taking the identification of two typical unsafe hoisting behaviors of "overload hoisting" and "cable-stayed hoisting" as an example, the specific implementation process of the application is described, and the whole process is as shown in Figure 1 The skilled in the art can extend the application to other unsafe hoisting behaviors such as sudden braking and long-time suspension of heavy objects based on the technical solution of this embodiment, which all fall within the protection scope of the application.

[0080] Step 1: Multi-sensor data acquisition, deploying sensor arrays on the tower crane hook, boom, luffing trolley, braking system and cab to collect multi-dimensional monitoring data in real time;

[0081] Step 2: Data preprocessing and data fusion, sequentially performing data synchronization and alignment, data cleaning and outlier processing, filtering and denoising, feature extraction and data standardization and normalization processing on the multi-dimensional monitoring data, and then using a weighted fusion algorithm to construct a multi-source fusion feature matrix;

[0082] Step 3: Core feature extraction, screening core features associated with unsafe hoisting behaviors from the multi-source fusion feature matrix;

[0083] Step 4: Deterministic identification based on safety rule library, establishing a quantitative safety rule library, matching the core features with rule library decision logic, and outputting preliminary identification results;

[0084] Step 5: Verification and early warning of preliminary identification results, correcting preliminary identification results through multi-source data consistency verification and time series continuity verification, and triggering hierarchical early warning based on the corrected results.

[0085] The multi-dimensional monitoring data includes: hoisting load tension data of the shaft pin tension sensor, hoist arm / tower body inclination data of the dual-axis inclination sensor, hoisted object acceleration data of the three-dimensional acceleration sensor, hoisting / amplitude / rotation motor speed of the incremental rotary encoder, amplitude trolley displacement data of the displacement sensor, environmental wind speed data of the wind speed sensor, and hoisting environment image data of the industrial camera.

[0086] Step 1: Sensor deployment and data collection

[0087] In combination with the structural characteristics of the QTZ125 tower crane and the identification requirements of the two types of unsafe behaviors, a multi-sensor array deployment scheme is adopted, and the types, installation positions, and functions of various sensors are as follows:

[0088] Shaft pin tension sensor: HBM U9B type (range 0-120t, accuracy ±0.1% FS, sampling frequency 10Hz) is selected and symmetrically installed on both sides of the hoist hook pulley set load shaft pin; core collection of hoisting load tension data provides basic mechanical characteristics for "overload hoisting", and auxiliary "inclined hoisting" judgment is provided through the tension difference on both sides; data is corrected in real time by Kalman filter algorithm to compensate for long-term running drift error.

[0089] Dual-axis inclination sensor: Bosch BNO055 type (range ±30°, accuracy 0.1°, sampling frequency 50Hz) is selected and installed at the front of the hoist arm and the middle of the tower body (80m height from the ground); the former collects the inclination of the hoist arm and the latter monitors the verticality of the tower body to provide data support for tower body stability judgment;

[0090] Three-dimensional acceleration sensor: ADI ADXL377 type (range ±10g, sampling rate 100Hz) is selected and fixed at the top of the hoist rope (1.5m from the hook); it collects horizontal / vertical acceleration (such as x-axis horizontal acceleration 1.96m / s²), calculates the horizontal offset distance 1.96m within 1 second through double integration, and deduces the hoist rope inclination angle 7°, which provides core data for the calculation of the hook horizontal offset rate of "inclined hoisting";

[0091] Wire displacement sensor: Micro-Epsilon WDS-1000 type (range 0-60 m, accuracy ±0.5 cm, sampling rate 10 Hz) is selected and fixed at the non-moving end of the boom root, and the wire end is connected to the variable amplitude trolley; the displacement of the variable amplitude trolley is measured 24 m, i.e. the current working amplitude of the boom is 24 m, combined with the horizontal offset distance of 1.96 m, the horizontal offset rate of the hook is calculated = (1.96 / 24) x 100% ≈ 8.17%;

[0092] Industrial camera: Hikvision MV-CA050-11GM type (5 million pixels, frame rate 30 fps, low illumination 0.01 lux) is selected and arranged under the cab (shooting the hoisted object) and at the head of the boom (shooting the hoisting rope) respectively; after collecting image data, the Retinex+CLAHE algorithm is enhanced, the volume of the hoisted object and the tilt state characteristics of the hoisting rope are extracted, and multi-source data cross verification is assisted.

[0093] Wind speed sensor: Lascar WS300 type (range 0-60 m / s, accuracy ±0.5 m / s, sampling frequency 10 Hz) is selected and installed on the windward surface of the top of the boom; monitor the environmental wind speed of 6 m / s (less than the rated wind speed of 12 m / s), which is used to correct the hook horizontal offset rate determination threshold value (no need to relax, still 8%) of "cable-stayed hoisting heavy objects" and dynamic load compensation coefficient of "overload hoisting";

[0094] Each sensor is connected to the NVIDIA Jetson AGXXavier edge computing gateway through "RS485 bus + gigabit Ethernet" hybrid networking: 6 numerical sensors are transmitted through RS485 bus with Modbus-RTU protocol (baud rate 9600 bps, 100 ms interval collection), industrial camera pushes image stream through Ethernet with H.265 encoded RTSP protocol; the gateway takes the industrial RTC clock as the reference, generates a hardware synchronization trigger signal every 100 ms, calibrates the heterogeneous data time axis with dynamic time warping (DTW) algorithm, and ensures that the synchronization error of the whole link data is ≤8 ms; at the same time, it is equipped with 24V UPS voltage stabilizer, which can ensure continuous collection within 30 minutes after power failure.

[0095] Step 2. Data preprocessing and fusion

[0096] According to the characteristic differences of "overload hoisting" and "cable-stayed hoisting heavy objects", a differentiated preprocessing and fusion process is constructed, as shown in Figure 2 , and the specific process is as follows:

[0097] Synchronous alignment: taking the gateway hardware trigger signal as the reference, all sensor data are uniformly resampled to 10 ms time interval, and the time and space correspondence table of industrial camera image frames and numerical data (such as horizontal acceleration, working amplitude) is established, ensuring the consistency of data time sequence required for calculating the hook horizontal offset rate;

[0098] Outlier processing:

[0099] For the tension data associated with "overload lifting", use "improved 3σ rule + isolation forest algorithm": take 50 sampling points (1 second) as the sliding window, calculate the mean μ and standard deviation σ, and preliminarily screen out suspected outliers with |x-μ|>3σ; input the isolation forest model (abnormal score threshold 0.6) for accurate determination and removal, and complete the data through linear interpolation of the redundant data of the two side tension sensors.

[0100] For the horizontal acceleration data associated with "cable-stayed lifting heavy objects", use "sliding window smoothing + zero drift calibration": smooth the data with a 10-point window, and calibrate the zero point once every 1 minute to avoid the calculation error of horizontal offset distance caused by integral drift; if the horizontal acceleration suddenly changes but the wind speed is stable, it is determined that the sensor has a transient fault, and the data is repaired using cubic spline interpolation of the previous and subsequent 5 normal data points;

[0101] Filtering and denoising: use adaptive Butterworth low-pass filtering algorithm to dynamically adjust the cutoff frequency after analyzing the signal frequency domain characteristics through wavelet transform: set the tension data (dominant frequency 1-3 Hz) to 3 Hz cutoff, and the attitude data (dominant frequency 0.5-2 Hz) to 5 Hz cutoff; use forward-backward filtering to eliminate phase delay and ensure that the filtered signal is aligned with the original event time.

[0102] Feature extraction and dimensionality reduction: extract features with a 1-second sliding window (50% overlap rate) and reduce dimensions through principal component analysis (PCA):

[0103] "Overload lifting": extract 12-dimensional features such as tension mean, peak value, variance, FFT dominant frequency, spectral energy, and boom amplitude, and retain 8-dimensional principal components with cumulative contribution rate ≥90% after PCA dimensionality reduction.

[0104] "Cable-stayed lifting heavy objects": extract 8-dimensional features including ①hook horizontal offset rate (8.17%), ②rope inclination angle (7°), ③hook tension difference on both sides (12%), ④horizontal acceleration mean (1.8 m / s²), and ⑤working amplitude (24 m); through PCA dimensionality reduction, retain 6-dimensional principal components (including hook horizontal offset rate principal component, contribution rate 18.5%) with cumulative contribution rate ≥90%;

[0105] Industrial camera image: extract 256-dimensional visual features through MobileNetV2, and convert them into probability feature vectors (such as "rope inclination" probability 0.92) through SVM.

[0106] Standardization and fusion:

[0107] The "overload lifting" feature adopts dynamic Min-Max standardization (updates the x_max corresponding to the rated lifting weight through a 10-minute sliding window), which is mapped to the [0, 1] interval;

[0108] The "inclined cable lifting" feature adopts Z-score standardization to highlight the degree of abnormal deviation of the posture;

[0109] Based on information entropy and attention mechanism, the weight distribution is as follows: in "overload lifting", the mechanical feature weight is 0.5, the motion feature weight is 0.3, the environmental feature weight is 0.15, and the image feature weight is 0.05; in "inclined cable lifting", the posture feature (including hook horizontal offset rate) weight is 0.45, the mechanical feature weight is 0.3, the environmental feature weight is 0.15, and the image feature weight is 0.1; after weighted fusion, the feature matrix is constructed (overload: 100x32 dimensions, inclined cable: 100x28 dimensions).

[0110] Step 3. Core feature extraction

[0111] Based on the multi-modal perception framework, combined with the gradient boosting decision tree (GBDT) algorithm (feature importance score threshold 0.05), the core features of the two types of behaviors are extracted in four dimensions of "mechanics / posture / motion / environment", and the specific examples are as follows:

[0112] Core features of overload lifting

[0113] Mechanical dimension: actual lifting weight 9t, rated lifting weight 8t under current amplitude (lifting weight ratio 1.125), lifting moment 270t・m (rated moment 240t・m), tension change rate 0.3t / s;

[0114] Motion dimension: lifting speed 0.8m / min, lifting acceleration 0.02m / s² (dynamic load coefficient 1.05);

[0115] Environmental dimension: real-time wind speed 6m / s (less than rated wind speed 12m / s, wind speed correction coefficient 1.0), image-derived lifting weight 9.2t (with a tension sensor error of 2.2%).

[0116] Core features of inclined cable lifting

[0117] Posture dimension: hook horizontal offset rate 8.17% (working amplitude 24m, horizontal offset distance 1.96m), rope inclination angle 7°, working amplitude 24m;

[0118] Mechanical dimension: left hook tension 8.8t, right hook tension 7.8t (tension difference 12%);

[0119] Environmental dimension: real-time wind speed 6 m / s (wind speed interference correction value for horizontal offset rate 0.1%, negligible), image recognition sling inclination angle 6.8° (error 0.2% with acceleration derived value). When it is necessary to expand the recognition of new behaviors such as "sudden braking", the new features such as "braking pressure change rate" are mapped to the existing feature space through the domain adaptive algorithm, without the need to reconstruct the extraction framework.

[0120] Step 4. Hybrid recognition based on "security rule base + intelligent optimization"

[0121] Call the preset quantitative security rule base for basic judgment, combine the SVM model with the evidence theory to optimize complex scene decision-making, and the specific process is as follows:

[0122] Rule base basic judgment:

[0123] Overload hoisting judgment rule

[0124] Judgment index: hoisting weight ratio ≥ 1.0 and duration ≥ 2s, or hoisting weight moment ratio ≥ 1.0 and duration ≥ 2s;

[0125] Correction condition: hoisting speed < 3 m / min (no need to lower threshold), wind speed < 8 m / s (no need for additional compensation);

[0126] Grade division: hoisting weight ratio 1.0-1.1 is slight overload, 1.1-1.2 is moderate overload, and ≥ 1.2 is serious overload; in this case, the hoisting weight ratio is 1.125 and the duration is 2.5s, which is preliminarily judged as "moderate overload".

[0127] Cable-stayed hoisting rule

[0128] Judgment index: hook horizontal offset rate > 8%, sling inclination angle > 5°, tension difference > 10%, any two items are met and the duration is ≥ 2s;

[0129] Correction condition: wind speed 6 m / s < 8 m / s (hook horizontal offset rate threshold remains 8%, no need to relax), hoisting object height from ground > 1m (no need to relax threshold);

[0130] Grade division: within 10% of the threshold value for slight cable-stayed, 10%-30% for moderate cable-stayed, and > 30% for serious cable-stayed; in this case, the hook horizontal offset rate exceeds 2.1% (8.17%-8%), the sling inclination angle exceeds 40% (7%-5%), and the tension difference exceeds 20% (12%-10%), meeting all three conditions and lasting 2.2s, which is preliminarily judged as "moderate cable-stayed".

[0131] Intelligent optimization correction

[0132] Input the core features of the two types of behaviors into the pre-trained SVM model:

[0133] Overload lifting scene: the model outputs "moderate overload" with a confidence of 0.93, and the consistency with the rule determination is ≥90%, directly confirming the recognition result;

[0134] Cable-stayed lifting scene: the model outputs "moderate cable-stayed" with a confidence of 0.88, and the consistency with the rule determination is <90%, the results of the two are fused through evidence theory: the rule determination trust degree is defined as 0.85, the model output trust degree is 0.88, and the synthesized comprehensive trust degree is 0.91 after synthesis, and finally it is confirmed as "moderate cable-stayed".

[0135] If new behaviors such as "lifting object swing amplitude exceeds limit" are added, only the rules need to be supplemented according to the four elements of "behavior type-determination index-threshold interval-duration", and the existing SVM model parameters are migrated to the new task through transfer learning, which can be completed with only 800 new samples.

[0136] Step 5. Verification and early warning of recognition results

[0137] The recognition results are corrected through multi-source verification, and the early warning level is dynamically evaluated through fuzzy logic, triggering differentiated control measures, such as Figure 3 as shown:

[0138] Multi-source data consistency verification: Bayesian inference algorithm is introduced to assign credibility weights to each sensor (tension sensor 0.9, three-dimensional acceleration sensor 0.85, wire displacement sensor 0.9, industrial camera 0.7):

[0139] Overload scene: tension data 9t (credibility 0.9) and image-derived weight 9.2t (credibility 0.7), calculate "real overload" posterior probability 0.89≥0.8, confirm recognition effective;

[0140] Cable-stayed scene: three-dimensional acceleration sensor-derived horizontal offset rate 8.17% (credibility 0.85), wire displacement sensor-derived working amplitude 24m (credibility 0.9), industrial camera-recognized lifting rope inclination angle 6.8° (credibility 0.7), calculate "real cable-stayed" posterior probability 0.87≥0.8, confirm recognition effective;

[0141] Time series continuity verification: long short-term memory network (LSTM) is used to predict feature trends:

[0142] Overload scene: the tension ratio is predicted to be stable at 1.12-1.13 within 100ms in the future, with no falling trend, excluding transient impact;

[0143] Cable-stayed scene: the hook horizontal offset rate continues to maintain 8.1%-8.2% within 100ms in the future, and the lifting rope inclination angle stabilizes at 6.9°-7.1°, confirming that the posture is abnormally stable and not transient swing;

[0144] Hierarchical early warning and control: combined with fuzzy logic algorithm to evaluate the final warning level and trigger corresponding measures:

[0145] Moderate overload (second-level warning): the control cabinet automatically limits the lifting speed ≤0.5 m / min and the rotation speed ≤5° / s; the cab alarms with sound and light, and the display screen displays "moderate overload (9t / 8t)" in real time; the alarm information (including time, location, and core data) is pushed to the cloud management platform, an early warning work order is generated and assigned to the on-site safety officer.

[0146] Moderate cable pull (second-level warning): the control cabinet automatically limits the rotation speed ≤3° / s and the amplitude speed ≤2 m / min; the display screen prompts "moderate cable pull (hook horizontal offset rate 8.17%, working amplitude 24 m)", guiding the operator to adjust the boom posture in the opposite direction of the offset; the platform records the early warning disposal process, and automatically removes the speed limit when the hook horizontal offset rate ≤7.5% and lasts for 30s;

[0147] Closed-loop management: the operator needs to confirm the disposal measures (such as "reduce the lifting speed to 0.4 m / min") on the platform, and the system archives the disposal records and sensor data in the corresponding period, forming a "detection-verification-warning-disposal-backtracking" safety management closed loop, which is convenient for subsequent safety audit and responsibility tracing.

[0148] The present application provides an unsafe hoisting behavior recognition method based on multi-sensor data fusion. There are many methods and ways to implement this technical solution. The above description is only the preferred embodiment of the present application. It should be noted that for ordinary technical personnel in this technical field, without departing from the principles of the present application, several improvements and refinements can be made, which should be considered as the protection scope of the present application. The components not explicitly described in the embodiment can be realized by existing technology.

Claims

1. An unsafe hoisting behavior recognition method based on multi-sensor data fusion, characterized in that, The method comprises the following steps: Step 1: multi-sensor data acquisition, deploying sensor arrays on the tower crane hook, jib, luffing trolley, brake system and cab, and collecting multi-dimensional monitoring data in real time; Step 2: data preprocessing and data fusion, sequentially performing data synchronization and alignment, data cleaning and outlier processing, filtering and denoising, feature extraction and data standardization and normalization processing on the multi-dimensional monitoring data, and then adopting a weighted fusion algorithm to construct a multi-source fusion feature matrix; Step 3: core feature extraction, screening core features associated with unsafe lifting behavior from the multi-source fusion feature matrix; Step 4: deterministic identification based on a safety rule base, establishing a quantitative safety rule base, matching the core features with the rule base judgment logic, and outputting a preliminary identification result; Step 5: verification and early warning of the preliminary identification result, correcting the preliminary identification result through multi-source data consistency verification and time series continuity verification, and triggering a hierarchical warning according to the corrected result.

2. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 1, characterized in that, The multi-dimensional monitoring data includes: hoisting load tension data of the shaft pin tension sensor, jib / tower body inclination data of the dual-axis inclination sensor, hoisted object acceleration data of the three-dimensional acceleration sensor, lifting / luffing / rotating motor speed of the incremental rotary encoder, luffing trolley displacement data of the displacement sensor, environmental wind speed data of the wind speed sensor, and lifting environment image data of the industrial camera.

3. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 2, characterized in that, The sensor array is deployed in step 1: The shaft pin tension sensor is installed on the hook pulley block; The dual-axis inclination sensor is installed on the jib head and the middle part of the tower body, respectively; The three-dimensional acceleration sensor is installed at the top of the hoisting rope; The displacement sensor is installed at the non-moving end of the jib root and the luffing trolley; The incremental rotary encoder is installed on the drum shaft or rotating drive shaft of the lifting / luffing / rotating mechanism; The wind speed sensor is installed at the top of the jib or the upper part of the tower; The industrial camera is installed below the cab; Each sensor is connected to the edge computing gateway through the RS485 bus or Ethernet, and the Kalman filter is used to correct the data and compensate for drift errors, realizing real-time data transmission and time stamp synchronization.

4. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 1, characterized in that, The data preprocessing and fusion process in step 2 is as follows: Step 2-1: data synchronization and alignment, hardware triggers sensor access to the same collection card, and the internal clock provides the sampling reference to realize hardware-level time synchronization; for heterogeneous sensors that cannot be directly synchronized, the dynamic time warping (DTW) algorithm is used to calibrate the time axis of heterogeneous sensor data to ensure data alignment; Step 2-2: data cleaning and outlier processing, using the improved 3σ rule combined with the isolation forest algorithm: taking n sampling points as a sliding window to calculate the mean μ and standard deviation σ, screening suspected outliers with | x - μ | > 3σ, where x represents a single sensor sampling data point in the sliding window, inputting the isolation forest model for judgment and removing outliers, and filling in the gaps by linear interpolation; Step 2-3: filtering and denoising, using forward-backward filtering of the Butterworth low-pass filter to achieve zero-phase delay noise smoothing, and adjusting the cutoff frequency as needed; Step 2-4: Feature extraction, extract time-domain and frequency-domain features in the sliding window corresponding to n sampling points, reduce dimensionality by principal component analysis, retain principal components with cumulative contribution rate greater than threshold value to form optimized feature vector; Step 2-5: Standardize / normalize the filtered de-noised data, Min-Max normalization for numerical data: Mapping to the interval [0, 1], where x min and x max are the minimum and maximum values of the feature within the sliding window, respectively, and x norm is the normalized feature value, and the image data is enhanced in contrast using the CLAHE algorithm; Step 2-6: Weighted fusion of feature groups contained in the optimized feature vector extracted in step 2-4, dynamically adjust feature group weights according to task relevance and information entropy of identification task to finally construct multi-source optimized fusion feature matrix.

5. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 4, characterized in that, The feature groups in step 2-6 refer to dividing the optimized feature vector extracted in step 2-4 into four categories of mechanics, posture, motion, and environment according to physical meaning and data source; within each feature group, information entropy is used to calculate the information amount of each feature, the greater the information amount, the higher the weight, realizing adaptive weight distribution of intra-group features; at the decision level, the class probability distribution output by processing image data through the classification model is probability fused with the decision results of other feature groups.

6. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 1, characterized in that, The core feature extraction process in step 3 is: Step 3-1, construct a "unsafe behavior type-core feature" mapping system for multi-modal perception, extract the core features according to the dimensions of mechanical perception, posture perception, motion perception, and environmental perception; Step 3-2, evaluate the importance of core features by gradient boosting decision tree, and select core features with importance score ≥ score threshold.

7. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 1, characterized in that, The quantitative decision logic of the safety rule base in step 4 is: The rule base is structured and stored according to six elements of "unsafe behavior type-determination index group-threshold interval matrix-duration threshold-correlation feature weight-priority level", supporting dynamic addition of rules; the determination logic of unsafe behavior includes: Cable-stayed hoisting: meet any two of the following conditions and the duration is ≥ t_d1, ① hook horizontal offset rate > offset threshold, ② cable inclination angle > inclination angle threshold, ③ difference in tension on both sides of the hook > tension difference threshold; Overload hoisting: ① actual hoisting weight / current amplitude rated hoisting weight ratio ≥ ratio threshold and duration ≥ t_d2, ② ratio meets interval 1 for slight overload, interval 2 for moderate overload, and interval 3 for severe overload; Sudden braking: ① brake pressure change rate > pressure threshold, ② hoisted object horizontal acceleration > acceleration threshold, ③ hoist arm swing angle > angle threshold, all three conditions are met and the duration is ≥ t_d3; Support vector machine is introduced to assist optimization in complex scenarios: input the core features into the pre-trained SVM model to obtain the behavior recognition confidence, i.e. SVM confidence; if the consistency with rule determination is ≥ consistency threshold, output the result directly, otherwise, determine through evidence theory fusion; The hook horizontal offset rate is calculated by multi-sensor data fusion: use the three-dimensional acceleration sensor installed on the hook to obtain the horizontal displacement D by double integration; combine the current working amplitude R obtained by the inhaul cable displacement sensor or amplitude encoder to calculate the value according to the formula D / R*100%; The actual hoisting weight / current amplitude rated hoisting weight ratio is obtained by dividing the actual hoisting weight measured by the shaft pin type tension sensor at the head of the hoist arm by the current amplitude rated hoisting weight obtained from the rated load characteristic table pre-stored in the tower crane factory; The brake pressure rate of change is obtained by collecting pressure signals P(t) at a fixed frequency by a pressure sensor in the hydraulic brake circuit, calculating the pressure difference of adjacent two sampling points, and dividing by the sampling interval Δt, i.e. (P(t)-P(t-1)) / Δt.

8. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 6, characterized in that, Step 5 specifically comprises: Step 5-1: Time series continuity verification: using long short-term memory network LSTM to predict feature trend and short-time Fourier transform (STFT) energy peak analysis, inputting the core features of the current time window T to predict the trend of the next time window; Step 5-2: Based on the analysis result of step 5-1, the preliminary identification result obtained in step 4 is screened: first, it is judged whether it is "instantaneous impact", if so, the preliminary result is excluded; if not, it is further verified: if the STFT energy peak is detected for more than n time windows and the LSTM predicted feature is stable and exceeds the limit, the preliminary result is retained and output to step 5-3 for multi-source data consistency verification; The "instantaneous impact" judgment condition is: if it meets "feature sudden rise" but the LSTM predicts that the next time window will fall, it is judged as instantaneous impact; The "feature sudden rise" refers to the change of core features in the time period T exceeding the preset threshold; Step 5-3: Introducing Bayesian inference algorithm for multi-source data consistency verification, assigning sensor credibility weight to build a probability model for "real unsafe behavior" posterior probability and weighted consistency index When the posterior probability ≥ n1 and WCI ≥ n2, it is determined that the identification is valid; when WCI < n2 or the posterior probability < n1, the system automatically triggers self-diagnosis, shields the abnormal data source and corrects the result. After correction, if the posterior probability ≥ n1 and WCI ≥ n2, it is confirmed that the identification is valid, and if it still does not meet the requirements, the sensor is excluded for troubleshooting.

9. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 8, wherein the step 5 triggers a hierarchical warning according to the correction result, specifically comprising: Using fuzzy logic to evaluate the warning level, taking the core features as input, and determining the final level through membership function: Level one warning triggers sound and light prompt; Level two warning limits operation speed; Level three warning emergency shutdown and alarm.

10. The unsafe lifting behavior recognition method based on multi-sensor data fusion according to claim 9, characterized in that, The "unsafe behavior type-core feature" mapping system is supplemented according to the four-element rule for new behaviors, and the transfer learning adaptation model is realized through "pre-training model parameter fine-tuning + a small amount of new behavior sample training", the transfer learning adaptation model refers to the support vector machine SVM model for complex scene auxiliary optimization in claim 6, and the long short-term memory network LSTM model for time series continuity verification in claim 9.

Images