Intelligent tower crane multi-sensor fusion anti-collision identification control system

Abstract

The application discloses a kind of intelligent tower crane multi-sensor fusion anti-collision identification control systems, belong to big data processing technical field;Through the complementary layout of multi-view sensor and geometric reasoning completion technology, the visual dead angle problem that is most likely to cause collision accident in tower crane operation is solved;By introducing real-time wind speed disturbance and emergency stop acceleration correction term, the large amplitude swing characteristics of load in strong wind weather and emergency braking can be accurately simulated, and with dynamic boundary expansion mechanism, the defect that traditional position interpolation method seriously underestimates the range of dangerous area is avoided;By constructing a multi-factor weighted risk assessment model based on three-dimensional intersection ratio, safety distance ratio and relative speed, the collision risk is converted into a continuous probability value between 0 and 1, replacing the traditional binary decision mode of collision or not;By adopting linear speed limiting mechanism, when the risk probability exceeds the secondary threshold, the tower crane operating speed gradually decreases with the increasing risk, rather than suddenly cutting off power.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, specifically to a multi-sensor fusion anti-collision recognition control system for intelligent tower cranes. Background Technology

[0002] When implementing existing multi-sensor fusion anti-collision identification and control schemes for tower cranes, the most dangerous collisions often occur between the suspended load and personnel or facilities below. However, the field of view of the sensors at the end of the boom is easily obstructed by the trolley, wire rope, and the suspended load itself, creating a blind spot below the hook. At the same time, the load produces pendulum-like swaying during lifting and rotation, and the sway envelope is affected by wind speed and sudden stops. Simple position interpolation will seriously underestimate the danger zone, resulting in predictive uncertainties caused by the blind spot below the hook and the large swing of the load. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent tower crane multi-sensor fusion anti-collision recognition control system to solve the technical problems mentioned in the background art.

[0004] The objective of this invention can be achieved through the following technical solutions: A smart tower crane multi-sensor fusion anti-collision recognition control system includes: The multi-source sensing data acquisition and preprocessing unit is used to collect tower crane attitude parameters, load status parameters, environmental meteorological parameters, three-dimensional data of the surrounding scene, and tower crane operation command parameters through a multi-dimensional sensor array deployed at the base of the boom, both sides of the trolley, the top of the tower body, and the hook body. It forms a multi-source sensing data matrix and performs format regularization and noise filtering. The multi-view blind spot completion and data fusion unit is connected to the multi-source sensing data acquisition and preprocessing unit. It has a built-in multi-view blind spot completion and data fusion model. It completes the non-transparent blind spot below the hook through multi-view registration and occlusion inference, and generates a point cloud of the full-domain three-dimensional scene of tower crane operation. The load swing dynamic envelope prediction unit is connected to the multi-source sensing data acquisition and preprocessing unit. It has a built-in nonlinear pendulum dynamic prediction model, which considers the effects of wind speed disturbance and operation emergency stop, and calculates the real-time swing state of the load and the swing trajectory envelope within a preset time period in the future. The collision risk assessment and hazardous area generation unit is connected to the multi-view blind spot completion and data fusion unit and the load swing dynamic envelope prediction unit, respectively. It is used to fuse the full-domain three-dimensional scene point cloud and the load swing trajectory envelope to generate multi-level hazardous areas and calculate the collision risk probability. The graded early warning and linkage control unit is connected to the collision risk assessment and danger zone generation unit, and is used to trigger corresponding level of audible and visual early warning signals and tower crane linkage control commands according to the collision risk level. The model parameter closed-loop feedback adjustment unit is connected to the multi-view blind zone completion and data fusion unit, the load swing dynamic envelope prediction unit, and the hierarchical early warning and linkage control unit, respectively. It is used to dynamically optimize the model parameters based on actual collision events, false alarms and missed alarms, to form a closed-loop anti-collision recognition and control mechanism.

[0005] Furthermore, the expression for the multi-view blind spot completion and data fusion model is as follows: ;in, For the generated full-domain 3D scene point cloud of tower crane operation, Here are the fusion weighting coefficients for the data from each sensor, and ; The point cloud matrix of the scene captured by the panoramic camera at the base of the boom. This is the effective region mask matrix for the camera at the root of the upper arm; The scene point cloud matrix captured by the binocular camera on the left side of the vehicle. The occlusion mask matrix for the camera on the left side of the car; The scene point cloud matrix captured by the binocular camera on the right side of the vehicle. The occlusion mask matrix for the camera on the right side of the car; The point cloud matrix of the scene collected by the millimeter-wave radar at the top of the tower. This is the mask matrix for the effective detection area of ​​millimeter-wave radar. The point cloud matrix projected below the suspended object is generated with the assistance of the hook body IMU. Complete the mask matrix for the blind spot below the hook.

[0006] Furthermore, the expression for the nonlinear pendulum dynamics prediction model is as follows: ; ;in, Let be the swing angle of the load relative to the vertical direction. The angular velocity of the swing. This is the angular acceleration of the swing. It is the acceleration due to gravity. This refers to the effective length of the wire rope. For load quality; For real-time wind load, The angle between the wind direction and the tower crane boom; This refers to the air damping coefficient; Tangential acceleration generated during tower crane operation, including lifting, slewing, luffing, and emergency stop acceleration; The azimuth angle of the load swing; The lateral angular velocity; This is the lateral angular acceleration; The envelope of the load swing trajectory within the next T seconds; The real-time coordinates of the hook suspension point; To predict the time step, The preset prediction duration.

[0007] Furthermore, the expression for the collision risk assessment model is: ;in, The collision risk probability has a value range of [0,1]. The envelope overlap weighting coefficient is used. This is the distance weighting coefficient. The relative velocity weighting coefficient, and ; For load swing envelope bounding box of the target object The intersection and union ratio; The minimum distance between the target object and the load swing envelope; To establish a preset safe distance; The relative speed between the load and the target object.

[0008] Furthermore, the expression for the hierarchical early warning and linkage control logic is as follows: ;in, The set of control commands to be output; The preset level three risk threshold, and ; This is a normal operation command; no warning is required. This is a Level 1 warning command, triggering an audible and visual warning in the driver's cab. This is a Level 2 warning command, triggering a comprehensive on-site audible and visual warning and limiting the tower crane's operating speed. This is a Level 3 emergency control command, triggering emergency braking and cutting off the tower crane's power output.

[0009] Furthermore, the expression for adjusting the model parameters using closed-loop feedback is: ;in, The adjusted model parameter set; The current model parameter set; Adjust the learning rate based on feedback; This represents the gradient of the loss function with respect to the current parameters. For false positive and false negative datasets The calculated model loss value; It is a linear rectified function; A preset loss threshold is set, and parameter adjustments are triggered when the loss value exceeds the threshold.

[0010] Furthermore, the load swing dynamic envelope prediction unit includes the following sub-steps: The load mass, effective length of the wire rope, and coordinates of the hook suspension point are collected in real time. The initial swing angle and swing angular velocity of the load are obtained through the hook IMU and used as the initial input parameters of the dynamic model. Real-time wind speed and direction data are collected to calculate the disturbance term of wind load on load swing. At the same time, tower crane operation commands are collected and the lifting, slewing, luffing and emergency stop accelerations are analyzed as external excitation terms of the model. Based on the nonlinear pendulum dynamics equation, the fourth-order Runge-Kutta method is used to solve for the load swing angle and azimuth angle at each time step within the next T seconds, and the load spatial coordinates at the corresponding time are calculated. Perform convex hull calculation on the load spatial coordinates at all prediction times to generate a continuous load swing trajectory envelope. Then, expand the boundary of the envelope according to the wind speed fluctuation range to obtain the final predicted swing envelope.

[0011] Furthermore, the collision risk assessment and hazardous area generation unit includes the following sub-steps: Extract target objects of people, fixed facilities and mobile devices from the point cloud of the global 3D scene, generate a 3D bounding box for each target object and update its position and motion status in real time; Spatial matching is performed between the load swing trajectory envelope and the 3D bounding box of all target objects, and the intersection-union ratio and minimum distance between each target object and the swing envelope are calculated. Differentiated safety distances and weighting coefficients are set according to the type of the target object, and the collision risk probability of each target object is calculated by combining the load and the relative speed of the target object; Based on the probability of collision risk, multiple levels of danger zones are divided, including no-entry zones, high-risk zones, general danger zones, and safe zones, and the boundaries of each danger zone are updated in real time.

[0012] Furthermore, the tiered early warning and linkage control unit includes the following steps: The calculated collision risk probability is compared with the preset three-level risk threshold to determine the current risk level; The warning signal is triggered according to the risk level, and the on-site broadcast and remote monitoring platform pop-up alarm are also triggered. Execute the corresponding tower crane control commands according to the risk level; Record the execution time, execution status, and corresponding risk data of early warning and control commands, and store them synchronously in the system database.

[0013] Furthermore, it includes the following steps: S1. Collect multi-source sensing data of tower crane operation and surrounding environment. The multi-source sensing data includes tower crane attitude parameters, load status parameters, environmental meteorological parameters, three-dimensional data of surrounding scene and tower crane operation command parameters. The data is collected synchronously by a multi-dimensional sensor array deployed at different locations of the tower crane to form a multi-source sensing data matrix. S2, input the multi-source perception data matrix into the multi-view blind spot completion and data fusion model, and through multi-view registration, occlusion area reasoning and point cloud fusion algorithm, complete the non-transparent blind spot below the hook caused by the trolley, wire rope and suspended object, and generate the point cloud of the full-domain three-dimensional scene of tower crane operation. S3. Input the load state parameters, environmental meteorological parameters and tower crane operation command parameters into the nonlinear pendulum dynamic prediction model, consider the dynamic effects of wind speed disturbance and emergency stop of operation, and calculate the real-time swing state of the load and the swing trajectory envelope within the future preset time period. S4, integrate the full-domain 3D scene point cloud and the load swing trajectory envelope to generate dynamically updated multi-level danger zones, and calculate the collision risk probability of personnel, facilities and load swing envelopes in different zones through a collision risk assessment model; S5, based on the collision risk probability, classify the risk level, trigger the corresponding level of audible and visual warning signal and tower crane linkage control command to realize active intervention to prevent collision; S6 collects actual collision events, false alarms and missed alarms data, and dynamically adjusts the parameters of the multi-view blind spot completion and data fusion model and the nonlinear pendulum dynamics prediction model to form a closed-loop collision avoidance identification and control mechanism.

[0014] Compared to existing solutions, the beneficial effects achieved by this invention are: This invention addresses the blind spot created by traditional visual sensors, which is obstructed by trolleys, wire ropes, and suspended loads, resulting in the area below the hook. Through a complementary multi-view sensor layout and geometric inference completion technology, it solves the visual blind spot problem, the most common cause of collisions in tower crane operations. By integrating multiple types of equipment, including visible light cameras, millimeter-wave radar, IMUs, and tension sensors, it retains the high-resolution detail of visual sensors while possessing the all-weather detection capability of millimeter-wave radar, unaffected by light, rain, snow, or dust. The detection range covers the entire 200m radius of the tower crane's operating area, maintaining stable sensing performance even in harsh environments. Employing a two-dimensional nonlinear pendulum dynamics model considering gravity, wind load, air damping, and tower crane operating acceleration, it can predict the load swing trajectory within the next 3-5 seconds. This system achieves a shift from post-collision alarm to pre-collision warning; by introducing real-time wind speed disturbance and emergency stop acceleration correction terms, it can accurately simulate the large-scale swaying characteristics of the load during strong winds and emergency braking. Combined with a dynamic boundary expansion mechanism, it avoids the defect of traditional position interpolation methods that severely underestimate the range of dangerous areas; by constructing a multi-factor weighted risk assessment model based on three-dimensional intersection-union ratio, safety distance ratio, and relative speed, it transforms collision risk into a continuous probability value between 0 and 1, replacing the binary judgment method of collision presence / absence in traditional systems; by adopting a linear speed limiting mechanism, when the risk probability exceeds the secondary threshold, the tower crane's operating speed gradually decreases as the risk increases, rather than suddenly cutting off power, thus ensuring safety while minimizing interference with normal operations. Attached Figure Description

[0015] The invention will now be further described with reference to the accompanying drawings.

[0016] Figure 1 This is a flowchart illustrating the operation of a multi-sensor fusion anti-collision recognition control system for intelligent tower cranes according to the present invention.

[0017] Figure 2 This is a block diagram of a multi-sensor fusion anti-collision recognition control system for intelligent tower cranes according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figures 1 to 2As shown, the present invention is an intelligent tower crane multi-sensor fusion anti-collision recognition control system, including a multi-source sensing data acquisition and preprocessing unit, a multi-view blind zone completion and data fusion unit, a load swing dynamic envelope prediction unit, a collision risk assessment and dangerous area generation unit, a graded early warning and linkage control unit, and a model parameter closed-loop feedback adjustment unit. The multi-source sensing data acquisition and preprocessing unit is used to collect tower crane attitude parameters, load status parameters, environmental meteorological parameters, 3D data of the surrounding scene, and tower crane operation command parameters through a multi-dimensional sensor array deployed at the base of the boom, both sides of the trolley, the top of the tower body, and the hook body. This data is then used to form a multi-source sensing data matrix and to perform format normalization and noise filtering. Specifically: The multi-source sensing data acquisition and preprocessing unit is the core of the entire collision avoidance recognition and control system. It is responsible for synchronously acquiring comprehensive data on the tower crane's operating status, load dynamics, environmental weather conditions, and surrounding operating scenarios through a multi-dimensional, multi-view distributed sensor array. A standardized preprocessing process eliminates data noise, format differences, and time deviations, providing high-precision, highly synchronized foundational data support for subsequent multi-view blind spot completion, load swing trajectory prediction, and collision risk assessment. During implementation, this unit achieves interconnection between all sensors and the edge computing unit via an industrial Ethernet bus, with data transmission latency less than 10ms and overall data acquisition and preprocessing time less than 50ms, meeting the real-time requirements of the tower crane collision avoidance system.

[0020] This unit adopts a layered deployment strategy with complementary sensors, deploying nine types of core data acquisition devices at four key locations: the base of the tower crane's boom, both sides of the trolley, the top of the tower body, and the hook itself. This forms a multi-source sensing network covering the entire operation area of ​​the tower crane. The specific deployment plan is as follows: Deployment area at the base of the boom: Install one 4K panoramic industrial camera, one industrial-grade six-axis attitude sensor, and one absolute rotary encoder; The panoramic camera is horizontally mounted with a field of view of 360°×180°, used to collect the full visible light scene in front of and to the side of the boom; The six-axis attitude sensor is fixed to the steel structure at the base of the boom, used to collect the boom pitch angle; The absolute encoder is coaxially connected to the boom rotation mechanism, used to collect the boom rotation angle.

[0021] Dual-sided deployment area for the trolley: Two industrial binocular stereo cameras are symmetrically installed on the left and right sides of the tower crane trolley. The camera baseline distance is 30cm, and they are installed vertically downwards with a field of view of 120°×90°. The overlapping area of ​​the two cameras' fields of view covers a radius of 3m directly below the hook, thus filling the blind spots and eliminating the obstruction of the field of view by the trolley and wire rope.

[0022] Deployment area at the top of the tower: Install one 77GHz millimeter-wave radar and one integrated weather station; The millimeter-wave radar is horizontally installed, with a detection range of 0.1-200m, an azimuth angle of ±60°, and an elevation angle of ±15°, and is unaffected by environmental factors such as sunlight, rain, snow, and dust, and is used to collect position information of long-distance moving targets and fixed facilities; The integrated weather station is installed in an unobstructed position at the top of the tower and is used to collect real-time wind speed, wind direction, and ambient temperature.

[0023] Hook body deployment area: A nine-axis IMU, an S-type tension sensor, and a wire rope length encoder are installed above the hook pulley block; the nine-axis IMU is used to collect the three-dimensional swing posture of the hook and load; the tension sensor is connected in series between the wire rope and the hook to measure the load mass; the wire rope length encoder is coaxially connected to the winch to calculate the effective length of the wire rope.

[0024] This unit collects 20 core parameters across 5 categories. All parameters are collected synchronously according to preset sampling frequency and accuracy requirements. The specific collection specifications are as follows: Tower crane attitude parameter acquisition: a total of 4 parameters, including boom slewing angle. Boom amplitude Hook suspension point height upper arm pitch angle The boom slewing angle is acquired by an absolute encoder with an accuracy of 0.01° and a sampling frequency of 50Hz; the boom luffing amplitude is acquired by a trolley position encoder with an accuracy of 0.01m and a sampling frequency of 50Hz; the hook suspension point height is calculated by combining the wire rope length encoder with the boom luffing amplitude, with an accuracy of 0.01m and a sampling frequency of 50Hz; and the boom pitch angle is acquired by a six-axis attitude sensor with an accuracy of 0.05° and a sampling frequency of 100Hz.

[0025] Load status parameter acquisition: a total of 5 parameters, including load quality Load swing angle angular velocity of load oscillation azimuth angle of load swing Effective length of wire rope The load mass is collected by a tension sensor with an accuracy of 0.01t and a sampling frequency of 100Hz; the load swing angle, swing angular velocity, and swing azimuth angle are collected by a nine-axis IMU with accuracies of 0.1°, 0.1° / s, and 0.1°, respectively, and a sampling frequency of 200Hz; the effective length of the wire rope is collected by a wire rope length encoder with an accuracy of 0.01m and a sampling frequency of 50Hz.

[0026] Environmental meteorological parameter collection: a total of 3 parameters, including real-time wind speed. ,wind direction Ambient temperature Data was collected by an integrated weather station, with wind speed accuracy of 0.1 m / s, wind direction accuracy of 1°, temperature accuracy of 0.1℃, and a sampling frequency of 10 Hz.

[0027] 3D data acquisition of the surrounding scene: Data is acquired jointly by a panoramic camera at the base of the boom, dual-lens cameras on both sides of the trolley, and a millimeter-wave radar at the top of the tower. The panoramic camera has a resolution of 3840×2160 and a frame rate of 30fps; the dual-lens camera has a resolution of 1280×720 and a frame rate of 30fps, with a single-frame point cloud density of 10,000 points; the millimeter-wave radar outputs a maximum of 256 target points per frame at a frame rate of 20fps.

[0028] Tower crane operation command parameter acquisition: Hoisting commands are acquired in real time via communication with the tower crane PLC control system using the Modbus-TCP protocol. , turn command Amplitude change command Emergency stop command There are a total of 4 instruction parameters, a sampling frequency of 100Hz, and an instruction response delay of less than 10ms.

[0029] The collected raw data is transmitted to the edge computing unit inside the tower crane control cabinet, where it undergoes time synchronization, noise filtering, and format normalization processes. The specific process is as follows: Hardware-level time synchronization is achieved using the IEEE 1588 precision time protocol. The system clock of the edge computing unit is used as the global reference clock, and the local acquisition timestamps of all sensors are uniformly calibrated. The time synchronization error is less than 1ms. The time synchronization correction formula is as follows: ;in, This is the synchronized standard timestamp, in seconds. This is the local data acquisition timestamp from the sensor, in seconds. This is the deviation between the sensor's local clock and the system's reference clock, measured in seconds (s), and is calibrated every 1 second via the PTP protocol.

[0030] Differentiated filtering algorithms are employed to remove noise interference based on the noise characteristics of different types of sensors: For continuous time-domain data such as tower crane attitude parameters and load state parameters, Extended Kalman Filter (EKF) is used for dynamic noise filtering. The filtering algorithm consists of a prediction step and an update step, expressed as follows: Prediction step: ; ;in, The predicted state value at time k; This is the optimal state estimate at time k-1; This is the state transition matrix; This is the transpose of the state transition matrix; To control the input matrix; This is the control input vector at time k; To predict the covariance matrix; Let be the prediction covariance matrix at time k-1; For the process noise covariance matrix; Update step: ; ; ;in, The Kalman gain matrix; The observation matrix; This is the transpose of the observation matrix; To observe the noise covariance matrix; The observation value at time k; This is the optimal state estimate at time k; This is the updated covariance matrix; It is an identity matrix.

[0031] For 3D point cloud data acquired by a stereo camera, a statistical filtering algorithm is used to remove outliers. The algorithm expression is as follows: ;in, Let be the average distance from the i-th point in the point cloud to its N nearest neighbors; Let i be the three-dimensional coordinates of the i-th point; Let be the three-dimensional coordinates of the j-th nearest neighbor of the i-th point; This represents the number of nearest neighbors, with a value of 50. If... If the point is an outlier, it is determined to be an outlier and removed. The mean of the average distances between all points. is the standard deviation of the average distance of all points.

[0032] For millimeter-wave radar data, a cell-average constant false alarm rate (CA-CFAR) detection algorithm is used to remove background clutter, and the false alarm rate is controlled within a certain range. The following are existing conventional technical solutions; the specific implementation steps will not be elaborated here.

[0033] All preprocessed data were converted to IEEE 754 single-precision floating-point format, retaining 4 decimal places. To eliminate the influence of dimensional differences between parameters on subsequent model calculations, the min-max normalization method was used to map all numerical parameters to the [0,1] interval. The normalization formula is as follows: ;in, These are the standardized parameter values; These are the original parameter values; This is the historical minimum value of this parameter; This is the historical maximum value of the parameter. For Boolean parameters such as emergency stop commands, they are directly converted to numerical forms of 0 (no emergency stop) and 1 (emergency stop triggered).

[0034] All parameters, after time synchronization, noise filtering, and format normalization, are arranged in chronological order according to standard timestamps, constructing a dimension of... Multi-source sensing data matrix ,in, The row dimension represents the time step, with a sampling interval of 10ms, meaning 100 rows of data are generated per second; the column dimension is 20, corresponding to all collected parameters, in the following order: boom slewing angle, boom luffing amplitude, hook suspension point height, boom pitch angle, load mass, load swing angle, load swing angular velocity, load swing azimuth angle, effective wire rope length, real-time wind speed, wind direction, ambient temperature, panoramic camera frame number, left binocular camera frame number, right binocular camera frame number, millimeter-wave radar frame number, hoisting command, slewing command, luffing command, and emergency stop command.

[0035] Each element in the matrix Indicates the first The time step The standardized values ​​of each parameter are cached in real time in the memory of the edge computing unit for 30 seconds. At the same time, the data is written to the local solid-state drive for long-term storage in the format of one file every 10 minutes, with a storage period of not less than one year. The generated multi-source sensing data matrix will be transmitted in real time to the multi-view blind spot completion and data fusion unit and the load swing dynamic envelope prediction unit to provide basic data input for subsequent collision avoidance recognition and control.

[0036] A multi-view blind spot completion and data fusion unit, connected to the multi-source sensing data acquisition and preprocessing unit, has a built-in multi-view blind spot completion and data fusion model. It completes the non-transparent blind spot below the hook through multi-view registration and occlusion inference, generating a full-domain 3D scene point cloud of the tower crane operation. Specifically: The multi-view blind spot completion and data fusion unit is connected to the multi-source sensing data acquisition and preprocessing unit. It undertakes the task of cross-view spatial integration of multi-source sensor data and completion of the blind spot below the hook. The core is to solve the industry pain point that the area below the hook cannot be effectively sensed during tower crane operation because the trolley shell, wire rope, and the load itself block the field of view of the sensors at the end of the boom. During implementation, this unit receives a standardized multi-source sensing data matrix with a dimension of T×20 output from the multi-source sensing data acquisition and preprocessing unit. It extracts the point cloud of the panoramic camera at the boom root (about 10,000 points), the point cloud of the binocular camera on the left side of the trolley (about 8,000 points), the point cloud of the binocular camera on the right side of the trolley (about 8,000 points), the point cloud of the millimeter-wave radar at the top of the tower (about 256 points), and the IMU attitude data of the hook body at the same time stamp. Through a four-level progressive processing logic, it generates a blind spot-free full-domain 3D scene point cloud of tower crane operation. The unit single-frame data processing latency is less than 40ms, the global point cloud update frequency is not less than 20Hz, the blind zone completion accuracy below the hook is better than ±0.2m, and the effective point cloud ratio is not less than 95%, meeting the real-time and reliability requirements of the tower crane anti-collision system.

[0037] The multi-view blind spot completion and data fusion unit has a built-in multi-view blind spot completion and data fusion model, the model expression of which is: ;in, The output is a full-domain 3D point cloud matrix of the tower crane operation, with dimensions of [missing information]. , 'This represents the total number of points in the point cloud, with each row corresponding to the 3D coordinates of one point. and confidence level The coordinate unit is meters, and the confidence level range is [value missing]. A higher confidence level indicates a stronger reliability of the spatial location of that point. These are the fusion weight coefficients for the panoramic camera at the base of the boom, the binocular camera on the left side of the trolley, the binocular camera on the right side of the trolley, the millimeter-wave radar at the top of the tower, and the auxiliary point cloud from the hook's IMU. The values ​​range from [value range missing]. to And satisfy The initial value is determined through offline calibration experiments and can be dynamically adjusted according to ambient light and weather conditions; The point cloud matrix of the scene generated by stereo matching from the panoramic camera at the base of the boom has dimensions of [dimensions missing]. , Approximately 10,000, including scene point cloud data within a range of 0-150m in front of and to the side of the upper arm; The effective region mask matrix for the camera at the base of the upper arm, with dimensions and Consistent, the matrix elements take values ​​of 0 or 1: a value of 1 indicates that the point is within the camera's unobstructed effective field of view, and a value of 0 indicates that the point is outside the camera's field of view or is obstructed by the tower crane's own structure, such as the boom steel structure or counterweight; The scene point cloud matrix generated by the binocular camera on the left side of the car has dimensions of , Approximately 8000, covering a close-range scene within a range of 0~20m to the lower left of the vehicle, serving as the primary source of perception for the area below the hook; The occlusion mask matrix for the camera on the left side of the car, with dimensions and Consistent, a value of 1 indicates that the point is not blocked by the trolley shell, wire rope or hanging object, and a value of 0 indicates that the point is in the blocked area; The scene point cloud matrix generated by the binocular camera on the right side of the car has a dimension of , Approximately 8000, covering the close-range scene within a range of 0~20m on the lower right side of the vehicle, forming a perception layout that complements the blind spots with the left-side camera; The occlusion mask matrix for the camera on the right side of the car, with dimensions and Consistency, definition and Completely identical; The scene point cloud matrix generated for the millimeter-wave radar at the top of the tower has dimensions of , The maximum is 256, containing target point cloud data in the range of 0.1~200m. It is not affected by environmental factors such as light, rain, snow, and dust, and is mainly used for long-distance target detection and blind spot filling in harsh environments. The effective detection area mask matrix for millimeter-wave radar, with dimensions and Consistent, a value of 1 indicates that the point is within the effective detection range of the radar and the signal-to-noise ratio is greater than 15dB, and a value of 0 indicates that the point is background clutter or an invalid point; The point cloud matrix projected below the suspended object, generated with the assistance of the hook's IMU, has dimensions of [missing information]. , Approximately 2000, specifically designed to fill the opaque blind spot directly below the hook; To complete the mask matrix for the blind spot below the hook, the dimension is... A value of 1 indicates that the point is within the blind zone below the hook and requires inference and completion; a value of 0 indicates that the point is effectively covered by other sensors. The above expression adopts a three-level computational architecture of effective area mask filtering, multi-source data weighted fusion, and dedicated blind zone inference and completion. First, the element-wise product operation of the mask matrix and the corresponding point cloud is used to remove the occluded areas and invalid clutter data in the field of view of each sensor. Then, the effective point cloud data of different locations and types of sensors are integrated through weighted summation to achieve complementary fusion of cross-view spatial information. Finally, the product operation of the hook-assisted point cloud and the blind zone mask is used to specifically perform inference and completion for the non-transparent blind zone below the hook, ultimately generating a blind-zone-free 3D scene point cloud covering the entire tower crane operation area.

[0038] When optimizing the multi-view blind spot completion and data fusion model, the model is set up with four levels of progressive processing logic. The first layer is the multi-view spatiotemporal registration layer, which realizes the spatial and temporal alignment of data from different sensors. The second layer is the dynamic occlusion detection and mask generation layer, which accurately identifies the occlusion areas in the field of view of each sensor. The third layer is the multi-source point cloud weighted fusion layer, which integrates effective point cloud data to generate a preliminary global point cloud. The fourth layer is the blind spot reasoning and completion layer under the hook, which completes the blind spot through geometric reasoning and prior information. The multi-view spatiotemporal registration layer employs a multi-sensor coordinate transformation formula, expressed as follows: ;in, These are the point cloud coordinates of the tower crane in the transformed global coordinate system, in meters. This is the real-time rotation matrix for the boom, determined by the boom rotation angle. With pitch angle The calculated dimension is 3×3; The translation vector of the boom root in the global coordinate system has a dimension of 3×1 and is fixed as the coordinates of the center point of the tower crane base. The offline calibration extrinsic parameter matrices for the sensor are a 3×3 rotation matrix and a 3×1 translation vector, which are predetermined by the hand-eye calibration method. These are the original point cloud coordinates in the sensor's local coordinate system, in meters. The above expression converts point cloud data from different sensor local coordinate systems into point cloud data in a unified tower crane global coordinate system, which is the spatial basis for multi-source data fusion. By correcting the sensor's extrinsic parameter matrix through the real-time attitude parameters of the boom, it ensures that the point cloud data from all sensors remain spatially aligned during the tower crane's rotation and luffing processes.

[0039] During the implementation of the multi-view spatiotemporal registration layer, the hand-eye calibration method and a 1.2m×1.2m checkerboard calibration board were used to complete the offline extrinsic parameter calibration of all sensors. The calibration board was placed at 10 different locations within the tower crane's operating range, and calibration images of each sensor were collected. The extrinsic parameter matrix of each sensor relative to the coordinate system at the base of the boom was calculated using Zhang's calibration method. and During online operation, based on the synchronous timestamp output by the multi-source sensing data acquisition and preprocessing unit, the point cloud data of all sensors at the same moment are time-aligned; simultaneously, the real-time rotation angle and pitch angle of the boom are read, and the real-time rotation matrix of the boom is calculated. By using coordinate transformation formulas, all local coordinate system point clouds are converted into global coordinate system point clouds, with a spatial alignment error of less than ±0.1m; The dynamic occlusion detection and mask generation layer uses a dynamic mask fusion formula, expressed as follows: ;in, This is the final effective region mask matrix of the sensor, with dimensions consistent with the point cloud matrix of the corresponding sensor, and elements taking values ​​of 0 or 1; The static occlusion mask matrix for the tower crane's own structure is pre-calculated and generated based on the tower crane's 3D CAD model and the real-time positions of the boom, trolley, and hook. The dynamic occlusion mask matrix for the suspended object is generated based on the inference results of the dual-side binocular camera images of the trolley using the YOLOv8 target detection algorithm. This can be achieved using existing conventional technical solutions, and the specific implementation steps will not be elaborated here. For logical OR operator; The above expression uses a logical OR operation to fuse the static occlusion mask of the tower crane's own structure with the dynamic occlusion mask of the suspended object, generating the final effective area mask matrix for each sensor, ensuring that invalid data in all occluded areas are accurately removed.

[0040] When implementing the dynamic occlusion detection and mask generation layer, the 3D CAD model of the tower crane is imported. Combined with real-time data on the boom slewing angle, trolley luffing amplitude, and hook height, the projection area of ​​the tower crane's own structure (boom steel structure, trolley shell, and wire rope) on the imaging planes of each sensor is pre-calculated to generate a static structural mask. The YOLOv8 object detection algorithm is used to infer real-time images from the dual-camera system on both sides of the vehicle, detecting the bounding box and instance segmentation mask of the suspended object, with a detection frame rate of no less than 30fps. The segmentation mask in the image coordinate system is converted into a 3D space mask through the camera intrinsic parameter matrix to generate a dynamic occlusion mask for the suspended object. Finally, the static and dynamic masks are fused using a logical OR operation to obtain the final effective region mask matrix for each sensor. The confidence level of the point cloud corresponding to the occluded area with a value of 0 in the mask is set to 0, and it is removed in the subsequent fusion process.

[0041] When implementing the multi-source point cloud weighted fusion layer, the initial fusion weight coefficients were calibrated through offline experiments. Under standard lighting conditions (light intensity 500-10000 lux, no rain or snow), the weights were set as follows: , , , , Under severe conditions (light intensity below 500 lux or rain / snow), the weights are dynamically adjusted as follows: , , , , To address the performance degradation of visual sensors, the weighting of millimeter-wave radar was significantly increased. All registered and masked point cloud data were voxelized and downsampled, with voxel sizes set to 0.1m × 0.1m × 0.1m. For multiple points within each voxel, the point with the highest confidence was retained as the representative point of that voxel. The total number of point clouds was controlled between 30,000 and 50,000 to ensure real-time performance. Finally, based on the multi-view blind zone completion and data fusion model formula, all voxelized point clouds were weighted and summed. For voxels covered by multiple sensors at the same spatial location, the final confidence level was the weighted average of the confidence levels of the corresponding points for each sensor.

[0042] Specifically, the blind zone inference and completion layer below the hook adopts the blind zone ground plane fitting formula, the expression of which is: ;in, Let be the normal vector of the plane. This is the offset of the plane; These are the coordinates of a point in three-dimensional space, in meters. The above expression is used to fit the ground plane around the blind zone below the hook, and the ground point cloud within the blind zone is generated based on the fitted plane equation, so as to realize the geometric reasoning completion of the non-transparent blind zone.

[0043] When implementing the blind zone inference and completion layer under the hook, the spatial range of the blind zone under the hook is first determined based on the real-time position of the hook, the size of the suspended object, and the length of the wire rope. The blind zone is defined as a conical area extending 5m downwards from the center of the bottom of the suspended object, with a horizontal radius of 1.5 times the maximum horizontal dimension of the suspended object. Effective unobstructed ground point cloud data within a 3m radius of the blind zone is extracted, and the ground plane equation is fitted using the least squares method. The fitting error is less than ±0.1m; using the attitude data of the suspended object acquired by the nine-axis IMU of the hook body, combined with the three-dimensional model of the suspended object, a projected outline of the bottom of the suspended object is generated; based on the fitted ground plane equation, a uniformly distributed ground point cloud is generated in the blind area below the projected outline, with a point cloud density of 100 points / m², forming an auxiliary point cloud for the hook. Finally, the blind spot mask matrix is ​​used. The auxiliary point cloud is merged into the global point cloud to complete the blind spot below the hook that cannot be seen.

[0044] When the multi-view blind spot completion and data fusion unit performs output processing, it generates a global 3D scene point cloud. After statistical filtering and confidence thresholding, the data is converted to standard PCL format, containing the 3D coordinates, confidence level, and sensor source information for each point. A confidence threshold of 0.3 is set to eliminate all point cloud data with confidence levels below the threshold, ensuring the reliability of the final point cloud. This point cloud is transmitted in real-time to the collision risk assessment and hazardous area generation unit via an industrial Ethernet bus, and simultaneously cached on the local hard drive of the edge computing unit for subsequent model training and accident backtracking. The generated global point cloud completely covers all areas within a 200m radius of the tower crane's operating radius, completely eliminating the blind spot below the hook, providing a complete and reliable spatial environmental data foundation for subsequent load swing trajectory matching and collision risk assessment.

[0045] The load swing dynamic envelope prediction unit, connected to the multi-source sensing data acquisition and preprocessing unit, incorporates a nonlinear pendulum dynamics prediction model. Considering wind speed disturbances and the effects of sudden operational stops, it calculates the real-time swing state of the load and the swing trajectory envelope within a preset future time period. Specifically: The load swing dynamic envelope prediction unit is connected to the multi-source sensing data acquisition and preprocessing unit, undertaking the tasks of predicting the dynamic swing trajectory of the load and generating the envelope of the dangerous area. Its core function is to address the industry pain point that traditional position interpolation methods severely underestimate the range of dangerous areas caused by the large-amplitude swing of the load, similar to a pendulum, resulting from lifting, slewing, luffing, and emergency stop operations during tower crane operations, and the swing amplitude being affected by wind speed disturbances. During implementation, this unit receives a standardized multi-source sensing data matrix output from the multi-source sensing data acquisition and preprocessing unit, extracts the load mass, effective wire rope length, hook suspension point coordinates, initial swing posture of the load, real-time wind speed and direction, and tower crane operation command parameters at the same timestamp, and generates the load swing trajectory envelope within a preset time period through a three-level progressive processing logic. The unit's single-frame data processing latency is less than 20ms, the envelope update frequency is not less than 50Hz, the swing trajectory error within a 3-second prediction time period is less than ±0.3m, and the envelope boundary expansion coefficient is not less than 1.2, meeting the real-time and accuracy requirements of the tower crane anti-collision system for dangerous area prediction.

[0046] The load swing dynamic envelope prediction unit incorporates a nonlinear simple pendulum dynamics prediction model, the model expression of which is: ; ; in, The longitudinal sway angle of the load relative to the vertical direction, in rad; The longitudinal angular velocity is expressed in rad / s. This is the longitudinal angular acceleration, expressed in rad / s². The lateral swing angle of the load relative to the boom direction is expressed in rad. The lateral angular velocity is expressed in rad / s. ω is the lateral angular acceleration, in rad / s²; g is the gravitational acceleration, with a value of 9.8 m / s²; L is the effective length of the wire rope, in m, which is collected in real time by the multi-source sensing data acquisition and preprocessing unit; m is the load mass, in kg, which is collected in real time by the tension sensor. This is the real-time wind load, in N, calculated from wind speed data; The angle between the wind direction and the tower crane boom is measured in rad and is collected by an integrated weather station. The air damping coefficient ranges from 0.05 to 0.2 and is positively correlated with the load's frontal area. Tangential acceleration generated during tower crane operation, measured in m / s², including hoisting, slewing, luffing, and emergency stop acceleration; Let T be the envelope of the load swing trajectory within the next T seconds, and let T be a three-dimensional convex hull. The real-time coordinates of the hook suspension point are in meters and are calculated from the boom slewing angle, luffing amplitude, and hook height. The time step is for prediction, and the unit is seconds (s). The preset prediction duration is 3 to 5 seconds.

[0047] The above expression uses a two-dimensional nonlinear pendulum dynamic equation to describe the coupled swing characteristics of the load in the vertical and horizontal planes. It comprehensively considers the influence of gravity, wind load, air damping and tower crane operation acceleration on the load swing. The load swing angle and azimuth angle for each time step in the future T seconds are solved by the fourth-order Runge-Kutta method. Then, the load spatial coordinates are calculated and a continuous swing trajectory envelope is generated by convex hull operation to accurately characterize the dynamic danger zone of the load.

[0048] When optimizing the nonlinear pendulum dynamic prediction model, a three-level progressive processing logic is set up inside the model. The first layer is the initial parameter acquisition and preprocessing layer, which extracts the real-time state parameters of the load and environment. The second layer is the dynamic equation solving layer, which considers multi-source external excitation to calculate the dynamic swing state of the load. The third layer is the swing trajectory envelope generation and expansion layer, which generates a conservative and reliable danger zone envelope. The real-time wind load calculation formula is as follows: ;in, The density of air is taken as 1.225 kg / m³ (under standard atmospheric pressure). The drag coefficient is 1.2 (cubic load) or 0.8 (cylindrical load), which is preset according to the load type; S is the windward area of ​​the load, in m², which is calculated from the three-dimensional model of the suspended object and its real-time swing posture. Real-time wind speed, in m / s, collected by an integrated weather station.

[0049] The above calculation formula is based on the principle of aerodynamics to calculate the real-time wind load acting on the load, taking into account the effects of air density, drag coefficient, load frontal area and wind speed, so as to provide accurate external disturbance input for the dynamic model.

[0050] During the initial parameter acquisition and preprocessing layer implementation, the core initial parameters for the current moment are first extracted from the standardized data matrix output by the multi-source sensing data acquisition and preprocessing unit: load mass m (accuracy 0.01t), effective wire rope length L (accuracy 0.01m), and coordinates of the hook suspension point. (Accuracy 0.01m), Initial longitudinal swing angle of the load (Accuracy 0.1°), Initial lateral swing angle (Accuracy 0.1°), Initial longitudinal oscillation angular velocity (Accuracy 0.1° / s), Initial lateral swing angular velocity (Accuracy 0.1° / s). Real-time wind speed is also extracted. (Accuracy 0.1m / s), wind direction (Accuracy 1°) and tower crane operation command parameters, including lifting acceleration. gyroscopic acceleration Variable amplitude acceleration and emergency stop command status. The air damping coefficient k and drag coefficient are preset according to the load type. The current windward area S is calculated by measuring the real-time load swing attitude, and then the real-time wind load is calculated using the wind load formula. .

[0051] During the implementation of the dynamic equation solution layer, the fourth-order Runge-Kutta method was used to numerically solve the two-dimensional nonlinear simple pendulum dynamic equation, and the prediction time step was set. The prediction time is T=3s, and the load oscillation state is solved for a total of 60 time steps. For each time step, the tangential acceleration is first calculated based on the tower crane operation command at the current moment. During normal operation, the tangential acceleration is the vector sum of the rotational, luffing, and lifting accelerations. When an emergency stop command is detected, the tangential acceleration suddenly increases to -15 m / s² (emergency stop deceleration) for 0.5 s. Substituting the current swing angle, angular velocity, wind load, and tangential acceleration into the dynamic equations, the swing angle and angular velocity at the next moment are calculated. During the solution process, the effective length L of the wire rope and the coordinates of the hook suspension point are updated in real time. This ensures that the model parameters are synchronized with the actual operating status of the tower crane.

[0052] Specifically, the formula for expanding the envelope boundary of the oscillating trajectory is: ;in, The extended load swing trajectory envelope; ⊕ is the three-dimensional morphological dilation operator; For radius spherical structural elements; This is the boundary expansion coefficient.

[0053] The above formula uses morphological dilation to expand the boundary of the original swing trajectory envelope, generating a conservative and reliable danger zone envelope. This compensates for the uncertainty caused by model prediction errors and wind speed fluctuations, ensuring that the danger zone covers all possible load swing ranges.

[0054] When generating the oscillation trajectory envelope and implementing the extended layer, the load space coordinates at 60 time steps are first obtained from solving the dynamic equations. The original swing trajectory envelope is generated using a three-dimensional convex hull algorithm. Then, the boundary spread coefficient is dynamically adjusted according to the current wind speed. When the wind speed is less than 5 m / s, When the wind speed is 5~10m / s, When the wind speed is greater than 10 m / s, The original envelope is expanded using the morphological dilation formula to obtain the final expanded oscillating trajectory envelope. Simultaneously, the load oscillation state data for each time step, including oscillation angle, angular velocity, spatial coordinates, and acceleration, are recorded for subsequent model calibration and error analysis.

[0055] Extended load swing trajectory envelope generated by the load swing dynamic envelope prediction unit The data is represented using a triangular mesh format, containing vertex coordinates and triangular face index information of the envelope surface, with a mesh resolution of 0.1m. This envelope data is transmitted in real time via an industrial Ethernet bus to the collision risk assessment and hazardous area generation unit, where it is spatially matched with the global 3D scene point cloud output by the multi-view blind spot completion and data fusion unit to calculate the collision risk probability. Simultaneously, the envelope data is cached on the local hard drive of the edge computing unit for 30 seconds for subsequent accident backtracking and model parameter optimization. The generated load swing trajectory envelope accurately depicts all possible movement ranges of the load within the next 3 seconds, effectively solving the problem of underestimating hazardous areas using traditional position interpolation methods, and providing a reliable dynamic hazardous area basis for collision risk warning and active control.

[0056] The collision risk assessment and hazardous area generation unit is connected to the multi-view blind spot completion and data fusion unit and the load swing dynamic envelope prediction unit, respectively. It is used to fuse the full-domain 3D scene point cloud and the load swing trajectory envelope to generate multi-level hazardous areas and calculate the collision risk probability. Specifically: The collision risk assessment and hazardous area generation unit is connected to the multi-view blind spot completion and data fusion unit and the load swing dynamic envelope prediction unit, respectively. It undertakes the tasks of fusion analysis of spatial scenes and dynamic loads, quantification calculation of collision risks, and generation of multi-level hazardous areas. Its core function is to address the industry pain points of traditional tower crane collision avoidance systems, which rely solely on static position detection, fail to differentiate target types and safety distances, and ignore the impact of relative motion speed on risk, leading to delayed warnings and high false alarm / missed alarm rates. During implementation, this unit simultaneously receives the tower crane operation's full-domain 3D scene point cloud (30,000-50,000 points) output by the multi-view blind spot completion and data fusion unit and the extended load swing trajectory envelope output by the load swing dynamic envelope prediction unit. Through a four-level progressive processing logic, it generates standardized collision risk probability data and multi-level dynamic hazardous areas. The unit's single-frame data processing latency is less than 30ms, the hazardous area update frequency is no less than 20Hz, and the target recognition accuracy is greater than 98%, meeting the real-time and reliability requirements of tower crane collision avoidance systems for risk warnings.

[0057] Specifically, the collision risk assessment and hazardous area generation unit incorporates a multi-factor weighted collision risk assessment model, the model expression of which is: ;in, The collision risk probability has a value range of [0,1]. These are the three-dimensional intersection-union ratio weighting coefficient, the safety distance weighting coefficient, and the relative velocity weighting coefficient, respectively, all ranging from 0.1 to 0.9, and satisfying the following conditions: Initial values ​​were calibrated through offline experiments; This is a 3D intersection-union ratio operator that calculates the ratio of the intersection volume to the union volume of the extended load swing trajectory envelope and the target 3D bounding box. The expanded load swing trajectory envelope is output by the load swing dynamic envelope prediction unit; Align the target object with its 3D axis-bounded bounding box (AABB). The minimum Euclidean distance, in meters, is the 3D bounding box of the target object and the envelope of the extended load swing trajectory. Preset safety distances for different types of targets, in meters, where type is the target type identifier; The magnitude of the relative velocity vector between the load center and the target center, in m / s; The preset maximum relative velocity threshold is set to 5 m / s.

[0058] The above expression uses three core dimensions—three-dimensional intersection-union ratio, normalized safe distance ratio, and normalized relative velocity—for weighted fusion to comprehensively quantify the collision risk between the load and the target object. The three-dimensional intersection-union ratio reflects the spatial overlap between the load's swing envelope and the target's bounding box; the normalized safe distance ratio reflects the proximity of the target to the danger zone; and the normalized relative velocity reflects the urgency of the collision. The weighted sum of these three values ​​yields a collision risk probability ranging from [0,1], with a higher value indicating a higher collision risk.

[0059] When optimizing the multi-factor weighted collision risk assessment model, the model is set up with four levels of progressive processing logic. The first layer is the target extraction and attribute quantification layer, which extracts and classifies target objects from the point cloud of the full-domain 3D scene. The second layer is the spatial matching and parameter calculation layer, which calculates the spatial correlation parameters between each target and the load swing envelope. The third layer is the collision risk quantification calculation layer, which calculates the collision risk probability of each target based on the multi-factor weighted model. The fourth layer is the multi-level danger zone generation layer, which generates dynamically updated danger zone boundaries based on the risk probability distribution.

[0060] Specifically, the formula for setting the target type differentiated safety distance and weighting coefficient is as follows: Among them, "person" is the target type identifier for personnel, including operators, inspectors, etc.; "facility" is the target type identifier for fixed facilities, including tower crane foundations, distribution boxes, building material storage areas, etc.; and "equipment" is the target type identifier for mobile equipment, including excavators, loaders, concrete pump trucks, etc. The safe distance corresponding to the target type.

[0061] During the target extraction and attribute quantization layer implementation, the system first receives the global 3D scene point cloud output by the multi-view blind spot completion and data fusion unit. It then uses the YOLOv8 instance segmentation algorithm combined with a point cloud clustering algorithm for target detection and segmentation, maintaining a detection frame rate of at least 20fps. Detection categories include personnel, fixed facilities, and mobile devices, representing a conventional technical solution. Specific implementation steps are not detailed here. For each detected target, a 3D axis-aligned bounding box is generated. Simultaneously, target attributes are quantified, and corresponding safety distances are assigned based on target type. With weighting coefficients The safe distance for personnel targets is 3m, for fixed facilities it is 1.5m, and for mobile devices it is 2m. For mobile device targets, their speed and direction of movement are calculated through point cloud sequence analysis to provide a basis for relative speed calculation.

[0062] During the implementation of the spatial matching and parameter calculation layer, the 3D bounding box of each target is spatially matched with the envelope of the extended load swing trajectory, and the following three core parameters are calculated: First, the 3D intersection-union ratio. By calculating the ratio of the intersection volume to the union volume of two 3D geometric shapes, it can be seen that when the envelope and the bounding box do not overlap, Second, the minimum Euclidean distance The GJK algorithm is used to calculate the shortest distance between the target bounding box surface and the load envelope surface, which is a conventional existing technical solution. The specific implementation steps are not detailed here. When the envelope overlaps with the bounding box... Third, relative speed The speed of the load center is obtained by subtracting the speed vector of the target center from the speed vector of the load center and taking the modulus. The speed of the load center is calculated by superimposing the speed vector of the hook suspension point and the speed vector of the load swing.

[0063] Specifically, the formula for classifying multi-level hazardous areas is as follows: ;in, This is a restricted area, corresponding to a Level 3 emergency control order; This is a high-risk area, corresponding to a Level 2 warning and speed limit control command; This is a general danger zone, corresponding to a Level 1 warning order; This is the safe zone, corresponding to normal operating instructions.

[0064] The above formula divides the tower crane operating area into four levels of danger zones based on the probability of collision risk. Different levels correspond to different warning levels and control strategies. The prohibited zone indicates that a collision is imminent and immediate emergency braking is required; the high-risk zone indicates an extremely high risk of collision and immediate deceleration and warning are required; the general danger zone indicates a potential risk of collision and requires continuous monitoring and warnings; the safe zone indicates no risk of collision and normal operation is permitted.

[0065] When implementing multi-level hazard zone generation, the collision risk probability of each target is first calculated based on a multi-factor weighted collision risk assessment model. Then, the hazard zone level of each target is determined according to the multi-level hazard zone classification formula. For multiple targets coexisting, the highest risk level is taken as the final risk level of the area. A contour surface generation algorithm is used to generate the 3D boundaries of each hazard zone, with a boundary resolution of 0.1m, and they are marked with different colors: red for prohibited areas, orange for highly hazardous areas, yellow for general hazardous areas, and green for safe areas. The hazard zone boundaries are dynamically updated as the load swing trajectory envelope is updated, with an update frequency of no less than 20Hz, ensuring that the hazard zones are synchronized with the actual risk status in real time.

[0066] The collision risk assessment and hazard area generation unit outputs collision risk and hazard area data in a standardized format, including: a list of collision risk probabilities for all targets (including target ID, type, location, and risk probability), 3D boundary data of multi-level hazard areas (including vertex coordinates, triangular face index, and area level), and detailed information on the highest-risk target (including risk level, distance, and relative speed). This data is transmitted in real-time to the hierarchical early warning and linkage control unit via an industrial Ethernet bus, triggering corresponding level of early warning and control commands. Simultaneously, all data is cached on the local hard drive of the edge computing unit for subsequent accident backtracking, model optimization, and safety analysis. The generated multi-level hazard areas and collision risk probabilities provide accurate risk quantification for the tower crane anti-collision system, effectively solving the problems of inaccurate early warnings and untimely responses in traditional systems, and significantly improving the safety of tower crane operations.

[0067] A graded early warning and linkage control unit, connected to the collision risk assessment and hazardous area generation unit, is used to trigger corresponding level audible and visual early warning signals and tower crane linkage control commands based on the collision risk level; specifically: The graded early warning and linkage control unit is connected to the collision risk assessment and hazardous area generation unit, undertaking the tasks of collision risk graded response, audible and visual early warning triggering, and linkage control of tower crane operation status. Its core function is to address the industry pain points of traditional tower crane anti-collision systems, such as single early warning methods, crude control strategies, and a lack of graded response mechanisms, leading to frequent false alarms interfering with normal operations and untimely emergency responses. During implementation, this unit receives collision risk probability data, multi-level hazardous area boundary data, and detailed information on the highest-risk target from the collision risk assessment and hazardous area generation unit. Through a four-level progressive processing logic, it generates standardized graded early warning signals and tower crane linkage control commands.

[0068] Specifically, the hierarchical early warning and linkage control unit has a built-in three-level hierarchical response control model, the model expression of which is: ;in, The set of control commands to be output; The preset three-level risk thresholds are set to 0.3, 0.6, and 0.9 respectively, and satisfy the following conditions: ; The system operates according to the normal operating instructions set. There are no warning signals, and the tower crane operates normally according to the operator's instructions. This is a set of Level 1 warning commands that triggers a local audible and visual warning in the cab, without restricting tower crane operation; This is a set of Level 2 early warning commands, triggering a comprehensive on-site audible and visual early warning system to limit the tower crane's operating speed. It is a set of three-level emergency control commands, which triggers a full-area audible and visual warning and a remote alarm, performs emergency braking, and cuts off power output.

[0069] The above formula uses a piecewise function to construct a four-level hierarchical response mechanism. Based on different ranges of collision risk probability, it triggers corresponding level warning signals and control commands to achieve a smooth transition from normal operation to emergency braking. In the low-risk range, only status monitoring is performed; in the medium- and high-risk range, the warning intensity is gradually increased and the tower crane's operating speed is limited; and in the extremely high-risk range, emergency braking is triggered immediately, balancing operational efficiency and safety requirements.

[0070] When optimizing the three-level graded response control model, the model is set up with four levels of progressive processing logic. The first level is the risk level determination layer, which determines the current risk level based on the collision risk probability; the second level is the graded early warning triggering layer, which generates audible and visual early warning signals of corresponding intensity based on the risk level; the third level is the linkage control execution layer, which generates tower crane operation control commands based on the risk level and sends them to the PLC; the fourth level is the feedback and safety verification layer, which monitors the command execution status and performs safety interlock verification.

[0071] Specifically, the dynamic adjustment formula for the graded audible and visual early warning parameters is as follows: ; ;in, This is the actual frequency at which the buzzer emits sound, measured in Hz. The basic frequency for Level 1 early warning is set at 1Hz. This is the frequency adjustment factor, with a default value of 10; The actual brightness of the warning light, with a value range of [0, 100]%; The base brightness for Level 1 warning is set at 30%. This is the brightness adjustment factor, with a default value of 0.7.

[0072] The above formula enables dynamic and continuous adjustment of the audible and visual warning intensity according to the probability of collision risk, ensuring that the urgency of the warning signal matches the actual risk level. Specifically, the buzzer frequency increases linearly with the risk probability, and the warning light brightness increases linearly with the risk probability, allowing operators to intuitively perceive the escalation of the risk level through changes in the warning signal.

[0073] When the tiered early warning system is implemented, corresponding warning signals are triggered based on the risk level: Level 1 warnings are triggered only in the tower crane operator's cab, with a buzzer frequency of 1Hz and a warning light brightness of 30%, flashing yellow, alerting the operator to potential risks; Level 2 warnings are triggered simultaneously in the operator's cab and throughout the entire tower crane operation area, with a buzzer frequency of 3-7Hz, a warning light brightness of 50%-70%, flashing orange, and a voice prompt broadcasting "Please note that there is a collision risk in the tower crane operation area"; Level 3 warnings are triggered throughout the entire area with the highest intensity, a buzzer frequency of 10Hz, a warning light brightness of 100%, solid red, and alarm information, including the tower crane number, risk location, risk level, and real-time on-site video, is simultaneously sent to the remote monitoring platform. The intensity of the warning signal is dynamically adjusted in real time according to changes in the risk probability to ensure the intuitiveness and effectiveness of the warnings.

[0074] Specifically, the formula for limiting the operating speed of tower cranes is as follows: ;in, This represents the maximum permissible operating speed of the tower crane, in m / s. The rated operating speed of the tower crane's corresponding mechanism includes the rated slewing speed, rated luffing speed, and rated hoisting speed; This is the speed limit coefficient, with a value of 1, indicating that when the risk probability reaches the level three threshold, the speed limit is 0; The risk thresholds are set at levels two and three, respectively, with values ​​of 0.6 and 0.9.

[0075] The above formula achieves a linear and smooth limit on the tower crane's operating speed as the collision risk probability increases. When the risk probability exceeds the secondary threshold, the operating speed gradually decreases as the risk increases. When the tertiary threshold is reached, the speed limit is 0, which avoids the violent load swaying caused by the sudden deceleration of the traditional system, while reserving sufficient reaction time for emergency braking.

[0076] When the linkage control execution layer is implemented, the corresponding tower crane control strategy is executed according to the risk level: Level 1 warning does not restrict any tower crane operation, but only provides a warning prompt; Level 2 warning restricts the operating speed of the tower crane's slewing, luffing, and hoisting mechanisms. When the risk probability is 0.6, the speed is limited to 100% of the rated speed. When the risk probability increases to 0.9, the speed is linearly reduced to 0% of the rated speed, and any reverse operation other than emergency stop is prohibited; Level 3 warning immediately executes emergency braking, cuts off the power output of all operating mechanisms, and activates the mechanical brake device. The braking process adopts a smooth deceleration curve, and the deceleration is controlled within 3m / s² to avoid large swings of the load due to emergency stop. All control commands are sent to the tower crane PLC control system via the Modbus-TCP protocol.

[0077] Specifically, the control command safety interlock formula is as follows: ;in, The final control command; The original control command output by the hierarchical response control model; ∧ represents the logical AND operator; For logical NOT operator; The manual intervention switch is set to a value of 1, which indicates that the operator has triggered manual intervention, and a value of 0 indicates that there is no manual intervention. This represents the system fault status. A value of 1 indicates that the system has abnormalities such as sensor failure or communication failure, while a value of 0 indicates that the system is normal.

[0078] The above formula implements a safety interlock mechanism for control commands, ensuring that automatic control commands can only take effect when there is no manual intervention and the system is fault-free; when the operator triggers the manual intervention switch or the system detects its own fault, the automatic control commands are blocked, and the system switches to manual operation mode to ensure operational safety.

[0079] When the feedback and safety verification layer is implemented, the operating status of each mechanism of the tower crane, the working status of sensors, and the status of communication links are monitored in real time. When any system fault is detected, all automatic control commands are immediately blocked, a system fault warning is triggered, and the system switches to manual operation mode. Simultaneously, a manual intervention priority mechanism is set up, allowing the operator to disconnect automatic control and take over tower crane operation at any time via a manual intervention switch in the cab. After executing an emergency braking command, the system automatically locks the tower crane's operating status; normal operation can only resume after the operator confirms the risk has been eliminated and manually resets the system. Furthermore, the system records the execution time, execution status, and corresponding risk data of all warning and control commands.

[0080] The tiered early warning and linkage control unit generates tiered early warning signals that directly drive corresponding audible and visual alarm devices. Linkage control commands are then sent to the tower crane's PLC control system for execution. Simultaneously, all early warning and control data is uploaded in real-time to a remote monitoring platform via 4G / 5G networks. The platform can view the tower crane's risk status, early warning information, and control records in real time, and can remotely trigger emergency braking commands. Through its tiered response and linkage control mechanism, this unit achieves precise early warning and proactive intervention for collision risks, effectively reducing the probability of tower crane collision accidents while minimizing interference with normal operations, thus improving the safety and efficiency of tower crane operations.

[0081] The model parameter closed-loop feedback adjustment unit is connected to the multi-view blind spot completion and data fusion unit, the load swing dynamic envelope prediction unit, and the hierarchical early warning and linkage control unit, respectively. It is used to dynamically optimize model parameters based on actual collision events, false alarms, and missed alarms, forming a closed-loop anti-collision recognition and control mechanism. Specifically: The model parameter closed-loop feedback adjustment unit is connected to the multi-view blind zone completion and data fusion unit, the load swing dynamic envelope prediction unit, the collision risk assessment and hazardous area generation unit, and the hierarchical early warning and linkage control unit. It undertakes the tasks of system operation data acquisition, error analysis, adaptive adjustment of model parameters, and effect verification. Its core function is to address the industry pain points of traditional tower crane anti-collision systems, which suffer from fixed model parameters that cannot adapt to different operating environments and conditions, leading to decreased blind zone completion accuracy, increased swing trajectory prediction errors, and higher false alarm and missed alarm rates after long-term operation. During implementation, this unit collects real-time operating data from each module, records of false alarms and missed alarms, and actual collision event data. It dynamically optimizes and adjusts the parameters of all core models through a four-level progressive processing logic. The unit parameter adjustment cycle is once per hour, and immediate adjustments can be triggered in emergencies to ensure the long-term stability and reliability of the system.

[0082] Specifically, the model parameter closed-loop feedback adjustment unit incorporates a gradient descent-based adaptive parameter adjustment model, the model expression of which is: ;in, The adjusted model parameter set includes all core parameters such as the multi-view blind spot completion model weights, dynamic model damping coefficients, and collision risk assessment weights. The current model parameter set; The learning rate is adjusted based on feedback, with a value ranging from 0.001 to 0.01, controlling the step size of the parameter adjustment; This is the gradient vector of the loss function with respect to the current parameters, reflecting the degree of influence of parameter changes on the loss value; For false positive and false negative datasets The calculated overall loss value of the model; This is a linear rectified function that outputs the input value when the input is greater than 0, and outputs 0 otherwise. The preset loss threshold is set to 0.1. When the loss value exceeds this threshold, parameter adjustment is triggered.

[0083] The above expression uses the gradient descent algorithm to achieve adaptive adjustment of model parameters. The direction of parameter adjustment is determined by calculating the gradient of the loss function with respect to the current parameters. The linear rectified function is used to control the adjustment triggering conditions. Parameters are updated only when the loss value exceeds a preset threshold, avoiding over-adjustment that could lead to system instability. The adjusted parameters can effectively reduce model errors and improve the system's adaptability under different operating conditions.

[0084] When implementing the optimization of the parameter adaptive adjustment model based on gradient descent, the model is set up with four levels of progressive processing logic. The first level is the running data acquisition and preprocessing layer, which collects and cleans the running data and error data of each module; the second level is the error analysis and attribution layer, which calculates the overall loss value of the model and locates the source of error; the third level is the model parameter adaptive adjustment layer, which updates the model parameters based on the gradient descent algorithm; and the fourth level is the adjustment effect verification layer, which verifies the model performance after parameter adjustment and confirms that the parameters are effective.

[0085] Specifically, the formula for calculating the model's overall loss is as follows: ;in, This is the overall loss value of the model, and its value ranges from [0,1]. The weights for false positive loss, false negative loss, trajectory prediction loss, and blind spot completion loss are respectively set to 0.2, 0.5, 0.2, and 0.1, and satisfy the following conditions: ; The value represents the loss due to false alarms, and is calculated as the ratio of the number of false alarms to the total number of warnings. For underreported losses, the value is the ratio of the number of underreported incidents to the actual number of hazardous events; The trajectory prediction loss is the ratio of the average Euclidean distance between the predicted trajectory and the actual trajectory to the length of the wire rope. The calibration loss is used to compensate for the blind zone, and its value is the ratio of the average error between the compensated point cloud and the actual point cloud to the radius of the blind zone.

[0086] The above formula comprehensively considers four dimensions: false alarm loss, missed alarm loss, trajectory prediction loss, and blind spot completion calibration loss, to fully quantify system operating errors. Among them, missed alarm loss has the highest weight because it may lead to serious safety accidents; false alarm loss is the second highest, to avoid frequent false alarms interfering with normal operations; trajectory prediction and blind spot completion losses are the basic losses, ensuring the system's core perception and prediction capabilities.

[0087] During the implementation of the data acquisition and preprocessing layer, this unit collects operational data from each module in real time via the industrial Ethernet bus, including: blind zone completion accuracy data and point cloud confidence data from the multi-view blind zone completion unit; trajectory prediction error data and envelope boundary data from the load swing prediction unit; risk probability data and target recognition accuracy data from the collision risk assessment unit; and warning records and control command execution records from the hierarchical early warning control unit. Simultaneously, manually labeled false alarm and missed alarm records, as well as actual collision event data, are collected at a frequency of once per minute. The collected data is preprocessed to remove duplicate and abnormal data, and missing data is filled using linear interpolation. The preprocessed data is then stored in a local database.

[0088] When implementing error analysis and attribution layer, first, system operation data is collected hourly to calculate the number of false alarms, the number of missed alarms, the average error of trajectory prediction, and the average error of blind spot completion per hour. Then, the comprehensive loss value per hour is calculated according to the model's comprehensive loss calculation formula. ,when When this occurs, the parameter adjustment process is triggered. Error attribution analysis is performed on loss values ​​exceeding the threshold. The main sources of error are determined by calculating the proportion of each sub-item of loss: if the proportion of missed reports exceeds 50%, the risk threshold and weight coefficients of the collision risk assessment model are mainly adjusted; if the proportion of trajectory prediction loss exceeds 40%, the damping coefficient and drag coefficient of the dynamics model are mainly adjusted; if the proportion of blind spot completion loss exceeds 30%, the sensor weight coefficients of the multi-view fusion model are mainly adjusted.

[0089] Specifically, the formula for adjusting the parameters of each module is as follows: Blind spot completion module: ; Dynamics prediction module: ; Risk assessment module: ;in, For the first The adjustment amount of the fusion weights of each sensor; This is the adjustment amount for the air damping coefficient; This is the adjustment amount for the weighting coefficients of the three-dimensional intersection-union ratio; These are the adjustment step sizes for each module, with values ​​of 0.005, 0.01, and 0.008 respectively. , , These are the gradients of the corresponding loss terms with respect to the parameters; This is a sign function that returns the sign of the argument.

[0090] The above formulas employ differentiated adjustment step sizes and directions to achieve precise parameter optimization, taking into account the error characteristics of different modules. Specifically, the blind spot completion module adjusts the sensor fusion weights, the dynamics prediction module adjusts the air damping coefficient, and the risk assessment module adjusts the 3D cross-validation ratio weights, ensuring that the parameter adjustments for each module can specifically reduce the corresponding errors.

[0091] When implementing the adaptive adjustment layer for model parameters, the corresponding parameter adjustment strategy is selected based on the error attribution results, and the adjustment amount for each parameter is calculated. Parameter adjustment adopts an incremental approach, with each adjustment not exceeding 10% of the current parameter value to avoid system instability caused by sudden parameter changes. For the sensor weight coefficients of the multi-view blind spot completion model, renormalization is required after adjustment to ensure that the sum of all weights remains 1. For the damping coefficient and drag coefficient of the dynamic model, the adjustment range is limited to 0.05~0.2 and 0.8~1.5, respectively. For the weight coefficients of the collision risk assessment model, renormalization is required after adjustment to ensure... All parameter adjustment records are stored in the database, including the adjustment time, parameters before adjustment, parameters after adjustment, and reasons for adjustment.

[0092] When implementing the effect verification layer, after the parameters are adjusted, the system operation data is continuously monitored for 1 hour, and the comprehensive loss value is recalculated. .like and If the parameter adjustment is confirmed to be effective, the adjusted parameters will be sent to the corresponding module to replace the original parameters; if If the error occurs, the parameter adjustment will be cancelled, the original parameters will be restored, and error attribution analysis will be performed again to adjust the optimization strategy. In case of emergencies, such as a collision event or three consecutive missed reports, the immediate parameter adjustment process will be triggered, skipping the waiting period, immediately performing error analysis and parameter updates, and continuously monitoring for 3 hours after the adjustment to verify the effect.

[0093] The model parameter closed-loop feedback adjustment unit transmits the adjusted effective parameters in real time via the industrial Ethernet bus to the multi-view blind zone completion and data fusion unit, the load swing dynamic envelope prediction unit, and the collision risk assessment and hazardous area generation unit. Each unit takes effect immediately upon receiving the new parameters. Simultaneously, all parameter adjustment records, error analysis reports, and effect verification data are stored in a local database and uploaded to a remote monitoring platform via a 4G / 5G network. The platform can view the historical trends and adjustment effects of system parameters. This unit, through its closed-loop feedback adjustment mechanism, enables the system to continuously adapt to different operating environments and conditions, maintaining high-precision collision avoidance recognition and control capabilities over the long term, significantly improving the system's robustness and lifespan.

[0094] In the several embodiments provided by this invention, it should be understood that the disclosed system can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for example, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0095] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module described above can be implemented in hardware or in the form of hardware plus software functional modules.

[0097] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the essential characteristics of the present invention.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multi-sensor fusion anti-collision recognition control system for intelligent tower cranes, characterized in that, include: The multi-source sensing data acquisition and preprocessing unit is used to collect tower crane attitude parameters, load status parameters, environmental meteorological parameters, three-dimensional data of the surrounding scene, and tower crane operation command parameters through a multi-dimensional sensor array deployed at the base of the boom, both sides of the trolley, the top of the tower body, and the hook body. It forms a multi-source sensing data matrix and performs format regularization and noise filtering. The multi-view blind spot completion and data fusion unit is connected to the multi-source sensing data acquisition and preprocessing unit. It has a built-in multi-view blind spot completion and data fusion model. It completes the non-transparent blind spot below the hook through multi-view registration and occlusion inference, and generates a point cloud of the full-domain three-dimensional scene of tower crane operation. The load swing dynamic envelope prediction unit is connected to the multi-source sensing data acquisition and preprocessing unit. It has a built-in nonlinear pendulum dynamic prediction model, which considers the effects of wind speed disturbance and operation emergency stop, and calculates the real-time swing state of the load and the swing trajectory envelope within a preset time period in the future. The collision risk assessment and hazardous area generation unit is connected to the multi-view blind spot completion and data fusion unit and the load swing dynamic envelope prediction unit, respectively. It is used to fuse the full-domain three-dimensional scene point cloud and the load swing trajectory envelope to generate multi-level hazardous areas and calculate the collision risk probability. The graded early warning and linkage control unit is connected to the collision risk assessment and danger zone generation unit, and is used to trigger corresponding level of audible and visual early warning signals and tower crane linkage control commands according to the collision risk level. The model parameter closed-loop feedback adjustment unit is connected to the multi-view blind zone completion and data fusion unit, the load swing dynamic envelope prediction unit, and the hierarchical early warning and linkage control unit, respectively. It is used to dynamically optimize the model parameters based on actual collision events, false alarms and missed alarms, to form a closed-loop anti-collision recognition and control mechanism.

2. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The expression for the multi-view blind spot completion and data fusion model is: ;in, For the generated full-domain 3D scene point cloud of tower crane operation, Here are the fusion weighting coefficients for the data from each sensor, and ; The point cloud matrix of the scene captured by the panoramic camera at the base of the boom. This is the effective region mask matrix for the camera at the root of the upper arm; The scene point cloud matrix captured by the binocular camera on the left side of the vehicle. The occlusion mask matrix for the camera on the left side of the car; The scene point cloud matrix captured by the binocular camera on the right side of the vehicle. The occlusion mask matrix for the camera on the right side of the car; The point cloud matrix of the scene collected by the millimeter-wave radar at the top of the tower. This is the mask matrix for the effective detection area of ​​millimeter-wave radar. The point cloud matrix projected below the suspended object is generated with the assistance of the hook body IMU. Complete the mask matrix for the blind spot below the hook.

3. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The expression for the nonlinear simple pendulum dynamics prediction model is: ； ;in, Let be the swing angle of the load relative to the vertical direction. The angular velocity of the swing. This is the angular acceleration of the swing. It is the acceleration due to gravity. This refers to the effective length of the wire rope. For load quality; For real-time wind load, The angle between the wind direction and the tower crane boom; This refers to the air damping coefficient; Tangential acceleration generated during tower crane operation, including lifting, slewing, luffing, and emergency stop acceleration; The azimuth angle of the load swing; The lateral angular velocity; This is the lateral angular acceleration; The envelope of the load swing trajectory within the next T seconds; The real-time coordinates of the hook suspension point; To predict the time step, The preset prediction duration.

4. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The expression for the collision risk assessment model is: ;in, The collision risk probability has a value range of [0,1]. The envelope overlap weighting coefficient is used. This is the distance weighting coefficient. The relative velocity weighting coefficient, and ; For load swing envelope bounding box of the target object The intersection and union ratio; The minimum distance between the target object and the load swing envelope; To establish a preset safe distance; The relative speed between the load and the target object.

5. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 4, characterized in that, The expression for the hierarchical early warning and linkage control logic is as follows: ;in, The set of control commands to be output; The preset level three risk threshold, and ; This is a normal operation command; no warning is required. This is a Level 1 warning command, triggering an audible and visual warning in the driver's cab. This is a Level 2 warning command, triggering a comprehensive on-site audible and visual warning and limiting the tower crane's operating speed. This is a Level 3 emergency control command, triggering emergency braking and cutting off the tower crane's power output.

6. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The expression for adjusting the model parameters using closed-loop feedback is: ;in, The adjusted model parameter set; The current model parameter set; Adjust the learning rate based on feedback; This represents the gradient of the loss function with respect to the current parameters. For false positive and false negative datasets The calculated model loss value; It is a linear rectified function; A preset loss threshold is set, and parameter adjustments are triggered when the loss value exceeds the threshold.

7. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The load swing dynamic envelope prediction unit includes the following steps: The load mass, effective length of the wire rope, and coordinates of the hook suspension point are collected in real time. The initial swing angle and swing angular velocity of the load are obtained through the hook IMU and used as the initial input parameters of the dynamic model. Real-time wind speed and direction data are collected to calculate the disturbance term of wind load on load swing. At the same time, tower crane operation commands are collected and the lifting, slewing, luffing and emergency stop accelerations are analyzed as external excitation terms of the model. Based on the nonlinear pendulum dynamics equation, the fourth-order Runge-Kutta method is used to solve for the load swing angle and azimuth angle at each time step within the next T seconds, and the load spatial coordinates at the corresponding time are calculated. Perform convex hull calculation on the load spatial coordinates at all prediction times to generate a continuous load swing trajectory envelope. Then, expand the boundary of the envelope according to the wind speed fluctuation range to obtain the final predicted swing envelope.

8. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The collision risk assessment and hazardous area generation unit includes the following steps: Extract target objects of people, fixed facilities and mobile devices from the point cloud of the global 3D scene, generate a 3D bounding box for each target object and update its position and motion status in real time; Spatial matching is performed between the load swing trajectory envelope and the 3D bounding box of all target objects, and the intersection-union ratio and minimum distance between each target object and the swing envelope are calculated. Differentiated safety distances and weighting coefficients are set according to the type of the target object, and the collision risk probability of each target object is calculated by combining the load and the relative speed of the target object; Based on the probability of collision risk, multiple levels of danger zones are divided, including no-entry zones, high-risk zones, general danger zones, and safe zones, and the boundaries of each danger zone are updated in real time.

9. The intelligent tower crane multi-sensor fusion anti-collision recognition control system according to claim 1, characterized in that, The graded early warning and linkage control unit includes the following steps: The calculated collision risk probability is compared with the preset three-level risk threshold to determine the current risk level; The warning signal is triggered according to the risk level, and the on-site broadcast and remote monitoring platform pop-up alarm are also triggered. Execute the corresponding tower crane control commands according to the risk level; Record the execution time, execution status, and corresponding risk data of early warning and control commands, and store them synchronously in the system database.

10. A multi-sensor fusion anti-collision recognition control system for intelligent tower cranes according to any one of claims 1-9, characterized in that, Includes the following steps: S1. Collect multi-source sensing data of tower crane operation and surrounding environment. The multi-source sensing data includes tower crane attitude parameters, load status parameters, environmental meteorological parameters, three-dimensional data of surrounding scene and tower crane operation command parameters. The data is collected synchronously by a multi-dimensional sensor array deployed at different locations of the tower crane to form a multi-source sensing data matrix. S2, input the multi-source perception data matrix into the multi-view blind spot completion and data fusion model, and through multi-view registration, occlusion area reasoning and point cloud fusion algorithm, complete the non-transparent blind spot below the hook caused by the trolley, wire rope and suspended object, and generate the point cloud of the full-domain three-dimensional scene of tower crane operation. S3. Input the load state parameters, environmental meteorological parameters and tower crane operation command parameters into the nonlinear pendulum dynamic prediction model, consider the dynamic effects of wind speed disturbance and emergency stop of operation, and calculate the real-time swing state of the load and the swing trajectory envelope within the future preset time period. S4, integrate the full-domain 3D scene point cloud and the load swing trajectory envelope to generate dynamically updated multi-level danger zones, and calculate the collision risk probability of personnel, facilities and load swing envelopes in different zones through a collision risk assessment model; S5, based on the collision risk probability, classify the risk level, trigger the corresponding level of audible and visual warning signal and tower crane linkage control command to realize active intervention to prevent collision; S6 collects actual collision events, false alarms and missed alarms data, and dynamically adjusts the parameters of the multi-view blind spot completion and data fusion model and the nonlinear pendulum dynamics prediction model to form a closed-loop collision avoidance identification and control mechanism.

Images