A tower crane collision warning system for multi-modal data fusion

Abstract

The present invention provides a tower crane collision warning system for multi-modal data fusion, which can obtain the dynamic data of tower crane operation, environmental data and personnel activity trajectories in real time through a multi-modal data acquisition module, and uses time synchronization technology to ensure the accuracy of data. The intelligent data fusion and analysis module adopts a multi-level data fusion strategy, combines an improved decision tree algorithm and a dynamic threshold method, calculates the collision risk in real time and outputs a warning result. The warning management module generates a collision warning signal based on a risk response mechanism, and automatically optimizes the warning management through a deep learning algorithm. The adaptive optimization decision module associates the dynamic data of the construction site with the collision warning signal, evaluates the safety risk in real time, adjusts the working parameters or scheduling of the tower crane, so as to reduce the collision risk and improve the construction efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of tower crane management, and specifically to a tower crane collision warning system with multi-modal data fusion. Background Art

[0002] In the field of tower crane management, the safe operation of tower cranes is crucial for ensuring construction progress and personnel safety. Traditional tower crane safety warning systems mainly rely on the monitoring of the tower crane's own status, such as parameters like load, height, and amplitude, as well as simple environmental monitoring, such as wind speed and wind direction. These systems can provide safety warnings to a certain extent, but there are obvious limitations.

[0003] The deficiencies of the existing technologies in the field of tower crane collision warning are mainly reflected in the insufficient ability to perceive complex construction environments, the singleness of data, and the lack of deep correlation between data. These deficiencies lead to the inability of existing systems to comprehensively and accurately evaluate collision risks, thus unable to effectively reduce collision risks and improve construction efficiency. Summary of the Invention

[0004] (1) Technical Problems to be Solved

[0005] Aiming at the deficiencies of the existing technologies, the present invention provides a tower crane collision warning system with multi-modal data fusion, which solves the problem of accurately evaluating the collision risk of tower cranes and real-time dynamic warning by fusing multi-modal data and performing data processing and deep learning technologies.

[0006] (2) Technical Solutions

[0007] To achieve the above objectives, the present invention is realized through the following technical solutions: A tower crane collision warning system with multi-modal data fusion, including:

[0008] Multi-modal data acquisition module: Real-time acquisition of each data source, including dynamic data of tower crane operation, environmental data, and the activity trajectories of construction site personnel. The data sources are synchronized in real time through time synchronization technology to ensure the timeliness and accuracy of data acquisition;

[0009] Intelligent data fusion and analysis module: Adopting a multi-level data fusion strategy, constructing a fusion network architecture based on the time-synchronized data, and dynamically weighted fusion of multi-modal data; Using an improved decision tree algorithm and a dynamic threshold method to calculate the collision risk in real time according to the tower crane and environmental data and output the warning result;

[0010] Early warning management module: Through a risk response mechanism, it generates a collision early warning signal according to the real-time early warning result. The collision early warning signal includes collision early warning information, collision risk level assessment and response suggestions. The early warning management module automatically optimizes early warning management through deep learning algorithms, continuously updates the intelligent data fusion and analysis module based on historical processing data and on-site feedback, and pushes the collision early warning signal to relevant personnel;

[0011] Adaptive optimization decision-making module: Adopts a decision-making mechanism based on feedback adjustment, correlates and analyzes the dynamic data of the construction site with the collision early warning signal, evaluates the safety risk of the construction site in real time, and adjusts the working parameters or scheduling of the tower crane according to the evaluated risk to reduce potential collision risks and improve construction efficiency.

[0012] In the multi-modal data acquisition module, sensors are installed at key positions of the tower crane, such as near the boom and the cab of the tower crane, to capture image information of the tower crane and its surrounding environment at a high frequency; for example, it can capture more than 30 high-definition images per second to identify the position, attitude of the tower crane and the distribution of surrounding obstacles, including identifying the relative positions of the tower crane and obstacles such as buildings, other tower cranes, and construction vehicles, as well as the tilt angle and rotation direction of the tower crane boom; a high-precision lidar measures the distance between the tower crane and surrounding objects, providing accurate three-dimensional spatial data; the scanning frequency of the lidar is as high as tens of thousands of times per second, generating high-density point cloud data. For example, during the rotation of the tower crane boom, the lidar monitors the distance between the boom and surrounding buildings in real time to ensure a safe distance; the high resolution and fast scanning ability of the lidar ensure the real-time and accuracy of the data, enabling it to provide reliable data support in a complex and changeable construction environment; ultrasonic sensors are used for detecting obstacles at close range. For example, in areas under the tower crane boom or where the line of sight is blocked, ultrasonic sensors detect obstacles within a few meters, such as construction workers or small equipment; tower crane status monitoring sensors monitor the operating status of the tower crane in real time, including key parameters such as load, height, and amplitude; time synchronization technology is used to process data. Through the built-in clock synchronization unit, the timestamps of all sensors and positioning systems are calibrated; for example, each sensor attaches a timestamp when collecting data, and the timestamps are synchronized through a computer to ensure data synchronization.

[0013] The time synchronization technology incorporates a high-precision atomic clock as the time synchronization reference source for the entire system. With its extremely high stability and precision, the atomic clock provides time accuracy at the microsecond level. When the system starts up, all sensors are connected to the atomic clock through a synchronization interface to receive the globally unified time reference signal. This time reference signal is used by each sensor as the starting time point for data acquisition. For example, when a high-resolution vision sensor starts capturing images, it records the time reference signal received from the atomic clock as the time origin of the image data. During data acquisition, each sensor embeds a timestamp in the collected data packet. The timestamp records the exact time of data acquisition, enabling accurate alignment and fusion of data from different sensors in subsequent data processing. To ensure the accuracy of the timestamp, the system employs a dynamic time calibration algorithm. This algorithm continuously monitors the time deviation between each sensor and the atomic clock and dynamically adjusts the sensor's timestamp based on the monitoring results. Specifically, the dynamic time calibration algorithm calculates the time deviation by periodically sending time synchronization signals to each sensor and receiving the response signals returned by the sensors. For example, a synchronization signal is sent once per second, and the sensor immediately returns a response signal upon receiving the signal. The time deviation of the sensor is determined by calculating the time difference between the sent and received signals. If it is found that the timestamp of a certain sensor is 10 microseconds slower than the atomic clock, this deviation value is calculated and subtracted from the timestamp in the data packet collected by that sensor, resulting in a timestamp that is fully synchronized with the atomic clock.

[0014] The multi-level data fusion strategy aligns the dynamic data of tower crane operations, environmental data, and the activity trajectory data of construction site personnel according to the same timestamp for input; for example, when the tower crane operates at a certain time, the relevant tower crane operation data, environmental data, and personnel activity data will all be marked with the same timestamp; in the parallel processing stage, the tower crane operation data is collected by sensors and then calibrated and noise-reduced; for example, the random fluctuations in the tower crane operation data are removed through a filtering algorithm, and the environmental data is standardized through meteorological parameters, converting meteorological parameters such as temperature, humidity, and wind speed into a unified format and dimension; the personnel activity trajectory data completes coordinate unification and abnormal trajectory elimination, unifying the personnel position data from different sources into the same coordinate system through coordinate transformation, and using anomaly detection to eliminate unreasonable trajectory points, such as trajectory points with sudden jumps or abnormal speeds; the processed data enters the interaction layer, and a cross-modal attention mechanism is used to make the features of each data interact and influence each other; by calculating the correlation between different data modalities, the influence of important features is enhanced; for example, when an abnormal operation in the tower crane operation data coincides with adverse weather conditions in the environmental data, the cross-modal attention mechanism increases the weights of these two features because they jointly indicate a higher collision risk; according to the interaction and influence between the data, the data enters the feature weight assignment layer, which calculates the weights of each feature based on historical collision cases and real-time data statistics; for example, if historical data shows that tower crane collision accidents occur frequently under strong wind conditions, the weight of the wind speed feature will be increased accordingly; similarly, if abnormal personnel activities are frequently detected in real-time monitoring, such as a large number of people gathering in the tower crane operation area, the weight of the personnel trajectory feature will be increased because this increases the risk of collision; with the assigned weights, the data enters the fusion layer, and weighted summation is used for feature fusion. Weighted summation enables each feature to be fused according to its weight contribution, finally forming a fusion dataset that comprehensively reflects the operating environment, operation status, and personnel activities of the tower crane; for example, if the weight of the wind speed feature is 0.3, the weight of the personnel trajectory feature is 0.4, and the weight of the tower crane operation data is 0.3, then the final fusion dataset will be the weighted sum of these three feature data.

[0015] The improved decision tree algorithm deeply analyzes the weather environment data in the fusion dataset to identify the key features that have a direct impact on the tower crane collision risk under specific weather conditions; for example, in strong wind weather, wind speed and wind direction are the key factors affecting the stability of tower cranes and collision risks; by real-time monitoring of weather dynamics, the decision tree can dynamically adjust the focus of attention to ensure that the analysis results are closely related to the current environmental conditions; after determining the key features, a weight adjustment mechanism is launched, which is achieved by analyzing the historical collision event data. Specifically, the occurrence frequency of key features at the time of collision and the correlation strength with the collision result are statistically analyzed; for example, if historical data shows that the incidence of collision accidents increases significantly when the wind speed exceeds a certain threshold under strong wind conditions, then the weight of the key feature of wind speed will be increased accordingly; for each key feature, the proportion of the number of times it appears in all collision events is statistically analyzed, as well as the correlation coefficient between the change amplitude of the key feature before and after the collision and the severity of the collision; the statistical data assigns a dynamic weight to each key feature, and the weight value reflects the risk contribution degree of the feature under the current weather conditions; taking the wind speed feature as an example, assume that in historical collision events, the incidence of collision accidents increases significantly when the wind speed exceeds 8 m / s, and the change amplitude of the wind speed is positively correlated with the severity of the collision; based on these statistical data, the algorithm assigns a relatively high dynamic weight to the wind speed feature, such as 0.6; the decision tree after weight adjustment takes the real-time collected tower crane operation data and environmental monitoring data as input. Starting from the root node, the decision tree screens the input data layer by layer according to the key features and their weights; at each node, according to the comparison result of the feature value with the preset threshold, the data flow is determined to different child nodes; for example, if the current wind speed feature value exceeds the preset threshold of 8 m / s, the data will flow to the child node representing high risk; if it is lower than this threshold, it will flow to the low-risk child node; as the data is transmitted in the decision tree, each layer of nodes calculates a local risk score according to the weight of the key feature and the actual value of the data; these local scores are accumulated layer by layer in the hierarchical structure of the decision tree, and finally a comprehensive collision risk value is obtained at the leaf node. This risk value is a continuous numerical value between 0 and 1, and the closer the numerical value is to 1, the higher the collision risk; for example, if the final risk value is 0.8, it indicates that there is a relatively high collision risk in the current tower crane operation environment and corresponding safety measures need to be taken immediately.

[0016] The risk response mechanism is used to connect the intelligent data fusion and analysis module with actual early warnings. This mechanism receives the collision risk values output from the intelligent data fusion and analysis module and constructs a set of risk grading criteria based on this, clearly dividing risks into three levels: low, medium, and high. For example, a risk value between 0 and 0.33 is defined as low risk, 0.33 to 0.67 as medium risk, and 0.67 to 1 as high risk. This grading method enables the system to take targeted response measures according to different risk levels. For risks of different levels, the system has preset corresponding response strategy templates. The templates cover in detail the content of early warning information, the scope of recipients, and recommended countermeasures. For example, in the case of low risk, the early warning information only reminds the operator to pay attention to potential minor risks, the recipients are mainly tower crane operators, and the recommended measure is to increase the observation frequency. In the case of high risk, the early warning information will be more urgent, the recipients are extended to all relevant personnel on site, including safety supervisors and project managers, and the recommended measures include stopping the tower crane operation and evacuating personnel. The deep learning algorithm conducts in-depth learning and analysis on historical processed data and on-site feedback by constructing a complex neural network model. The data input into the model is multi-dimensional, including the operation data of the tower crane, environmental data, and personnel activity data, etc. A combined architecture of LSTM and fully connected layers is adopted. LSTM captures the long-term dependencies in time series data and understands the temporal correlation between changes in risk levels and the implementation of countermeasures. For example, LSTM identifies how the gradual increase in wind speed within a certain time period affects the stability of the tower crane and how this change is related to the increase in collision risk. The fully connected layer is responsible for integrating and mapping the features output by LSTM, and finally outputs optimized early warning management parameters. During the training process, the model aims to minimize the difference between the predicted risk level and the actual event that occurs, and continuously adjusts the network weights through the backpropagation algorithm. As the training progresses, the deep learning algorithm gradually masters the mapping relationship between the risk change law and the optimal countermeasure plan. For example, the algorithm can learn which countermeasures can most effectively reduce the collision risk under a certain meteorological condition.

[0017] The adaptive optimization decision-making module performs correlation analysis on the dynamic data of the construction site, including tower crane operation data, environmental data, and personnel activity trajectory data, and the collision warning signals output by the warning management module; by receiving this data in real time, it comprehensively evaluates the safety risks of the construction site; during the risk assessment process, the collision risk value is set as the main weight factor and a relatively high weight is assigned, such as 70%, because the collision risk value directly reflects the current safety status; the remaining data, such as the load of the tower crane, wind speed, and personnel distance, are used as auxiliary variables and relatively low weights are assigned, such as 10% for each variable; the total weight is 100%, ensuring the comprehensiveness and balance of the evaluation; each variable will be standardized before evaluation to eliminate the influence of different dimensions and data ranges; for example, the wind speed data is standardized to between 0 and 1 to make different variables comparable, and the standardized variables are multiplied by the corresponding weights and weighted and summed to obtain a comprehensive risk score; the comprehensive risk score is compared with a preset threshold to determine the risk level as low, medium, or high; for example, if the comprehensive risk score exceeds 0.8, it is determined as a high risk; after determining the risk level, the pre-stored decision rules are triggered according to the level; for example, for high-risk situations, the decision rule is to immediately stop the tower crane operation and issue an alarm; after the system runs, it will collect real-time effect data after the decision is executed, such as the on-site situation and personnel reactions after the tower crane stops, as feedback input; the feedback data is compared with the expected performance indicators to evaluate the effectiveness of the decision. If the feedback shows that the collision risk is not effectively controlled or the construction progress is adversely affected, it indicates that the current decision needs to be adjusted; according to the adjustment logic corresponding to the risk level, the parameters in the decision rules are dynamically adjusted; for example, if it is found that the decision in high-risk situations is too conservative, resulting in unnecessary shutdowns, the system will reduce the sensitivity of high-risk decisions and adjust the parameters to optimize the decision-making process.

[0018] (III) Beneficial effects

[0019] The present invention provides a tower crane collision warning system with multi-modal data fusion, having the following beneficial effects:

[0020] 1. By fusing multi-modal data, the present invention realizes a comprehensive perception of the tower crane operation environment, significantly improves the accuracy and reliability of the warning system, effectively reduces false alarms and missed alarms, and ensures the safety of the construction site.

[0021] 2. Through a multi-level data fusion strategy and an improved decision tree algorithm, the present invention realizes the dynamic assessment and real-time warning of collision risks, significantly improves the timeliness and effectiveness of the warning, and effectively reduces the accident risk caused by warning delays.

[0022] 3. Through the adaptive optimization decision-making module, the present invention realizes the dynamic adjustment of the working parameters of tower cranes, significantly improves the construction efficiency, effectively reduces the construction delays caused by collision risks, and provides an intelligent solution for the safety management of the construction industry.

[0023] 4. Through the learning and analysis of historical data and on-site feedback by the deep learning algorithm, the present invention realizes the automatic optimization of early warning management, improves the intelligent level of the system, reduces the need for manual intervention, improves the performance of the early warning system, and also provides a more scientific and systematic decision-making basis for construction safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic flow diagram of the present invention;

[0025] Figure 2 It is a schematic flow diagram of multi-level data fusion;

[0026] Figure 3 It is a schematic diagram of the improved decision tree algorithm. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0028] Reference Figure 1, the multi-modal data acquisition module is responsible for real-time acquisition of dynamic data of tower crane operations, environmental data, and the movement trajectories of construction site personnel; specifically, this module is equipped with a variety of high-precision sensors, including high-resolution visual sensors, high-precision lidar, ultrasonic sensors, and tower crane status monitoring sensors; the high-resolution visual sensors are installed at key positions of the tower crane, such as near the boom and cab of the tower crane, to capture images of the tower crane and its surrounding environment at high frequency to identify the position, attitude of the tower crane, and the distribution of surrounding obstacles; for example, identify the relative positions of the tower crane with obstacles such as buildings, other tower cranes, and construction vehicles, as well as the tilt angle and rotation direction of the tower crane boom; the high-precision lidar is responsible for measuring the distance between the tower crane and surrounding objects, providing accurate three-dimensional spatial data; the scanning frequency of the lidar is as high as tens of thousands of times per second, generating high-density point cloud data; for example, during the rotation of the tower crane boom, the lidar monitors the distance between the boom and surrounding buildings in real time to ensure a safe distance; the high resolution and fast scanning ability of the lidar ensure the real-time and accuracy of the data, enabling it to provide reliable data support in complex and changing construction environments; ultrasonic sensors are used for detecting close-range obstacles, especially when the visual and lidar sensors are blocked; for example, in areas under the tower crane boom or where the line of sight is blocked, the ultrasonic sensors can detect obstacles within a few meters, such as construction workers or small equipment; the tower crane status monitoring sensors monitor the operating status of the tower crane in real time, including key parameters such as load, height, and amplitude; the data from these sensors are analyzed in real time by a computer system to evaluate the operating safety and stability of the tower crane; for example, when the load of the tower crane exceeds the preset safety threshold, the system will immediately issue an alarm and take corresponding safety measures.

[0029] To ensure the time consistency of all sensor data, a high-precision atomic clock is built into the multi-modal data acquisition module as the time synchronization reference source for the entire system. With extremely high stability and precision, the atomic clock provides time accuracy at the microsecond level. When the system starts, all sensors are connected to the atomic clock through a synchronization interface to receive the globally unified time reference signal. This time reference signal is used by each sensor as the starting time point for data acquisition. During data acquisition, each sensor embeds a timestamp in the collected data packet. The timestamp records the exact time of data acquisition, enabling accurate alignment and fusion of data from different sensors in subsequent data processing. To ensure the accuracy of the timestamp, the system adopts a dynamic time calibration algorithm. This algorithm continuously monitors the time deviation between each sensor and the atomic clock and dynamically adjusts the sensor's timestamp according to the monitoring results. Specifically, the dynamic time calibration algorithm calculates the time deviation by periodically sending time synchronization signals to each sensor and receiving the response signals returned by the sensors. For example, a synchronization signal is sent once per second, and the sensor immediately returns a response signal after receiving the signal. The time deviation of the sensor is determined by calculating the time difference between sending and receiving the signal. If it is found that the timestamp of a certain sensor is 10 microseconds slower than the atomic clock, this deviation value is calculated and subtracted from the timestamp in the data packet collected by the sensor, resulting in a timestamp that is completely synchronized with the atomic clock.

[0030] Reference Figure 2 , the intelligent data fusion and analysis module is the core of the system, responsible for processing and analyzing multi-modal data, calculating the collision risk in real time, and outputting early warning results. This module adopts a multi-level data fusion strategy, constructs a fusion network architecture based on the time-synchronized data, and performs dynamic weighted fusion on multi-modal data. The multi-level data fusion strategy aligns and inputs the dynamic data of tower crane operations, environmental data, and the activity trajectory data of construction site personnel according to the same timestamp. In the parallel processing stage, the tower crane operation data is calibrated and noise-reduced, the environmental data is standardized according to meteorological parameters, and the activity trajectory data of personnel is unified in coordinates and abnormal trajectories are eliminated. The processed data enters the interaction layer, where a cross-modal attention mechanism is used to make the features of each data interact and influence each other. According to the interaction and influence between the data, the data enters the feature weight assignment layer, and based on historical collision cases and real-time data statistics, the weights of each feature are calculated. With the assigned weights, the data enters the fusion layer, and feature fusion is performed by weighted summation, finally forming a fusion data set that comprehensively reflects the operating environment, operation status of the tower crane, and personnel activities.

[0031] Reference Figure 3, The improved decision tree algorithm is a crucial step in achieving accurate assessment of tower crane collision risks. First, the algorithm conducts in-depth analysis of the weather environment data in the fusion dataset to identify the key features that have a direct impact on tower crane collision risks under specific weather conditions. After determining the key features, a weight adjustment mechanism is initiated, which is achieved by analyzing historical collision event data. For each key feature, the proportion of the number of times it appears in all collision events is statistically calculated, as well as the correlation coefficient between the change amplitude of the feature value before and after the collision and the severity of the collision. The statistical data assigns a dynamic weight to each key feature, and the weight value reflects the risk contribution degree of the feature under the current weather conditions. The decision tree with adjusted weights takes the real-time collected tower crane operation data and environmental monitoring data as input. Starting from the root node, the decision tree screens the input data layer by layer according to the key features and their weights. As the data is transmitted in the decision tree, each layer of nodes calculates a local risk score based on the weights of the key features and the actual values of the data. These local scores are accumulated layer by layer in the hierarchical structure of the decision tree, and finally a comprehensive collision risk value is obtained at the leaf node. This risk value is a continuous numerical value between 0 and 1, and the closer the value is to 1, the higher the collision risk.

[0032] The early warning management module is the bridge between the system and the construction site personnel. It is responsible for generating collision warning signals based on real-time warning results and automatically optimizing early warning management through deep learning algorithms. This module receives the collision risk value output from the intelligent data fusion and analysis module through a risk response mechanism, and based on this, constructs a set of risk grading criteria, clearly dividing the risks into three levels: low, medium, and high. For risks of different levels, the system has preset corresponding response strategy templates, covering the content of warning information, the scope of push objects, and recommended countermeasures. The deep learning algorithm conducts in-depth learning and analysis of historical processed data and on-site feedback by constructing a complex neural network model. The data input into the model is multi-dimensional, including tower crane operation data, environmental data, and personnel activity data, etc. Adopting a combined architecture of LSTM and fully connected layers, LSTM captures the long-term dependencies in time series data and understands the temporal correlation between risk level changes and the implementation of countermeasures. The fully connected layer is responsible for integrating and mapping the features output by LSTM, and finally outputs optimized early warning management parameters. During the training process, the model aims to minimize the difference between the predicted risk level and the actual events that occurred, and continuously adjusts the network weights through the backpropagation algorithm. As the training progresses, the deep learning algorithm gradually masters the mapping relationship between risk change rules and the optimal countermeasure plan.

[0033] The adaptive optimization decision-making module conducts correlation analysis on the dynamic data at the construction site, including tower crane operation data, environmental data, and personnel activity trajectory data, and the collision warning signals output by the warning management module; by receiving this data in real time, comprehensively evaluate the safety risks of the construction site; during the risk assessment process, set the collision risk value as the main weight factor and assign a higher weight, and use the remaining data as auxiliary variables and assign lower weights; each variable will be standardized before evaluation to eliminate the influence of different dimensions and data ranges; the standardized variables are multiplied by the corresponding weights and summed up to obtain a comprehensive risk score; the comprehensive risk score is compared with a preset threshold to determine the risk level as low, medium, or high; after determining the risk level, trigger the pre-stored decision rules according to the level; after the system runs, it will collect the real-time effect data after the decision is executed as feedback input; the feedback data is compared with the expected performance indicators to evaluate the effectiveness of the decision; if the feedback shows that the collision risk is not effectively controlled, or the construction progress is adversely affected, it indicates that the current decision needs to be adjusted; according to the adjustment logic corresponding to the risk level, dynamically adjust the parameters in the decision rules.

[0034] Through this multi-modal data fusion tower crane collision warning system, it can evaluate the collision risk of tower cranes in real time and dynamically, providing scientific and accurate decision-making support for the safety management of the construction site; this data-driven risk assessment method not only improves the accuracy and timeliness of early warning, but also can be flexibly adjusted according to real-time environmental conditions to ensure the safe operation of tower cranes under various complex working conditions.

[0035] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A tower crane collision warning system for multi-modal data fusion, characterized in that, Including: Multi-modal data acquisition module: Real-time acquisition of each data source, including dynamic data and environmental data of tower crane operation, and the activity trajectories of construction site personnel. The data sources are synchronized in real time through time synchronization technology to ensure the timeliness and accuracy of data acquisition; Intelligent data fusion and analysis module: Adopting a multi-level data fusion strategy, constructing a fusion network architecture based on the time-synchronized data, and dynamically weighted fusing multi-modal data; Using an improved decision tree algorithm and a dynamic threshold method to calculate the collision risk in real time according to tower crane and environmental data and output early warning results; Among them, the multi-level data fusion strategy aligns and inputs the dynamic data of tower crane operation, environmental data, and the activity trajectory data of construction site personnel according to the same time stamp; In the parallel processing stage, calibrating and denoising the tower crane operation data; The processed data enters the interaction layer, and a cross-modal attention mechanism is used to make the features of each data pay attention to and influence each other; According to the attention and influence between the data, the data enters the feature weight allocation layer, and based on historical collision cases and real-time data statistics, calculates the weights of each feature; With the allocated weights, the data enters the fusion layer, and feature fusion is carried out by means of weighted summation to finally form a fusion data set comprehensively reflecting the tower crane operation environment, operation status, and personnel activities; Early warning management module: Through a risk response mechanism, generating a collision early warning signal according to the real-time early warning result. The collision early warning signal includes collision early warning information, collision risk level assessment, and response suggestions. The early warning management module automatically optimizes the early warning management through a deep learning algorithm, continuously updates the intelligent data fusion and analysis module based on historical processing data and on-site feedback, and pushes the collision early warning signal to relevant personnel; Adaptive optimization decision module: Adopting a decision-making mechanism based on feedback adjustment, associating and analyzing the dynamic data of the construction site with the collision early warning signal, real-time evaluating the safety risk of the construction site, and adjusting the working parameters or scheduling of the tower crane according to the evaluated risk to reduce potential collision risks and improve construction efficiency; The improved decision tree algorithm deeply analyzes the weather environment data in the fusion dataset to identify the key features that have a direct impact on the tower crane collision risk under specific weather conditions; after determining the key features, a weight adjustment mechanism is activated to count the occurrence frequency of the key features during collision and the correlation strength with the collision result; for each key feature, count the proportion of the number of times it appears in all collision events, and the correlation coefficient between the change range of the key feature before and after the collision and the severity of the collision; the statistical data assigns a dynamic weight to each key feature, and the weight value reflects the risk contribution degree of the feature under the current weather conditions; the decision tree after weight adjustment takes the real-time collected tower crane operation data and environmental monitoring data as input, and the decision tree starts from the root node and filters the input data layer by layer according to the key features and their weights; at each node, according to the comparison result of the feature value and the preset threshold, determine the data flow to different child nodes; each layer of nodes calculates a local risk score according to the weight of the key feature and the actual value of the data; these local scores are accumulated layer by layer in the hierarchical structure of the decision tree, and finally a comprehensive collision risk value is obtained at the leaf node.

2. The tower crane collision warning system for multimodal data fusion according to claim 1, wherein: The multi-modal data acquisition module is equipped with a high-resolution visual sensor for capturing image information of the tower crane and its surrounding environment, and real-time identifying the position, posture of the tower crane and the distribution of surrounding obstacles; measuring the distance between the tower crane and surrounding objects through a high-precision lidar to provide three-dimensional space data; an ultrasonic sensor is used for detecting close-range obstacles in the case where the visual and lidar sensors are blocked; the tower crane status monitoring sensor monitors the operation status of the tower crane in real time; adopting time synchronization technology, through the built-in clock synchronization unit, calibrate the timestamps of all sensors and positioning systems, and transmit the collected data to the intelligent data fusion and analysis module.

3. The tower crane collision warning system for multimodal data fusion according to claim 2, characterized in that: The time synchronization technology built in the multi-modal data acquisition module has an atomic clock as the reference source for time synchronization. Through the atomic clock, a globally unified time reference signal is established; when each sensor starts up, it receives the time reference signal through a synchronization interface and uses it as the starting point of its own data acquisition; during the data acquisition process, each sensor embeds a timestamp in the collected data packet. Adopting a dynamic time calibration algorithm, it monitors the time deviation between each sensor and the atomic clock in real time and dynamically adjusts the time point of the sensor according to the monitoring result; the dynamic time calibration algorithm calculates the deviation value of each sensor's time, and then subtracts this deviation value from the time point in the data packet collected by the sensor, so as to obtain a time point that is completely synchronized with the atomic clock.

4. The tower crane collision warning system for multimodal data fusion according to claim 1, characterized in that: The multi-level data fusion strategy aligns the tower crane operation dynamic data, environmental data and personnel activity trajectory data at the same time point and inputs them; processes them in parallel, wherein the dynamic data of the tower crane operation is collected by sensors and calibrated and denoised; the environmental data is standardized by meteorological parameters; the activity trajectory data of the construction site personnel completes coordinate unification and abnormal trajectory elimination; the data after parallel management enters the interaction layer, and the cross-modal attention mechanism is used to make the features of each data pay attention to and influence each other; according to the attention and influence between the data, it enters the feature weight allocation layer, and calculates the weight of each feature based on historical collision cases and real-time data statistics; if the historical data shows that a specific environmental factor is highly correlated with the collision, its weight is increased accordingly; if the personnel activities are abnormally frequent in real-time monitoring, the weight of the personnel trajectory feature is increased; With the assigned weights, the data enters the fusion layer and features are fused using a weighted summation method. The weighted summation allows each feature to be fused according to its weighted contribution, ultimately forming a fused data set that comprehensively reflects the tower crane's operating environment, operating status, and personnel activities.

5. A tower crane collision warning system for multimodal data fusion according to claim 4, characterized in that: Analyze the weather environment data in the fused dataset to identify key features that directly affect collisions under specific weather conditions; Key features are selected based on real-time weather dynamics, allowing the decision tree to focus on the real-time environment. After the key features are determined, the weight adjustment mechanism is immediately activated. By analyzing historical collision event data, the frequency of occurrence of key features when collisions occur and the strength of correlation with collision results are statistically analyzed. For each key feature, the proportion of its occurrence in all collision events is calculated, as well as the correlation coefficient between the change in the data before and after the collision and the severity of the collision. Based on statistical data, a dynamic weight is assigned to each key feature, and the weight value reflects the risk contribution of the feature under current weather conditions. The decision tree after adjusting the weights takes the tower crane operation data and environmental monitoring data collected in real time as input. Starting from the root node, the decision tree judges and screens the input data layer by layer according to the key features and their weights. At each node, the data flows to different child nodes based on the comparison results between the feature value and the preset threshold. According to the transmission of data in the decision tree, each layer of nodes calculates a local risk score based on the weight of the key features and the actual value of the data. The local score is accumulated layer by layer in the hierarchical structure of the decision tree, and finally a collision risk value is obtained at the leaf node. The risk value is a continuous value between 0 and 1. The closer the value is to 1, the higher the collision risk.

6. The tower crane collision warning system for multimodal data fusion according to claim 1, characterized in that: The risk response mechanism receives the collision risk value output by the intelligent data fusion and analysis module, builds a risk classification standard, and presets corresponding response strategy templates for different levels of risk; the deep learning algorithm learns and analyzes historical processing data and on-site feedback by building a neural network model; Input multi-dimensional data, and adopt the combination of LSTM and fully connected layers. The LSTM layer is responsible for capturing the long-term dependencies in time series data and understanding the temporal correlation between the changes in risk levels and the implementation of countermeasures; the fully connected layer integrates and maps the features output by the LSTM, and outputs the optimized early warning management strategy parameters; During the training process, with the goal of minimizing the difference between the predicted risk level and the actual events that occur, continuously adjust the network weights; With the deepening of learning, the deep learning algorithm gradually masters the mapping relationship between the risk change law and the optimal countermeasure plan, so as to realize the automatic optimization of early warning management.

7. The tower crane collision warning system for multimodal data fusion according to claim 6, wherein: The adaptive optimization decision-making module correlates and analyzes the dynamic data of the construction site with the collision warning signal; according to the dynamic data of tower crane operation, environmental data and personnel activity trajectory data received in real time, and combining with the collision warning signal of the early warning management module, evaluate the safety risk of the construction site; the risk assessment is based on the comprehensive analysis of real-time data and collision warning signals, where the collision risk value is set as the main weight factor and assigned a high weight, and the remaining data are auxiliary variables and assigned a low weight, and the total weight is 100%; After standardizing each variable, multiply it by the weight and sum them up to obtain the comprehensive risk score; compare the score with the preset threshold to determine the low, medium and high risk levels; After determining the risk level, trigger the decision-making rules according to the level; pre-store the corresponding strategies for each level. After the system runs, collect the real-time effect data after the decision is executed, and use it as the feedback input to compare with the expected performance indicators; if the feedback shows that the collision risk has not been effectively controlled or the construction progress is affected, it indicates that the current decision needs to be adjusted.

Images