Intelligent safety monitoring method and device for bridge crane based on multi-source data fusion

Abstract

The embodiment of the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion. By integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors, multi-dimensional collection of weight, image, position and broken wire data is achieved. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of suspended objects, broken wire ropes and hook trajectory deviation, achieving all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capabilities of traditional single monitoring solutions and improves the accuracy of safety warnings.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion. Background Art

[0002] With the development of intelligent manufacturing, bridge crane safety monitoring technology faces new challenges and requirements. Traditional monitoring methods rely primarily on single sensor data, lack a systematic safety assessment mechanism, and are unable to cope with complex operating environments and diverse safety risks.

[0003] Existing systems generally suffer from insufficient sensing capabilities. Single-source weight or position monitoring makes it difficult to fully understand the crane's operating status, particularly in areas such as personnel safety and wire rope status monitoring. Furthermore, traditional image monitoring systems struggle with complex lighting conditions, making it difficult to accurately detect personnel in hazardous areas.

[0004] From a data processing perspective, existing solutions lack effective multi-source data fusion mechanisms. Data collected by different sensors is often processed independently, failing to fully leverage inter-data correlations. This leads to incomplete or delayed monitoring results. Furthermore, the system lacks sufficient consideration for data collection frequency and synchronization, impacting the timeliness and accuracy of early warnings.

[0005] From a safety assessment perspective, traditional systems often rely on fixed thresholds and lack intelligent risk assessment capabilities. This simplistic approach is inadequate for adapting to diverse operating conditions and is prone to false positives and omissions. This is especially true when multiple risk factors coexist, making it difficult for the system to make accurate safety assessments.

[0006] Therefore, building an intelligent monitoring system that integrates multi-source data and improves the system's perception capabilities and early warning accuracy is a key issue that needs to be addressed in current bridge crane safety monitoring technology. This not only affects operational safety but also forms an important foundation for intelligent safety management. Summary of the Invention

[0007] In response to the problems in the existing technology, the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion, which can effectively improve the system's perception ability and warning accuracy.

[0008] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0009] In a first aspect, the present application provides an intelligent safety monitoring method for bridge cranes based on multi-source data fusion, comprising:

[0010] Heterogeneous sensors are installed on the bridge crane to collect weight data, image data, position data, and broken wire data, and the weight data, image data, position data, and broken wire data are written into a multidimensional data cache matrix according to timestamps;

[0011] Dividing the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism;

[0012] The preprocessed data is read from the distributed database, and the image data stream is input into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on a network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through a fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on a set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0013] Furthermore, the noise filtering and outlier correction of the position data stream, load data stream, image data stream and broken wire data stream are performed, and the pre-processed data are written into a distributed database based on a time synchronization mechanism, including:

[0014] Establishing a four-dimensional data cache form in the bridge crane site controller, adding a collection time mark to the weight data, image data, position data, and broken wire data, and writing the data into the corresponding time axis position in the four-dimensional data cache form according to the collection time mark, wherein the four-dimensional data cache form includes a time dimension, a sensor type dimension, a data attribute dimension, and a value dimension;

[0015] A feature classification operation is performed on the four-dimensional data cache form, and the functional characteristics of the sampled data are extracted from the data attribute dimension through a sliding window algorithm. The functional characteristics are grouped using a clustering analysis method, and data with the same functional attributes are organized into independent data streams to generate data structure models of position data stream, load data stream, image data stream and broken wire data stream.

[0016] Furthermore, the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, including:

[0017] A multi-channel data acquisition module is configured in the field control unit of the bridge crane. The sampling clock frequency of the position data stream and the load data stream is set to 50 Hz, and data is collected at 20 millisecond intervals through a timed trigger mechanism. The sampling clock frequency of the image data stream is set to 30 Hz, and data is collected at 33 millisecond intervals through a timed trigger mechanism. The buffer capacity of the data stream is calculated based on the sampling clock frequency.

[0018] Wavelet transform is applied to the data stream to remove high-frequency noise, a sliding median filter is used to eliminate outliers, an anomaly detection model is established based on the statistical characteristics of the data, data outside the confidence interval is marked as outliers and interpolation correction is performed, a timestamp alignment algorithm is used to correct the sampling delay between different data streams, and the preprocessed data is written into the corresponding table items of the distributed database in time series.

[0019] Furthermore, the pre-processed data is read from the distributed database, and the image data stream is input into a convolutional neural network for person detection. The convolutional neural network includes an input layer, a feature extraction layer, and an output layer. The feature extraction layer determines the optimal residual block configuration through an adaptive search algorithm, including:

[0020] Constructing a distributed data reading module, dividing the distributed database into a read and write buffer, establishing a data index table in the read buffer, retrieving the storage address of the target data block from the data index table according to the timestamp and data type identifier, and loading the target data block into the video memory buffer of the neural network computing unit in a batch processing manner;

[0021] A basic convolutional layer is constructed in the neural network computing unit, and the image data stream is input into the basic convolutional layer after data normalization. A candidate residual block configuration pool is constructed, and the candidate residual block configuration pool contains a set of residual blocks with different number combinations. A network performance evaluation function is used as the objective function of the adaptive search, and the candidate residual block configuration is iteratively optimized based on the gradient descent method. The residual block number combination with the optimal objective function value is selected to construct a feature extraction layer.

[0022] Furthermore, the iterative calculation of the objective function values ​​for different combinations of the number of residual blocks on the training set includes:

[0023] Constructing a neural network training module, setting the search range of the first group of residual blocks to 2 to 4, the search range of the second group of residual blocks to 3 to 5, the search range of the third group of residual blocks to 5 to 7, and the search range of the fourth group of residual blocks to 2 to 4, generating a combination matrix for the number of residual blocks within the search range, inputting each residual block number configuration in the combination matrix into the objective function calculation unit in sequence, and calculating the value of the objective function through verification of the training set;

[0024] A three-layer convolution structure is constructed for each residual block. The first layer is a dimensionality reduction convolution layer with a 1×1 convolution kernel, which is used to reduce the number of feature map channels. The second layer is a standard convolution layer with a 3×3 convolution kernel, which is used to extract spatial features. The third layer is a dimensionality increase convolution layer with a 1×1 convolution kernel, which is used to restore the number of feature map channels. A batch normalization layer and an activation function layer are added between the three-layer convolution structure. The combination of the number of residual blocks with the optimal objective function value is determined as the final structure of the feature extraction layer.

[0025] Furthermore, the output layer outputs the coordinates of the personnel position in the dangerous area through the fully connected layer, detects the overweight state of the hanging object in the load data stream based on the set threshold, and calculates the cumulative value of the number of broken wires in the broken wire data stream, including:

[0026] A feature vector fusion module is constructed at the end of the neural network to flatten the feature map output by the feature extraction layer into a one-dimensional vector. The one-dimensional vector is mapped into the position coordinates and confidence scores of the target detection box through a three-layer fully connected network. The number of nodes in the first layer of the fully connected network is 1024, the number of nodes in the second layer is 512, and the number of nodes in the third layer is 256. The coordinates of the detection box with the highest confidence are screened out using a non-maximum suppression algorithm.

[0027] The load data stream and the broken wire data stream are read from the distributed database, the load data stream is compared with the preset safe load threshold, a weight status mark is generated, the broken wire signals in the broken wire data stream are accumulated and counted, a broken wire quantity statistical table is established, the weight status mark and the broken wire quantity statistical table are stored in the status monitoring database, and the equipment operation status is judged according to the safety level threshold.

[0028] Furthermore, the method of combining the position data stream to determine whether the hook motion trajectory exceeds the safety boundary and generating a control instruction based on the detection result and sending it to the on-site execution system includes:

[0029] Construct a trajectory analysis module to read the three-dimensional coordinate data of the hook in the position data stream, calculate the hook motion trajectory curve based on the cubic spline interpolation algorithm, perform real-time collision detection between the motion trajectory curve and the boundary surface of the safe working area, calculate the minimum distance between the trajectory curve and the boundary surface through the vector projection method, and generate the hook position status mark;

[0030] A multi-source information decision module is established, and the personnel detection results, weight status mark, broken wire quantity statistics and hook position status mark are input into the status evaluation function. An execution instruction code is generated according to the safety assessment rules, and the execution instruction code is sent to the execution unit of the crane controller through the fieldbus network, wherein the execution unit includes a motor driver, a brake and an alarm device.

[0031] In a second aspect, the present application provides an intelligent safety monitoring device for a bridge crane based on multi-source data fusion, comprising:

[0032] A data acquisition module is used to install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps;

[0033] a data preprocessing module, configured to divide the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes, perform noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and write the preprocessed data into a distributed database based on a time synchronization mechanism;

[0034] A fusion monitoring module is used to read preprocessed data from the distributed database and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the broken wire number in the broken wire data stream, and determines whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream, and generates control instructions based on the detection results and sends them to the on-site execution system.

[0035] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion are implemented.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion.

[0037] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion.

[0038] As can be seen from the above technical solution, the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion, which realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of hoisted objects, broken wire ropes and hook trajectory deviation, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capabilities of traditional single monitoring solutions and improves the accuracy of safety warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0040] Figure 1 This is one of the flow charts of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in an embodiment of the present application;

[0041] Figure 2 This is a second flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in an embodiment of the present application;

[0042] Figure 3 This is the third flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application;

[0043] Figure 4 This is a fourth flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in an embodiment of the present application;

[0044] Figure 5 This is a fifth flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in an embodiment of the present application;

[0045] Figure 6 This is a sixth flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in an embodiment of the present application;

[0046] Figure 7 This is the seventh flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application;

[0047] Figure 8This is a structural diagram of an intelligent safety monitoring device for a bridge crane based on multi-source data fusion in an embodiment of the present application;

[0048] Figure 9 Schematic diagram of the structure of the electronic device in the embodiment of the present application.

[0049] Reference numerals:

[0050] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0051] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0052] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0053] Taking into account the problems existing in the prior art, the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion, which realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of hoisted objects, broken wire ropes and hook trajectory deviation, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0054] In order to effectively improve the system's perception capability and warning accuracy, this application provides an embodiment of a bridge crane intelligent safety monitoring method based on multi-source data fusion, see Figure 1 The intelligent safety monitoring method for bridge cranes based on multi-source data fusion specifically includes the following contents:

[0055] Step S101: Installing heterogeneous sensors on a bridge crane to collect weight data, image data, position data, and broken wire data, and writing the weight data, image data, position data, and broken wire data into a multidimensional data cache matrix according to timestamps;

[0056] Optionally, in an embodiment of the present application, a lateral pressure sensor may be installed at the fixed end of the wire rope of the mobile trolley of the bridge crane to collect weight data, an intelligent camera may be installed on the top of the mobile trolley to collect images of the working area, a displacement sensor may be installed at the end of the mobile trolley and hook running track to collect spatial position data, and an electromagnetic sensor may be installed on the hoisting system to collect broken wire data. The weight data, image data, position data, and broken wire data may be written into a multidimensional data cache matrix according to timestamps.

[0057] Optionally, this embodiment deploys multiple sensors at key locations on the bridge crane, creating a comprehensive data acquisition network. The lateral pressure sensor installed at the fixed end of the mobile trolley's wire rope utilizes a high-precision strain gauge structure, converting wire rope tension into an electrical signal for real-time monitoring of the load weight. The sensor utilizes the principle of elastic deformation and uses a compensation circuit to eliminate the effects of temperature drift, ensuring the stability of the measured data.

[0058] Mounted atop a mobile cart, the intelligent camera utilizes a wide-angle lens to provide full coverage of the work area. Equipped with automatic exposure and a wide dynamic range, the camera maintains image quality in varying indoor and outdoor lighting environments. In low-light conditions, an adaptive gain algorithm enhances image brightness, ensuring continuous monitoring of hazardous areas.

[0059] The displacement sensors utilize photoelectric encoders, mounted at the ends of the trolley and hook tracks. The encoders' output pulse signals are used to calculate the spatial coordinates of the trolley and hook. To minimize mechanical vibration, the sensors utilize a special vibration-damping mounting structure, enhancing the accuracy of position data.

[0060] The electromagnetic sensor in the hoisting system detects wire rope breaks using the magnetic flux leakage detection principle. The sensor coil generates an alternating magnetic field. When a wire rope breaks, the local change in magnetic permeability causes an induced voltage fluctuation. The signal processing circuit then identifies the broken wire.

[0061] The data acquisition system utilizes a distributed architecture, with each sensor equipped with an independent data acquisition module that transmits data to the central processing unit via industrial Ethernet. The acquisition modules incorporate high-speed A / D converters for synchronous data sampling. Time synchronization utilizes the IEEE 1588 protocol, ensuring microsecond-level clock deviation at each acquisition point.

[0062] The multidimensional data cache matrix utilizes a shared memory structure, supporting concurrent multi-threaded access. The matrix organizes data by timestamp, data type, sampling frequency, and other dimensions, facilitating subsequent data analysis and processing. Cache management utilizes a circular queue mechanism, automatically clearing the oldest data records when the data volume exceeds a preset threshold.

[0063] The data writing process uses a double buffering mechanism to avoid data read and write conflicts. Newly acquired data is first written to the buffer and then transferred to the main storage area after data integrity verification. This mechanism effectively prevents data loss and write errors.

[0064] This embodiment establishes a holographic perception network for the operating status of a bridge crane through the collaborative operation of multiple sensors. The rational sensor layout and reliable data collection mechanism lay a solid foundation for subsequent safety monitoring and intelligent control. This solution effectively addresses the problems of large blind spots and poor data synchronization associated with traditional single-sensor monitoring, improving the reliability and real-time performance of the monitoring system.

[0065] In practical applications, this embodiment can rapidly respond to various hazardous working conditions. For example, when a crane is lifting large objects, the coordinated use of pressure sensors and displacement sensors can accurately monitor the weight and trajectory of the objects, promptly detecting abnormalities such as overload or trajectory deviation. Simultaneously, the image monitoring system tracks human activity in the work area in real time, effectively preventing collisions between humans and machines.

[0066] Step S102: Dividing the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream, and broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism;

[0067] Optionally, in an embodiment of the present application, the data in the multidimensional data cache matrix may be divided into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes, a 50 Hz sampling frequency is set for the position data stream and the load data stream, a 30 Hz sampling frequency is set for the image data stream, noise filtering and outlier correction are performed on the data streams, and the preprocessed data is written to a distributed database based on a time synchronization mechanism;

[0068] Optionally, this embodiment first conducts an in-depth analysis of the raw data in the multidimensional data cache matrix. Through the study of data characteristics, it is found that different types of data have their own unique time domain characteristics and frequency domain characteristics. For example, position data is manifested as continuous changes in spatial coordinates, load data exhibits characteristics that fluctuate with the lifting process, image data contains rich visual information, and broken wire data is manifested as discrete state quantity changes. Based on these feature differences, a data classification algorithm based on cluster analysis is designed to accurately divide the data into four independent data streams.

[0069] Regarding the sampling frequency setting, this embodiment determined the optimal sampling parameters through extensive experimentation and theoretical analysis. For the position data stream, based on the maximum operating speed and acceleration characteristics of the bridge crane, it was deduced that the highest frequency component of position changes is approximately 25 Hz. To avoid aliasing distortion, the sampling frequency of the position data was set to 50 Hz, in accordance with the Nyquist sampling theorem. The load data also uses a 50 Hz sampling frequency because the transient load changes and mechanical vibration frequencies during the lifting process typically do not exceed 25 Hz, and this sampling frequency is sufficient to fully record the load changes.

[0070] The adoption of a 30Hz sampling frequency for the image data stream is the result of careful consideration. The human eye can track moving objects at approximately 24 frames per second. A 30Hz sampling frequency not only meets the visual requirements of the human eye but also allows ample computing time for image processing. In practical applications, this frequency accurately captures rapid motion within the operating area while balancing system resource usage.

[0071] Data preprocessing utilizes a multi-level, progressive approach. The first layer involves noise filtering. For the position data stream, a wavelet transform is applied to decompose the signal into frequency bands. A soft threshold function is then used to process the high-frequency noise coefficient, and the resulting signal is reconstructed to reduce the noise. Load data is filtered using an adaptive bandpass filter. The filter's center frequency is dynamically adjusted based on the crane's current operating status, preserving the valid signal while suppressing interference.

[0072] The second layer is outlier correction. This embodiment innovatively proposes an adaptive thresholding method based on a sliding window. By calculating the mean and standard deviation within a local data window, a dynamically changing confidence interval is established. For data points outside the confidence interval, the system uses cubic spline interpolation, combined with physical model constraints, to correct them. This method maintains data continuity while ensuring that the correction results conform to physical laws.

[0073] Time synchronization is a critical component of data processing. This embodiment employs a hierarchical time synchronization scheme, with the master controller acting as a time server and providing a unified time base to each sensor node via the Precision Time Protocol (PTP). During data stream processing, a timestamp alignment algorithm aligns data streams with different sampling frequencies to the same time base, ensuring data temporal consistency.

[0074] The distributed database uses a tiered storage strategy for writing data. High-frequency data, such as location and load data, is first written to the cache layer and then stored in batches before being stored in the persistent storage layer. Image data is stored incrementally, saving only the differences between adjacent frames, significantly reducing storage space usage. The data writing process utilizes a double-buffering mechanism, alternating between the write buffer and the persistent storage area to avoid data congestion.

[0075] This embodiment achieves high-quality data collection and storage through a carefully designed data processing process. In practical applications, this solution can provide stable and reliable condition monitoring data when a crane performs precision lifting operations. For example, when lifting critical equipment, the system can accurately record every tiny position change and load fluctuation, providing operators with accurate decision-making basis. Furthermore, the complete data record provides reliable data support for subsequent equipment maintenance and fault diagnosis.

[0076] Step S103: Read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the hook movement trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0077] Optionally, the embodiment of the present application can read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection. The convolutional neural network includes an input layer, a feature extraction layer, and an output layer. The feature extraction layer determines the optimal residual block configuration through an adaptive search algorithm. The adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set. The search range of the residual block number combination is 2-4 for the first group, 3-5 for the second group, and 5-7 for the third group. The fourth group has 2-4, and the combination of residual blocks with the optimal objective function value is selected as the final network structure. Each residual block contains a 1×1 dimensionality reduction convolution layer, a 3×3 standard convolution layer and a 1×1 dimensionality increase convolution layer. The output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the weight of the hanging object in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and judges whether the hook movement trajectory exceeds the safety boundary in combination with the position data stream. According to the detection results, a control instruction is generated and sent to the on-site execution system.

[0078] Optionally, this embodiment designs a complete data analysis and decision-making system, the core of which is a deep learning-based personnel detection module and a multi-source data fusion security monitoring mechanism. First, when reading pre-processed data from the distributed database, a parallel reading strategy is adopted, and unified scheduling is performed through the data cache pool to ensure the real-time and complete data reading.

[0079] For personnel detection, this embodiment innovatively designs a convolutional neural network with an adaptive architecture. The input layer uses a multi-scale image pyramid structure to scale the original image to varying proportions, enhancing the network's ability to detect objects of varying sizes. In practical applications, this architecture can simultaneously detect workers both near and far, improving detection reliability.

[0080] The feature extraction layer is designed using a residual learning framework, optimizing the network structure through an adaptive search algorithm. The objective function comprehensively considers detection accuracy, computational complexity, and model size, taking the form of a weighted combination of evaluation metrics. During the search process, the algorithm first divides the residual blocks into four groups based on their function, each responsible for extracting features at a different level. The first group extracts low-level edge features, searching for 2-4 residual blocks; the second group extracts mid-level texture features, searching for 3-5 residual blocks; the third group extracts high-level semantic features, searching for 5-7 residual blocks; and the fourth group integrates features, searching for 2-4 residual blocks.

[0081] Each residual block adopts a "bottleneck" architecture, consisting of three convolutional layers. A 1×1 dimensionality reduction convolution layer reduces the number of feature channels, reducing computational effort; a 3×3 standard convolution layer performs primary feature extraction; and a 1×1 dimensionality increase convolution layer restores the number of feature channels, preserving information integrity. This architecture significantly reduces computational overhead while maintaining network performance.

[0082] During the adaptive search process, a gradient-based iterative optimization strategy is employed. In each iteration, the algorithm randomly selects a set of residual block configurations and evaluates network performance on the training set. To improve search efficiency, an early stopping strategy is employed. If the initial training performance of a configuration is significantly poor, the evaluation of that configuration is terminated prematurely.

[0083] The output layer generates the precise coordinates of the personnel within the danger zone through a fully connected layer. To improve positioning accuracy, a multi-task learning framework is employed to simultaneously predict the personnel's bounding box and keypoint locations. This design enables more accurate determination of the positional relationship between the personnel and the danger zone.

[0084] In terms of multi-source data fusion, this embodiment has designed a hierarchical safety monitoring mechanism. The load monitoring module uses an adaptive threshold strategy to dynamically adjust the weight limit based on the load type and operating conditions. Broken wire monitoring uses a cumulative statistical method combined with a life prediction model to assess the safety status of the wire rope. The trajectory monitoring module constructs a three-dimensional safety envelope to determine in real time whether the hook movement poses a risk of crossing the boundary.

[0085] This embodiment achieves intelligent safety monitoring of bridge crane operations through deep learning and multi-source data analysis. In practical applications, when a dangerous situation is detected, the system can immediately generate appropriate control instructions. For example, if a person enters a dangerous area, the system will generate a deceleration or shutdown command based on the person's location and hook motion status. If overload or trajectory anomalies are detected, the system will trigger appropriate safety protection mechanisms. This intelligent monitoring approach significantly improves the safety and reliability of crane operations.

[0086] From the above description, it can be seen that the intelligent safety monitoring method for bridge cranes based on multi-source data fusion provided by the embodiment of the present application can realize multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of suspended objects, broken wire ropes and hook trajectory deviation, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capabilities of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0087] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 2 , and can also include the following:

[0088] Step S201: establishing a four-dimensional data cache table in the bridge crane site controller, adding a collection time mark to the weight data, image data, position data, and broken wire data, and writing the data into the corresponding time axis position in the four-dimensional data cache table according to the collection time mark. The four-dimensional data cache table includes a time dimension, a sensor type dimension, a data attribute dimension, and a value dimension;

[0089] Step S202: perform feature classification operations on the four-dimensional data cache form, extract functional features of the sampled data from the data attribute dimension through a sliding window algorithm, group the functional features using a clustering analysis method, organize data with the same functional attributes into independent data streams, and generate data structure models of position data stream, load data stream, image data stream and broken wire data stream.

[0090] Optionally, this embodiment designs an efficient data caching and processing solution to address the collection and organization of multi-source, heterogeneous data from bridge cranes. First, a four-dimensional data cache table is constructed in the field controller, employing a multidimensional array storage structure to achieve efficient data access and management. The time dimension uses a unified timestamp format with millisecond accuracy to ensure data temporal consistency. The sensor type dimension records data source information, including sensor number and type identifier. The data attribute dimension stores characteristic descriptions of the sampled data. The value dimension stores the actual measured values.

[0091] The addition of the acquisition time stamp utilizes a high-precision clock synchronization mechanism. This embodiment deploys a clock synchronization module in each sensor node, synchronizing it with the master clock via the IEEE 1588 Precision Time Protocol. Clock deviation is controlled to the microsecond level. During data collection, the current timestamp is written to the packet header as a marker, ensuring precise temporal location of the data.

[0092] The data writing process utilizes a dual-buffer mechanism, with a write buffer and a storage buffer. While one buffer is writing data, the other performs data storage operations, improving data throughput through parallel processing. During the writing process, data is mapped to the corresponding position in a four-dimensional table based on the timestamp, and an index structure is used to accelerate data location.

[0093] To effectively manage and utilize cached data, this embodiment designs a feature extraction algorithm based on a sliding window. The window size is dynamically adjusted based on the temporal characteristics of different data types. For example, a smaller time window is used for rapidly changing position data, while a larger time window is used for slowly changing broken wire data. Within each window, the algorithm extracts multiple statistical features, including mean, variance, peak value, and frequency components, to form a feature vector.

[0094] The feature classification operation uses a two-stage processing strategy. In the first stage, principal component analysis is used to reduce the dimensionality of the feature vectors, retaining the main feature components. In the second stage, a modified K-means clustering algorithm is used to cluster data with similar functional attributes into the same group by calculating the Euclidean distance between feature vectors. Inertia weighting is introduced during the clustering process to ensure that data with similar time periods are more easily classified into the same category.

[0095] Based on the clustering results, this embodiment constructs structural models for four data streams. The position data stream uses a time-series linked list structure to support rapid trajectory reconstruction; the payload data stream uses a circular buffer structure to facilitate real-time monitoring and trend analysis; the image data stream uses a frame sequence structure to support continuous processing of video streams; and the broken wire data stream uses a state record table structure, suitable for statistical analysis of discrete events.

[0096] In practical applications, this embodiment enables real-time data collection and efficient organization of multi-source data when bridge cranes perform multi-point coordinated lifting operations. For example, when hoisting large equipment, the system simultaneously records the position and load data of multiple lifting points. This unified information management is achieved through a data flow model, providing data support for collaborative control. Furthermore, this complete data organization structure provides a reliable data foundation for subsequent condition monitoring and fault diagnosis.

[0097] This data management solution significantly improves data processing efficiency and reduces storage space usage, providing strong support for intelligent control and safety monitoring of overhead cranes. Especially under complex operating conditions, the system can quickly respond to data requests and promptly detect abnormal conditions, ensuring the safety and reliability of crane operations.

[0098] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 3 , and can also include the following:

[0099] Step S301: A multi-channel data acquisition module is configured in the field control unit of the overhead crane. The sampling clock frequency of the position data stream and the load data stream is set to 50 Hz, and data is collected at 20 millisecond intervals through a timed trigger mechanism. The sampling clock frequency of the image data stream is set to 30 Hz, and data is collected at 33 millisecond intervals through a timed trigger mechanism. The buffer capacity of the data stream is calculated based on the sampling clock frequency.

[0100] Step S302: Apply wavelet transform to the data stream to remove high-frequency noise, use a sliding median filter to eliminate outliers, establish an anomaly detection model based on the statistical characteristics of the data, mark data outside the confidence interval as outliers and perform interpolation correction, use a timestamp alignment algorithm to correct the sampling delay between different data streams, and write the preprocessed data into the corresponding table entry of the distributed database in time series.

[0101] Optionally, this embodiment designs a precise data acquisition and preprocessing solution, focusing on addressing the synchronous acquisition and data quality optimization of multi-source heterogeneous data. In terms of hardware configuration, a multi-channel data acquisition module is used, with each channel equipped with independent signal conditioning circuits and analog-to-digital converters to ensure parallel and real-time data acquisition.

[0102] The sampling frequency was set based on in-depth theoretical analysis and experimental verification. For the position and load data streams, considering the dynamic characteristics of the bridge crane, by analyzing the spectral characteristics of its mechanical motion and load changes, it was determined that the main frequency components of the signals do not exceed 25Hz. Based on the sampling theorem, a sampling frequency of 50Hz was selected to fully preserve the signal characteristics while avoiding the waste of resources caused by oversampling. The image data stream uses a sampling frequency of 30Hz, which is the optimal choice after balancing image quality, storage space, and processing power, and can clearly capture dynamic scenes within the working area.

[0103] The timing trigger mechanism is implemented using a high-precision hardware timer. This embodiment configures an independent timer for each sampling channel, precisely controlling the sampling interval through interrupts. The 20-millisecond sampling interval for position and load data is set using the timer's frequency division factor, ensuring sampling stability. The 33-millisecond interval for image acquisition is also hardware-timed, combined with the image sensor's exposure synchronization signal, to achieve precise image acquisition.

[0104] Cache capacity calculation takes into account the characteristics of the data stream and system response requirements. This embodiment uses a two-tier cache structure. The first-level cache is used for real-time data acquisition, with a capacity set to twice the number of sampling points of the sampling frequency. The second-level cache is used for data preprocessing, with a capacity determined by processing latency and data throughput. For example, for a data stream sampled at 50Hz, the first-level cache can store data from 100 sampling points, ensuring that the acquisition process does not lose data due to processing delays.

[0105] Data preprocessing utilizes a multi-level optimization strategy. First, noise reduction is performed through wavelet transform. This example uses the db4 wavelet basis function to perform a five-layer decomposition of the signal. High-frequency coefficients are processed using a soft thresholding method, effectively removing measurement noise while preserving the signal's mutation characteristics. This is particularly important for detecting critical events such as crane emergency braking.

[0106] A sliding median filter is used to eliminate outliers, and the window size is dynamically adjusted based on the data type. For position data, a 5-point window is used to fully preserve the continuity of the motion trajectory; for load data, a 7-point window is used to better suppress interference caused by load fluctuations.

[0107] The anomaly detection model is built based on the statistical characteristics of the data. This example establishes an adaptive confidence interval by calculating the mean and standard deviation within a sliding time window. Data points outside the confidence interval are corrected using cubic spline interpolation, ensuring that the corrected data conforms to physical laws while maintaining temporal continuity.

[0108] The timestamp alignment algorithm solves the synchronization problem between data streams with different sampling frequencies. This embodiment uses linear interpolation to align all data streams to a unified time base. For example, when aligning 30Hz image data with 50Hz position data, interpolation is used to calculate the precise position data corresponding to the image acquisition moment, ensuring the timing consistency of multi-source data.

[0109] During database writes, a batch write strategy is employed, organizing pre-processed data by time period and ensuring write atomicity through a transaction mechanism. Data tables utilize a time-partitioned structure, facilitating historical data queries while improving access efficiency to current data.

[0110] This embodiment significantly improves data quality in practical applications. For example, when hoisting precision equipment, high-quality position and load data provide a reliable basis for precise control, while the noise-reduced image data provides clear visual feedback for monitoring the work area, ultimately ensuring safe and efficient operation of the bridge crane.

[0111] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 4, and can also include the following:

[0112] Step S401: constructing a distributed data reading module, dividing the distributed database into a read and write buffer, establishing a data index table in the read buffer, retrieving the storage address of the target data block from the data index table according to the timestamp and data type identifier, and loading the target data block into the video memory buffer of the neural network computing unit in a batch processing manner;

[0113] Step S402: Build a basic convolution layer in the neural network computing unit, input the image data stream into the basic convolution layer after data normalization, build a candidate residual block configuration pool, the candidate residual block configuration pool contains residual block sets of different quantity combinations, use the network performance evaluation function as the objective function of the adaptive search, iteratively optimize the candidate residual block configuration based on the gradient descent method, and select the residual block quantity combination with the optimal objective function value to build a feature extraction layer.

[0114] Optionally, this embodiment proposes an efficient distributed data processing and deep learning model building solution for intelligent monitoring scenarios of bridge cranes, enabling rapid access and feature extraction of massive amounts of data. First, a partitioning management strategy is employed during data access, dividing the distributed database into independent read and write buffers to prevent interference between read and write operations. The read buffer utilizes a multi-level index structure, encompassing both time and data type indexes, to enable rapid data location.

[0115] The data index table is implemented using a B+ tree structure, with leaf nodes storing the physical addresses of actual data blocks and non-leaf nodes storing index key values. The timestamp serves as the primary index key, and the data type identifier serves as the secondary index key, forming a composite index structure. During retrieval, the timestamp is used to locate the corresponding time partition, followed by a precise search within the partition based on the data type identifier, significantly reducing the time complexity of data retrieval.

[0116] Batch data loading uses a pre-read mechanism to pre-load data blocks that may be used later into the GPU memory buffer based on the timing characteristics of data access. This embodiment designs an adaptive batch resizing strategy that dynamically adjusts the size of each batch by monitoring GPU memory usage and computational load, ensuring computational efficiency while avoiding GPU memory overflow.

[0117] In terms of neural network design, this embodiment adopts a modular network structure. The basic convolutional layer uses a standard 3×3 convolution kernel, combined with a batch normalization layer and ReLU activation function to extract low-level image features. Data normalization uses a batch normalization method to scale image data to the range [-1, 1], improving model training stability.

[0118] The design of the candidate residual block configuration pool reflects the innovation of this embodiment. Each residual block contains two convolutional layers and a shortcut connection. A feature extraction layer is formed by combining different numbers of residual blocks. The configuration pool includes multiple combinations, such as [2,2,2] and [3,4,6] depth configurations, each corresponding to different feature extraction capabilities and computational complexity.

[0119] The network performance evaluation function comprehensively considers multiple metrics, including feature extraction accuracy, computational latency, and resource consumption. This evaluation function uses a weighted summation approach, with weight coefficients optimized using experimental data. In crane operation scenarios, real-time feature extraction and accuracy are particularly important, so these two metrics are given higher weights in the evaluation function.

[0120] The adaptive search process uses a modified gradient descent algorithm. First, an initial configuration is randomly selected and the objective function value is calculated. A local search is then performed within the configuration space, generating neighborhood solutions by adding or removing residual blocks, and the objective function value of the new configuration is calculated. If the new configuration is superior, it is accepted; otherwise, a suboptimal solution is accepted with a certain probability to avoid being trapped in a local optimum.

[0121] This embodiment demonstrates excellent performance in practical applications. For example, when real-time monitoring of image data from multiple cameras is required during crane hoisting operations, the distributed data reading module can rapidly respond to data requests, ensuring a continuous data supply. The optimized neural network model accurately identifies key targets within the operating area, such as the position and posture of hoisted objects, providing reliable visual perception support for intelligent crane control.

[0122] This design significantly improves data processing efficiency and feature extraction accuracy. The system demonstrates excellent adaptability and reliability, particularly in complex operating conditions, such as multi-machine collaborative operations or monitoring operations in adverse weather conditions. This provides a strong foundation for the intelligent and safe operation of bridge cranes.

[0123] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 5 , and can also include the following:

[0124] Step S501: Constructing a neural network training module, setting the search range of the first group of residual blocks to 2 to 4, the search range of the second group of residual blocks to 3 to 5, the search range of the third group of residual blocks to 5 to 7, and the search range of the fourth group of residual blocks to 2 to 4, generating a combination matrix for the number of residual blocks within the search range, inputting each residual block number configuration in the combination matrix into the objective function calculation unit in sequence, and calculating the value of the objective function through verification of the training set;

[0125] Step S502: Construct a three-layer convolution structure for each residual block, where the first layer is a dimensionality reduction convolution layer with a 1×1 convolution kernel, which is used to reduce the number of feature map channels; the second layer is a standard convolution layer with a 3×3 convolution kernel, which is used to extract spatial features; the third layer is a dimensionality increase convolution layer with a 1×1 convolution kernel, which is used to restore the number of feature map channels; a batch normalization layer and an activation function layer are added between the three-layer convolution structure, and the combination of the number of residual blocks with the optimal objective function value is determined as the final structure of the feature extraction layer.

[0126] Optionally, this embodiment details the search optimization and structural design of residual blocks in a neural network, achieving adaptive optimization of the network structure for the feature extraction requirements of a bridge crane visual monitoring system. During the residual block search process, based on an in-depth analysis of the visual feature hierarchy, the network is divided into four functional groups, with the search range of each group differentiated based on the required feature extraction complexity.

[0127] The first group of residual blocks is mainly responsible for basic feature extraction, with a search range of 2 to 4. This is because low-level features such as edges and textures are relatively simple and do not require too many calculation levels. The second group of residual blocks is responsible for mid-level feature combination, with a search range of 3 to 5, which is used to capture the local structural features of objects. The third group of residual blocks is the core processing unit of the network, with a search range of 5 to 7, responsible for extracting complex high-level semantic features. The fourth group of residual blocks is used for feature integration and optimization, with a search range of 2 to 4, to complete feature dimensionality reduction and refinement.

[0128] The combination matrix is ​​generated recursively, traversing each possible value within the search range to generate a complete set of configurations. For example, if the first group selects 2 residual blocks, the second group selects 3, the third group selects 5, and the fourth group selects 2, the resulting configuration is [2, 3, 5, 2]. This configuration method can effectively balance the depth of feature extraction and computational efficiency in practical applications.

[0129] The objective function was designed by comprehensively considering multiple performance metrics. During the training set validation process, each configuration was fully trained and evaluated. Evaluation metrics included feature extraction accuracy, real-time performance, and computational resource consumption. In crane operation scenarios, particular attention was paid to the model's detection accuracy for key targets (such as hoisted objects and obstacles), as well as whether the processing speed met real-time control requirements.

[0130] The internal structure of the residual block utilizes a three-layer convolutional design, forming an efficient feature extraction pipeline. The 1×1 dimensionality reduction convolution layer reduces computational complexity by reducing the number of channels in the feature map, for example, compressing 256 channels to 64. The 3×3 standard convolution layer extracts spatial features from the compressed feature space, effectively capturing the object's shape and position. The 1×1 dimensionality increase convolution layer restores the feature map to its original number of channels, preserving information integrity.

[0131] The insertion of a batch normalization layer solves the vanishing gradient problem in deep network training. This implementation adds a batch normalization layer after each convolutional layer to normalize the feature distribution to a mean of 0 and a variance of 1, improving training stability. The activation function uses ReLU, which introduces nonlinear transformations while maintaining computational simplicity, enhancing the model's expressiveness.

[0132] The final architecture selection process for the feature extraction layer uses a comprehensive scoring mechanism. First, each configuration is evaluated on a validation set to calculate an objective function value. Then, the configuration with the optimal objective function value is selected as the final architecture. This approach ensures that the selected architecture has the best overall performance in real-world applications.

[0133] This embodiment demonstrates significant advantages in practical applications. Taking the precision lifting task of a crane as an example, the optimized network structure accurately identifies the position and posture of hoisted objects, maintaining stable detection performance even under complex backgrounds and lighting conditions. Furthermore, through the combination of dimensionality reduction and dimensionality increase convolution, computational complexity is effectively controlled, enabling the model to achieve real-time processing with limited computing resources, providing timely and reliable visual information support for intelligent crane control.

[0134] Through this adaptive optimization design, this embodiment significantly improves the model's computational efficiency while ensuring feature extraction accuracy. Particularly in scenarios involving simultaneous multi-target monitoring, the system demonstrates excellent real-time processing capabilities and robustness, providing reliable technical support for the intelligent upgrade of overhead cranes.

[0135] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 6 , and can also include the following:

[0136] Step S601: constructing a feature vector fusion module at the end of the neural network, flattening the feature map output by the feature extraction layer into a one-dimensional vector, mapping the one-dimensional vector into the position coordinates and confidence scores of the target detection box through a three-layer fully connected network, where the number of nodes in the first layer of the fully connected network is 1024, the number of nodes in the second layer is 512, and the number of nodes in the third layer is 256. The detection box coordinates with the highest confidence are screened out using a non-maximum suppression algorithm;

[0137] Step S602: Read the load data stream and the broken wire data stream from the distributed database, compare the load data stream with the preset safe load threshold, generate a weight status mark, accumulate and count the broken wire signals in the broken wire data stream, establish a broken wire quantity statistics table, store the weight status mark and the broken wire quantity statistics table in the status monitoring database, and judge the equipment operation status according to the safety level threshold.

[0138] Optionally, this embodiment details a complete implementation of feature vector fusion and condition monitoring. This solution enables collaborative analysis of visual inspection results and equipment status data for the safety monitoring needs of bridge cranes. The feature vector fusion module uses a multi-level feature mapping structure to convert two-dimensional feature map information into the specific parameters required for target detection.

[0139] The feature map flattening process uses row-first order to maintain spatial feature continuity. The flattened one-dimensional vector contains key feature information such as the target's location and shape. The fully connected network adopts a layer-by-layer dimensionality reduction design. The first layer of 1024 nodes is used to preserve the rich expression of the original features. The second layer of 512 nodes performs feature compression and filtering. The third layer of 256 nodes completes the final feature extraction and outputs the location coordinates (x, y, width, height) and confidence score of the target detection box.

[0140] Regarding node activation functions, the first two layers use ReLU functions to introduce nonlinear transformations. The final layer uses a Sigmoid function to ensure the output of position coordinates is within the valid range, and a Softmax function to normalize the confidence scores. This design ensures the effectiveness and reliability of detection results.

[0141] The non-maximum suppression algorithm plays a key role in object detection box selection. It first sorts the detection boxes in descending order based on their confidence scores, then calculates the intersection-over-union (IoU) ratio between the detection boxes. When the IoU ratio exceeds a preset threshold, the detection box with the higher confidence is retained, effectively resolving the issue of multiple detections. In crane operation scenarios, this mechanism can accurately locate the position of hoisted objects and avoid interference caused by duplicate detection.

[0142] The load data stream is processed using a real-time comparison strategy. The safe load threshold is set based on the crane's rated capacity, taking into account safety factors under different operating conditions. When the load exceeds the threshold, an overload warning flag is generated. A multi-level weight status flag design, including safe, critical, and overload states, facilitates subsequent safety control decisions.

[0143] Processing the broken wire data stream focuses on the cumulative effects of wire rope damage. Wire breakage signals are collected in real time by sensors, and a cumulative counter records the number of broken wires in each rope. A broken wire count table is constructed in matrix format, with rows representing wire rope locations and columns representing time series, making it easy to track wire breakage trends.

[0144] The condition monitoring database uses a time-series database structure, supporting efficient time-series data storage and query. In addition to storing basic status data, the database also contains auxiliary information such as data timestamps and device numbers to facilitate subsequent data analysis and traceability.

[0145] Safety thresholds are set using a multi-dimensional assessment approach. Different alert levels are determined by comprehensively considering multiple indicators, including weight status and the number of broken wires. For example, if the number of broken wires reaches 80% of the regulatory limit, an early warning signal will be triggered, prompting prompting timely equipment maintenance, even if the load is within the limit.

[0146] This embodiment demonstrates significant safety monitoring effectiveness in practical applications. For example, in a large-scale port crane operation, the feature vector fusion module accurately locates and tracks hoisted objects. Combined with load and wire breakage monitoring data, it enables comprehensive monitoring of equipment status. When an abnormality occurs, the system issues a timely warning, preventing accidents.

[0147] This multi-source data fusion monitoring solution significantly improves the safety and reliability of crane operations. Especially in heavy-load, high-frequency operation scenarios, the system can effectively warn of potential safety hazards, providing reliable data support for safe equipment operation and preventive maintenance.

[0148] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 7 , and can also include the following:

[0149] Step S701: Constructing a trajectory analysis module, reading the three-dimensional coordinate data of the hook in the position data stream, calculating the hook motion trajectory curve based on the cubic spline interpolation algorithm, performing real-time collision detection between the motion trajectory curve and the boundary surface of the safe working area, calculating the minimum distance between the trajectory curve and the boundary surface through the vector projection method, and generating a hook position status mark;

[0150] Step S702: Establish a multi-source information decision module, input the personnel detection results, weight status mark, broken wire quantity statistics and hook position status mark into the status evaluation function, generate an execution instruction code according to the safety assessment rules, and send the execution instruction code to the execution unit of the crane controller through the fieldbus network, wherein the execution unit includes a motor driver, a brake and an alarm device.

[0151] This embodiment also details the implementation of trajectory analysis and multi-source information decision-making in a crane intelligent safety control system, focusing on trajectory planning and safety control during lifting operations. The trajectory analysis module uses precise mathematical models and real-time calculations to achieve intelligent monitoring and early warning of hook motion.

[0152] The hook's three-dimensional coordinate data is collected using high-frequency sampling, acquiring real-time position information through a distributed sensor network. A cubic spline interpolation algorithm constructs a smooth trajectory curve between discrete coordinate points, ensuring both continuity and differentiability. The algorithm uses cubic polynomials as its basis functions, ensuring computational efficiency while ensuring smooth transitions and avoiding sudden changes.

[0153] The modeling of the safe operating area boundary surface takes into account fixed obstacles, dynamic equipment, and prohibited areas in the work environment. This boundary surface is represented using a piecewise 3D grid, with each grid cell containing position and normal information. During real-time collision detection, a bounding box is first used for rapid judgment, followed by a vector projection method to calculate the precise distance.

[0154] The vector projection method discretizes the trajectory curve into a dense set of points when calculating the minimum distance. For each point, the perpendicular projection distance to the boundary surface is calculated. When the distance falls below the safety threshold, an early warning signal is immediately generated. The hook position status marker, comprised of multiple levels, reflects the degree of proximity to the boundary, providing a basis for subsequent control decisions.

[0155] The multi-source information decision-making module utilizes a hierarchical decision-making structure. Personnel detection results focus on the distribution and movement of personnel within the operating area. When a person is detected entering a hazardous area, a high-priority safety response is triggered. Weight status marking and wire breakage statistics jointly determine the crane's load capacity, influencing the control parameters for operating speed and braking force.

[0156] The state assessment function uses a weighted scoring mechanism, assigning different weights to different safety factors based on their importance. For example, if a person is detected and the hook trajectory predicts a potential collision, the system immediately generates an emergency braking command. The assessment rules are based on expert experience and safety regulations, covering common hazardous conditions and emergency response plans.

[0157] Execution instruction codes use a standardized format, including operation type, parameters, and priority information. The fieldbus network utilizes real-time Ethernet protocol to ensure reliable transmission and timely execution of instructions. The motor driver regulates operating speed and acceleration and deceleration characteristics, while the brake provides emergency braking and position holding. Alarms include audible and visual alarms and remote notification capabilities.

[0158] This embodiment demonstrates excellent safety control effectiveness in practical applications. For example, in a large-scale lifting operation, when a hook carrying a heavy object approaches equipment or a building, the system can predict potential collision risks in advance and automatically adjust its trajectory or issue a warning. Furthermore, through comprehensive analysis of multi-source information, the system can make accurate control decisions when safety hazards are detected.

[0159] Through this intelligent trajectory analysis and decision-making control solution, this embodiment significantly improves the safety and intelligence of crane operations. Especially in complex operating environments, the system effectively prevents collisions, protects operators and equipment, and improves operational efficiency and automation. The system's real-time performance and reliability have been verified in practical applications, providing strong technical support for the intelligent upgrade of cranes.

[0160] In order to effectively improve the system's perception capability and warning accuracy, the present application provides an embodiment of a bridge crane intelligent safety monitoring device based on multi-source data fusion for realizing all or part of the contents of the bridge crane intelligent safety monitoring method based on multi-source data fusion, see Figure 8 The intelligent safety monitoring device for bridge cranes based on multi-source data fusion specifically includes the following contents:

[0161] The data acquisition module 10 is used to install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps;

[0162] a data preprocessing module 20 for dividing the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and writing the preprocessed data into a distributed database based on a time synchronization mechanism;

[0163] The fusion monitoring module 30 is used to read the preprocessed data from the distributed database and input the image data stream into the convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set. The output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the suspended object weight in the load data stream based on the set threshold, calculates the cumulative value of the broken wire number in the broken wire data stream, and determines whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream. According to the detection results, a control instruction is generated and sent to the on-site execution system.

[0164] From the above description, it can be seen that the intelligent safety monitoring device for bridge cranes based on multi-source data fusion provided by the embodiment of the present application can realize multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of suspended objects, broken wire ropes and hook trajectory deviation, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capabilities of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0165] From a hardware perspective, in order to effectively improve the system's perception capabilities and warning accuracy, this application provides an embodiment of an electronic device for implementing all or part of the contents of the multi-source data fusion-based intelligent safety monitoring method for bridge cranes. The electronic device specifically includes the following contents:

[0166] A processor, a memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to implement information transmission between the intelligent safety monitoring device for a bridge crane based on multi-source data fusion and related devices such as core business systems, user terminals, and related databases; the logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited thereto. In this embodiment, the logic controller can be implemented with reference to the embodiments of the intelligent safety monitoring method for a bridge crane based on multi-source data fusion and the embodiments of the intelligent safety monitoring device for a bridge crane based on multi-source data fusion in the embodiments, the contents of which are incorporated herein and repeated parts are not repeated.

[0167] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0168] In practical applications, portions of the multi-source data fusion-based intelligent safety monitoring method for bridge cranes can be executed on the electronic device as described above, or all operations can be performed on the client device. The specific selection can be based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any restrictions on this. If all operations are performed on the client device, the client device may also include a processor.

[0169] The aforementioned client device may include a communication module (i.e., a communication unit) capable of establishing a communication connection with a remote server to facilitate data transmission with the server. The server may include a server at the task scheduling center or, in other implementation scenarios, a server on an intermediate platform, such as a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may comprise a single computer device, a server cluster consisting of multiple servers, or a distributed server configuration.

[0170] Figure 9 Schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Figure 9 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that the Figure 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0171] In one embodiment, the function of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0172] Step S101: Installing heterogeneous sensors on a bridge crane to collect weight data, image data, position data, and broken wire data, and writing the weight data, image data, position data, and broken wire data into a multidimensional data cache matrix according to timestamps;

[0173] Step S102: Dividing the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream, and broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism;

[0174] Step S103: Read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the hook movement trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0175] As can be seen from the above description, the electronic device provided in the embodiment of the present application realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of suspended objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capabilities of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0176] In another embodiment, the intelligent safety monitoring device for bridge cranes based on multi-source data fusion can be configured separately from the central processing unit 9100. For example, the intelligent safety monitoring device for bridge cranes based on multi-source data fusion can be configured as a chip connected to the central processing unit 9100, and the function of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion can be realized through the control of the central processing unit.

[0177] like Figure 9 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Figure 9 In addition, the electronic device 9600 may also include all components shown in Figure 9 For components not shown, reference may be made to the prior art.

[0178] like Figure 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0179] Memory 9140 can be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It can store the aforementioned failure-related information and also store programs that execute the relevant information. The CPU 9100 can execute the programs stored in memory 9140 to implement information storage or processing.

[0180] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 may be, for example, a keypad or touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0181] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), or SIM card. Alternatively, it may be a memory that retains information even when power is off, can be selectively erased, and is capable of storing additional data. Examples of such memory are sometimes referred to as EPROMs. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs, or processes used by the central processing unit 9100 to execute operations of the electronic device 9600.

[0182] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, images, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various driver programs for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0183] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0184] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless local area network modules. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130, providing audio output via the speaker 9131 and receiving audio input from the microphone 9132, thereby implementing common telecommunication functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Furthermore, the audio processor 9130 is coupled to the central processing unit 9100, enabling local recording via the microphone 9132 and playback of stored audio via the speaker 9131.

[0185] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion, where the execution subject is a server or a client, in the above-mentioned embodiments. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion, where the execution subject is a server or a client, in the above-mentioned embodiments. For example, when the processor executes the computer program, the following steps are implemented:

[0186] Step S101: Installing heterogeneous sensors on a bridge crane to collect weight data, image data, position data, and broken wire data, and writing the weight data, image data, position data, and broken wire data into a multidimensional data cache matrix according to timestamps;

[0187] Step S102: Dividing the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream, and broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism;

[0188] Step S103: Read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the hook movement trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0189] As can be seen from the above description, the computer-readable storage medium provided in the embodiment of the present application realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of suspended objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capabilities of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0190] The embodiments of the present application also provide a computer program product capable of implementing all steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the above-mentioned embodiments, where the execution subject is a server or a client. When the computer program / instructions are executed by a processor, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion are implemented. For example, the computer program / instructions implement the following steps:

[0191] Step S101: Installing heterogeneous sensors on a bridge crane to collect weight data, image data, position data, and broken wire data, and writing the weight data, image data, position data, and broken wire data into a multidimensional data cache matrix according to timestamps;

[0192] Step S102: Dividing the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream, and broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream, and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism;

[0193] Step S103: Read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the hook movement trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0194] As can be seen from the above description, the computer program product provided in the embodiment of the present application realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. Frequency division sampling and time synchronization mechanisms are used to process multi-source data streams, and a convolutional neural network with adaptive residual block configuration is combined to achieve accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of suspended objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0195] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatuses, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0196] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0197] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0199] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. An intelligent safety monitoring method for bridge cranes based on multi-source data fusion, characterized in that: The method comprises: Heterogeneous sensors are installed on the bridge crane to collect weight data, image data, position data, and broken wire data, and the weight data, image data, position data, and broken wire data are written into a multidimensional data cache matrix according to timestamps; The data in the multidimensional data cache matrix is ​​divided into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes. A multi-channel data acquisition module is configured in the on-site control unit of the bridge crane. The sampling clock frequency of the position data stream and the load data stream is set to 50 Hz, and data is collected at a time interval of 20 milliseconds through a timing trigger mechanism. The sampling clock frequency of the image data stream is set to 30 Hz, and data is collected at a time interval of 33 milliseconds through a timing trigger mechanism. The cache capacity of each data stream is calculated based on the sampling clock frequency. A wavelet transform is applied to the data stream to remove high-frequency noise. A sliding median filter is used to eliminate outliers. An anomaly detection model is established based on the statistical characteristics of the data. Data outside the confidence interval is marked as an outlier and interpolation correction is performed. A timestamp alignment algorithm is used to correct the sampling delay between different data streams. The preprocessed data is written into the corresponding table entry of the distributed database in time series. The preprocessed data is read from the distributed database, and the image data stream is input into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through a fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

2. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The step of writing the weight data, image data, position data, and broken wire data into a multidimensional data cache matrix according to timestamps, and dividing the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes includes: Establishing a four-dimensional data cache form in the bridge crane site controller, adding a collection time mark to the weight data, image data, position data, and broken wire data, and writing the data into the corresponding time axis position in the four-dimensional data cache form according to the collection time mark, wherein the four-dimensional data cache form includes a time dimension, a sensor type dimension, a data attribute dimension, and a value dimension; A feature classification operation is performed on the four-dimensional data cache form, and the functional characteristics of the sampled data are extracted from the data attribute dimension through a sliding window algorithm. The functional characteristics are grouped using a clustering analysis method, and data with the same functional attributes are organized into independent data streams to generate data structure models of position data stream, load data stream, image data stream and broken wire data stream.

3. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, including: Constructing a distributed data reading module, dividing the distributed database into a read and write buffer, establishing a data index table in the read buffer, retrieving the storage address of the target data block from the data index table according to the timestamp and data type identifier, and loading the target data block into the video memory buffer of the neural network computing unit in a batch processing manner; A basic convolutional layer is constructed in the neural network computing unit, and the image data stream is input into the basic convolutional layer after data normalization. A candidate residual block configuration pool is constructed, and the candidate residual block configuration pool contains a set of residual blocks with different number combinations. A network performance evaluation function is used as the objective function of the adaptive search, and the candidate residual block configuration is iteratively optimized based on the gradient descent method. The residual block number combination with the optimal objective function value is selected to construct a feature extraction layer.

4. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The objective function values ​​of different combinations of residual block numbers are iteratively calculated on the training set, including: Constructing a neural network training module, setting the search range of the first group of residual blocks to 2 to 4, the search range of the second group of residual blocks to 3 to 5, the search range of the third group of residual blocks to 5 to 7, and the search range of the fourth group of residual blocks to 2 to 4, generating a combination matrix for the number of residual blocks within the search range, inputting each residual block number configuration in the combination matrix into the objective function calculation unit in sequence, and calculating the value of the objective function through verification of the training set; A three-layer convolution structure is constructed for each residual block. The first layer is a dimensionality reduction convolution layer with a 1×1 convolution kernel, which is used to reduce the number of feature map channels. The second layer is a standard convolution layer with a 3×3 convolution kernel, which is used to extract spatial features. The third layer is a dimensionality increase convolution layer with a 1×1 convolution kernel, which is used to restore the number of feature map channels. A batch normalization layer and an activation function layer are added between the three-layer convolution structure. The combination of the number of residual blocks with the optimal objective function value is determined as the final structure of the feature extraction layer.

5. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The output layer outputs the coordinates of the personnel position in the dangerous area through the fully connected layer, detects the overweight state of the hanging object in the load data stream based on the set threshold, and calculates the cumulative value of the number of broken wires in the broken wire data stream, including: A feature vector fusion module is constructed at the end of the neural network to flatten the feature map output by the feature extraction layer into a one-dimensional vector. The one-dimensional vector is mapped into the position coordinates and confidence scores of the target detection box through a three-layer fully connected network. The number of nodes in the first layer of the fully connected network is 1024, the number of nodes in the second layer is 512, and the number of nodes in the third layer is 256. The coordinates of the detection box with the highest confidence are screened out using a non-maximum suppression algorithm. The load data stream and the broken wire data stream are read from the distributed database, the load data stream is compared with the preset safe load threshold, a weight status mark is generated, the broken wire signals in the broken wire data stream are accumulated and counted, a broken wire quantity statistical table is established, the weight status mark and the broken wire quantity statistical table are stored in the status monitoring database, and the equipment operation status is judged according to the safety level threshold.

6. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The method of combining the position data stream to determine whether the hook motion trajectory exceeds the safety boundary and generating a control instruction based on the detection result and sending it to the on-site execution system includes: Construct a trajectory analysis module to read the three-dimensional coordinate data of the hook in the position data stream, calculate the hook motion trajectory curve based on the cubic spline interpolation algorithm, perform real-time collision detection between the motion trajectory curve and the boundary surface of the safe working area, calculate the minimum distance between the trajectory curve and the boundary surface through the vector projection method, and generate the hook position status mark; A multi-source information decision module is established, and the personnel detection results, weight status mark, broken wire quantity statistics and hook position status mark are input into the status evaluation function. An execution instruction code is generated according to the safety assessment rules, and the execution instruction code is sent to the execution unit of the crane controller through the fieldbus network, wherein the execution unit includes a motor driver, a brake and an alarm device.

7. An intelligent safety monitoring device for bridge cranes based on multi-source data fusion, characterized in that: The device comprises: A data acquisition module is used to install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; a data preprocessing module for dividing the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream, and a broken wire data stream according to functional attributes; configuring a multi-channel data acquisition module in the on-site control unit of the bridge crane; setting the sampling clock frequency of the position data stream and the load data stream to 50 Hz, and acquiring data at a timed trigger mechanism at a time interval of 20 milliseconds; setting the sampling clock frequency of the image data stream to 30 Hz, and acquiring data at a timed trigger mechanism at a time interval of 33 milliseconds; calculating the cache capacity of each data stream based on the sampling clock frequency; applying a wavelet transform to the data stream to remove high-frequency noise; using a sliding median filter to eliminate outliers; establishing an anomaly detection model based on data statistical characteristics; marking data outside the confidence interval as an outlier and performing interpolation correction; using a timestamp alignment algorithm to correct the sampling delay between different data streams; and writing the preprocessed data into the corresponding table entry of the distributed database in time series; A fusion monitoring module is used to read preprocessed data from the distributed database and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, iteratively calculates the objective function value of different residual block number combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through a fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the broken wire number in the broken wire data stream, and determines whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion as described in any one of claims 1 to 6 are implemented.

Images