Bridge crane intelligent safety monitoring method and device based on multi-source data fusion

Abstract

According to the bridge crane intelligent safety monitoring method and device based on multi-source data fusion, by integrating the lateral pressure type pressure sensor, the intelligent camera, the displacement sensor and the electromagnetic sensor, multi-dimensional collection of weight, image, position and broken wire data is achieved. A multi-source data stream is processed by adopting frequency division sampling and a time synchronization mechanism, and accurate detection of personnel in a dangerous area is realized by combining a convolutional neural network configured by a self-adaptive residual block. The system monitors multi-dimensional safety risks such as weight overrun of a hoisted object, breakage of a steel wire rope and deviation of a lifting hook track at the same time, and all-dimensional intelligent monitoring of the operation process of the crane is achieved. The problem that a traditional single monitoring scheme is insufficient in perception capability is effectively solved, and the accuracy of safety early warning is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion. Background Art

[0002] With the development of intelligent manufacturing, bridge crane safety monitoring technology faces new challenges and requirements. Traditional monitoring methods mainly rely on single sensor data, lack a systematic safety assessment mechanism, and are difficult to cope with complex operating environments and diverse safety risks.

[0003] From the monitoring method point of view, the existing system generally has the problem of insufficient perception ability. Single weight monitoring or position monitoring is difficult to fully grasp the operating status of the crane, especially in terms of personnel safety protection and wire rope status monitoring. At the same time, the traditional image monitoring system has poor recognition effect under complex lighting conditions and it is difficult to accurately detect the situation of personnel in dangerous areas.

[0004] From the perspective of data processing, the existing solutions lack an effective multi-source data fusion mechanism. Data collected by different sensors are often processed independently, without fully utilizing the correlation between data, resulting in one-sided or delayed monitoring results. At the same time, the system does not take into account the frequency and synchronization of data collection, which affects the timeliness and accuracy of early warning.

[0005] From the perspective of safety assessment, traditional systems mostly use fixed threshold judgments and lack intelligent risk assessment capabilities. This simple judgment method cannot adapt to the needs of different operating conditions and is prone to false positives or omissions. Especially when multiple risk factors exist at the same time, it is difficult for the system to make accurate safety judgments.

[0006] Therefore, how to build an intelligent monitoring system with multi-source data fusion and improve the system's perception ability and early warning accuracy is a key issue that needs to be solved in the current bridge crane safety monitoring technology. This is not only related to operational safety, but also an important basis for realizing intelligent safety management. Summary of the invention

[0007] In response to the problems in the prior art, the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion, which can effectively improve the system's perception capability and warning accuracy.

[0008] In order to solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides an intelligent safety monitoring method for a bridge crane based on multi-source data fusion, comprising: Installing heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and writing the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Dividing the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream and broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism; The preprocessed data is read from the distributed database, and the image data stream is input into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on a network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on a training set, the output layer outputs the position coordinates of personnel in the dangerous area through a fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on a set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0009] Furthermore, the noise filtering and outlier correction of the position data stream, load data stream, image data stream and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism, include: A four-dimensional data cache form is established in the field controller of the bridge crane, a collection time mark is added to the weight data, image data, position data and broken wire data, and the data is written into the corresponding time axis position in the four-dimensional data cache form according to the collection time mark, wherein the four-dimensional data cache form includes a time dimension, a sensor type dimension, a data attribute dimension and a value dimension; A feature classification operation is performed on the four-dimensional data cache form, the functional characteristics of the sampled data are extracted from the data attribute dimension through a sliding window algorithm, the functional characteristics are grouped using a clustering analysis method, and data with the same functional attributes are organized into independent data streams to generate data structure models of position data stream, load data stream, image data stream and broken wire data stream.

[0010] Furthermore, the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, including: A multi-channel data acquisition module is configured in the field control unit of the bridge crane, the sampling clock frequency of the position data stream and the load data stream is set to 50 Hz, and data is collected at a time interval of 20 milliseconds through a timing trigger mechanism, the sampling clock frequency of the image data stream is set to 30 Hz, and data is collected at a time interval of 33 milliseconds through a timing trigger mechanism, and the cache capacity of the data stream is calculated according to the sampling clock frequency; Wavelet transform is applied to the data stream to remove high-frequency noise, a sliding median filter is used to eliminate outliers, an anomaly detection model is established based on data statistical characteristics, data beyond the confidence interval is marked as outliers and interpolation correction is performed, a timestamp alignment algorithm is used to correct the sampling delay between different data streams, and the preprocessed data is written into the corresponding table item of the distributed database in time series.

[0011] Further, the pre-processed data is read from the distributed database, and the image data stream is input into a convolutional neural network for personnel detection, wherein the convolutional neural network includes an input layer, a feature extraction layer, and an output layer, and the feature extraction layer determines the optimal residual block configuration through an adaptive search algorithm, including: Constructing a distributed data reading module, dividing the distributed database into a read-write buffer, establishing a data index table in the read buffer, retrieving the storage address of the target data block from the data index table according to the timestamp and data type identifier, and loading the target data block into the video memory buffer of the neural network computing unit in a batch processing manner; A basic convolution layer is built in the neural network computing unit, and the image data stream is input into the basic convolution layer after data normalization processing, and a candidate residual block configuration pool is constructed, wherein the candidate residual block configuration pool includes a residual block set of different quantity combinations, and a network performance evaluation function is used as the objective function of the adaptive search. The candidate residual block configuration is iteratively optimized based on the gradient descent method, and the residual block quantity combination with the optimal objective function value is selected to construct a feature extraction layer.

[0012] Furthermore, the iterative calculation of the objective function values ​​of different combinations of residual block numbers on the training set includes: Constructing a neural network training module, setting the search range of the first group of residual blocks to 2 to 4, the search range of the second group of residual blocks to 3 to 5, the search range of the third group of residual blocks to 5 to 7, and the search range of the fourth group of residual blocks to 2 to 4, generating a combination matrix for the number of residual blocks in the search range, inputting each residual block number configuration in the combination matrix into the objective function calculation unit in sequence, and calculating the value of the objective function through the training set verification; A three-layer convolution structure is constructed for each residual block. The first layer is a dimensionality reduction convolution layer with a 1×1 convolution kernel, which is used to reduce the number of feature map channels. The second layer is a standard convolution layer with a 3×3 convolution kernel, which is used to extract spatial features. The third layer is a dimensionality increase convolution layer with a 1×1 convolution kernel, which is used to restore the number of feature map channels. A batch normalization layer and an activation function layer are added between the three-layer convolution structure, and the combination of the number of residual blocks with the optimal objective function value is determined as the final structure of the feature extraction layer.

[0013] Furthermore, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, and calculates the cumulative value of the number of broken wires in the broken wire data stream, including: A feature vector fusion module is constructed at the end of the neural network, the feature map output by the feature extraction layer is flattened into a one-dimensional vector, and the one-dimensional vector is mapped into the position coordinates and confidence score of the target detection box through a three-layer fully connected network, the number of nodes in the first layer of the fully connected network is 1024, the number of nodes in the second layer is 512, and the number of nodes in the third layer is 256, and the coordinates of the detection box with the highest confidence are screened out through a non-maximum suppression algorithm; The load data stream and the broken wire data stream are read from the distributed database, the load data stream is compared with the preset safe load threshold, a weight status mark is generated, the broken wire signals in the broken wire data stream are accumulated and counted, a broken wire quantity statistical table is established, the weight status mark and the broken wire quantity statistical table are stored in the status monitoring database, and the equipment operation status is judged according to the safety level threshold.

[0014] Furthermore, the method of combining the position data stream to determine whether the hook motion trajectory exceeds the safety boundary and generating a control instruction according to the detection result and sending it to the on-site execution system includes: Constructing a trajectory analysis module, reading the three-dimensional coordinate data of the hook in the position data stream, calculating the hook motion trajectory curve based on the cubic spline interpolation algorithm, performing real-time collision detection between the motion trajectory curve and the boundary surface of the safe operation area, calculating the minimum distance between the trajectory curve and the boundary surface by the vector projection method, and generating a hook position status mark; A multi-source information decision-making module is established, and the personnel detection results, weight status marks, broken wire quantity statistics and hook position status marks are input into the state evaluation function. An execution instruction code is generated according to the safety assessment rules, and the execution instruction code is sent to the execution unit of the crane controller through the field bus network, wherein the execution unit includes a motor driver, a brake and an alarm device.

[0015] In a second aspect, the present application provides an intelligent safety monitoring device for a bridge crane based on multi-source data fusion, comprising: A data acquisition module, which is used to install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to a timestamp; A data preprocessing module, used for dividing the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream and a broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, the load data stream, the image data stream and the broken wire data stream, and writing the preprocessed data into a distributed database based on a time synchronization mechanism; A fusion monitoring module is used to read preprocessed data from the distributed database and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on a network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on a training set, the output layer outputs the position coordinates of personnel in dangerous areas through a fully connected layer, detects the over-limit state of the weight of the hanging object in the load data stream based on a set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0016] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion are implemented.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion.

[0019] It can be seen from the above technical solution that the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion, which realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The multi-source data streams are processed by frequency division sampling and time synchronization mechanism, and the convolutional neural network with adaptive residual block configuration is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as overweight of suspended objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception ability of traditional single monitoring solutions and improves the accuracy of safety warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is one of the flow charts of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application; Figure 2 This is the second flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application; Figure 3 This is a third flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application; Figure 4 This is a fourth flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in an embodiment of the present application; Figure 5 This is a fifth flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application; Figure 6 This is a sixth flow chart of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the embodiment of the present application; Figure 7 FIG7 is a flow chart of an intelligent safety monitoring method for a bridge crane based on multi-source data fusion in an embodiment of the present application; Figure 8 It is a structural diagram of an intelligent safety monitoring device for a bridge crane based on multi-source data fusion in an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0022] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0025] Taking into account the problems existing in the prior art, the present application provides an intelligent safety monitoring method and device for bridge cranes based on multi-source data fusion, which realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The multi-source data streams are processed by frequency division sampling and time synchronization mechanisms, and the convolutional neural network with adaptive residual block configuration is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as overweight of suspended objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception capability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0026] In order to effectively improve the system's perception capability and warning accuracy, the present application provides an embodiment of an intelligent safety monitoring method for a bridge crane based on multi-source data fusion, see Figure 1 The intelligent safety monitoring method for bridge crane based on multi-source data fusion specifically includes the following contents: Step S101: Install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Optionally, the embodiment of the present application can install a lateral pressure sensor at the fixed end of the wire rope of the mobile trolley of the bridge crane to collect weight data, install an intelligent camera on the top of the mobile trolley to collect images of the working area, install a displacement sensor at the end of the mobile trolley and the hook running track to collect spatial position data, and install an electromagnetic sensor on the hoisting system to collect broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Optionally, this embodiment builds a comprehensive data acquisition network by deploying multiple sensors at key locations of the bridge crane. The lateral pressure sensor installed at the fixed end of the wire rope of the mobile trolley adopts a high-precision strain gauge structure, which can convert the wire rope tension into an electrical signal to achieve real-time monitoring of the weight of the hanging object. The sensor adopts the principle of elastic body deformation and eliminates the influence of temperature drift through the compensation circuit to ensure the stability of the measurement data.

[0027] The smart camera is installed on the top of the mobile vehicle and uses a wide-angle lens to achieve full coverage monitoring of the working area. The camera has automatic exposure and wide dynamic range functions, which can cope with different lighting environments indoors and outdoors to ensure image quality. In low-light conditions, the image brightness is improved through an adaptive gain algorithm to ensure the continuity of monitoring in dangerous areas.

[0028] The displacement sensor adopts a photoelectric encoder solution and is installed at the ends of the moving trolley and hook running track respectively. By measuring the pulse signal output by the encoder, the spatial position coordinates of the trolley and hook are calculated. In order to avoid the influence of mechanical vibration, the sensor adopts a special vibration reduction installation structure to improve the accuracy of the position data.

[0029] The electromagnetic sensor in the hoisting system is used to detect wire rope breakage, using the magnetic flux leakage detection principle. The sensor coil generates an alternating magnetic field. When the wire rope breaks, the local magnetic permeability changes, causing the induced voltage to fluctuate, and the broken wire characteristics are identified through the signal processing circuit.

[0030] The data acquisition system adopts a distributed architecture. Each sensor is equipped with an independent data acquisition module to transmit data to the central processing unit via industrial Ethernet. The acquisition module has a built-in high-speed AD converter to achieve synchronous data sampling. The time synchronization adopts the IEEE1588 protocol to ensure that the clock deviation of each acquisition point is at the microsecond level.

[0031] The multidimensional data cache matrix adopts a shared memory structure and supports multi-threaded concurrent access. The matrix organizes data by dimensions such as timestamp, data type, and sampling frequency to facilitate subsequent data analysis and processing. The cache management adopts a circular queue mechanism. When the data volume exceeds the preset threshold, the earliest data record is automatically cleared.

[0032] The data writing process uses a double buffer mechanism to avoid data read and write conflicts. The newly collected data is first written to the buffer, and then transferred to the main storage area after data integrity verification. This mechanism effectively prevents data loss and write errors.

[0033] This embodiment establishes a holographic perception network of the operating status of the bridge crane through the collaborative work of multi-source sensors. The reasonable arrangement of sensors and reliable data collection mechanism lay a solid foundation for subsequent safety monitoring and intelligent control. This solution effectively solves the problems of large blind spots and poor data synchronization of traditional single sensor monitoring, and improves the reliability and real-time performance of the monitoring system.

[0034] In practical applications, this embodiment can quickly respond to various dangerous working conditions. For example, when a crane is lifting large objects, the coordination of pressure sensors and displacement sensors can accurately monitor the weight and movement trajectory of the objects, and promptly detect abnormal conditions such as overload or trajectory deviation. At the same time, the image monitoring system tracks the activities of personnel in the operating area in real time, effectively preventing the occurrence of human-machine collision accidents.

[0035] Step S102: Divide the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream and broken wire data stream according to functional attributes, perform noise filtering and outlier correction on the position data stream, load data stream, image data stream and broken wire data stream, and write the pre-processed data into a distributed database based on a time synchronization mechanism; Optionally, the embodiment of the present application can divide the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream and a broken wire data stream according to functional attributes, set a 50 Hz sampling frequency for the position data stream and the load data stream, set a 30 Hz sampling frequency for the image data stream, perform noise filtering and outlier correction on the data streams, and write the preprocessed data into a distributed database based on a time synchronization mechanism; Optionally, this embodiment first conducts an in-depth analysis of the original data in the multidimensional data cache matrix. Through the study of data characteristics, it is found that different types of data have their own unique time domain characteristics and frequency domain characteristics. For example, the position data is manifested as continuous changes in spatial coordinates, the load data shows the characteristics of fluctuations during the lifting process, the image data contains rich visual information, and the broken wire data is manifested as discrete state quantity changes. Based on these feature differences, a data classification algorithm based on cluster analysis is designed to accurately divide the data into four independent data streams.

[0036] In terms of sampling frequency setting, this embodiment determines the optimal sampling parameters through a large number of experiments and theoretical analysis. For the position data stream, according to the maximum operating speed and acceleration characteristics of the bridge crane, it is deduced that the highest frequency component of the position change is about 25Hz. According to the Nyquist sampling theorem, in order to avoid aliasing distortion, the sampling frequency of the position data is set to 50Hz. The load data also uses a 50Hz sampling frequency because the transient load changes and mechanical vibration frequencies during the lifting process usually do not exceed 25Hz, and this sampling frequency can fully record the load change process.

[0037] The 30Hz sampling frequency for image data streams is the result of in-depth consideration. The human eye can track moving targets at about 24 frames per second. The 30Hz sampling frequency not only meets the visual needs of the human eye, but also reserves sufficient computing time for image processing. In practical applications, this frequency can accurately capture fast movements in the operating area while balancing system resource usage.

[0038] Data preprocessing adopts a multi-level progressive solution. The first layer is noise filtering. For the position data stream, wavelet transform is first applied to decompose the signal into different frequency bands, and the high-frequency noise coefficient is processed by soft threshold function, and then the noise-reduced signal is reconstructed. The noise filtering of load data adopts adaptive bandpass filter. The center frequency of the filter is dynamically adjusted according to the current working status of the crane, which not only retains the effective signal but also suppresses the interference component.

[0039] The second layer is outlier correction. This embodiment innovatively proposes an adaptive threshold method based on a sliding window. By calculating the mean and standard deviation in the local data window, a dynamically changing confidence interval is established. For data points that exceed the confidence interval, the system will use the cubic spline interpolation method in combination with the physical model constraints for correction. This method not only maintains the continuity of the data, but also ensures that the correction results conform to the laws of physics.

[0040] The time synchronization mechanism is a key link in data processing. This embodiment adopts a hierarchical time synchronization scheme. The main controller acts as a time server and provides a unified time reference to each sensor node through the Precision Time Protocol (PTP). During the processing of the data stream, the data streams with different sampling frequencies are unified to the same time reference through the timestamp alignment algorithm to ensure the timing consistency of the data.

[0041] The distributed database uses a layered storage strategy for writing. High-frequency data such as location and load data are first written to the cache layer, and then stored in the persistent storage layer after batch sorting. Image data is stored incrementally, saving only the difference information between adjacent frames, significantly reducing storage space usage. The data writing process uses a double buffering mechanism, with the write buffer and persistent storage area working alternately to avoid data congestion.

[0042] This embodiment achieves high-quality data collection and storage through a carefully designed data processing process. In practical applications, when the crane performs precision lifting operations, the solution can provide stable and reliable status monitoring data. For example, when lifting important equipment, the system can accurately record every tiny position change and load fluctuation, providing operators with accurate decision-making basis. At the same time, complete data records also provide reliable data support for subsequent equipment maintenance and fault diagnosis.

[0043] Step S103: read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0044] Optionally, the embodiment of the present application can read the preprocessed data from the distributed database, input the image data stream into a convolutional neural network for personnel detection, the convolutional neural network includes an input layer, a feature extraction layer and an output layer, the feature extraction layer determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on a network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on the training set, and the search range of the residual block quantity combination is 2-4 in the first group, 3-5 in the second group, and 5-7 in the third group. The fourth group has 2-4, and the combination of the number of residual blocks with the optimal objective function value is selected as the final network structure. Each residual block contains a 1×1 dimensionality reduction convolution layer, a 3×3 standard convolution layer and a 1×1 dimensionality increase convolution layer. The output layer outputs the coordinates of the personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates control instructions according to the detection results and sends them to the on-site execution system.

[0045] Optionally, this embodiment designs a complete data analysis and decision-making system, the core of which is a personnel detection module based on deep learning and a security monitoring mechanism based on multi-source data fusion. First, when reading pre-processed data from a distributed database, a parallel reading strategy is adopted, and unified scheduling is performed through a data cache pool to ensure the real-time and integrity of data reading.

[0046] In terms of personnel detection, this embodiment innovatively designs a convolutional neural network with an adaptive structure. The input layer uses a multi-scale image pyramid structure to scale the original image at different ratios, thereby enhancing the network's ability to detect objects of different sizes. In practical applications, this structure can simultaneously detect workers at both far and near locations, thereby improving the reliability of detection.

[0047] The design of the feature extraction layer adopts the residual learning framework and optimizes the network structure through an adaptive search algorithm. The objective function comprehensively considers the detection accuracy, computational complexity and model size in the form of a weighted combination of evaluation indicators. During the search process, the algorithm first divides the residual blocks into four groups according to their functions, and each group is responsible for extracting features at different levels. The first group is responsible for low-level edge feature extraction, with a search range of 2-4 residual blocks; the second group is responsible for mid-level texture feature extraction, with a range of 3-5; the third group is responsible for high-level semantic feature extraction, with a range of 5-7; and the fourth group is responsible for feature integration, with a range of 2-4.

[0048] Each residual block adopts a "bottleneck" structure design, which contains three convolution layers. The 1×1 dimensionality reduction convolution layer is used to reduce the number of feature channels and reduce the amount of calculation; the 3×3 standard convolution layer performs the main feature extraction; and the 1×1 dimensionality increase convolution layer restores the number of feature channels to maintain the integrity of the information. This structure significantly reduces the computational overhead while ensuring network performance.

[0049] During the adaptive search process, a gradient-based iterative optimization strategy is used. In each iteration, the algorithm randomly selects a set of residual block number configurations and evaluates the network performance on the training set. To improve the search efficiency, an early stopping strategy is used. When the initial training effect of a configuration is obviously poor, the evaluation of the configuration is terminated in advance.

[0050] The output layer generates the precise position coordinates of the personnel in the dangerous area through the fully connected layer. To improve the positioning accuracy, a multi-task learning framework is used to simultaneously predict the personnel bounding box and key point positions. This design can more accurately determine the positional relationship between the operator and the dangerous area.

[0051] In terms of multi-source data fusion, this embodiment designs a hierarchical safety monitoring mechanism. The load monitoring module adopts an adaptive threshold strategy to dynamically adjust the weight limit according to the type of hanging object and the working condition. The broken wire monitoring adopts a cumulative statistical method and combines the life prediction model to evaluate the safety status of the wire rope. The trajectory monitoring module constructs a three-dimensional safety envelope surface to determine in real time whether there is a risk of crossing the boundary when the hook moves.

[0052] This embodiment realizes intelligent safety monitoring of the operation process of the bridge crane through deep learning and multi-source data analysis. In practical applications, when a dangerous situation is detected, the system can immediately generate corresponding control instructions. For example, when a person enters a dangerous area, the system will generate a deceleration or stop instruction based on the person's position and the hook movement state; when overweight or abnormal trajectory is detected, the system will trigger the corresponding safety protection mechanism. This intelligent monitoring method significantly improves the safety and reliability of crane operations.

[0053] From the above description, it can be seen that the intelligent safety monitoring method for bridge cranes based on multi-source data fusion provided in the embodiment of the present application can realize multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The multi-source data streams are processed by frequency division sampling and time synchronization mechanism, and the convolutional neural network with adaptive residual block configuration is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as overweight of hanging objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception ability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0054] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 2 , and can also include the following: Step S201: establishing a four-dimensional data cache form in the field controller of the bridge crane, adding a collection time mark to the weight data, image data, position data and broken wire data, and writing the data into the corresponding time axis position in the four-dimensional data cache form according to the collection time mark, wherein the four-dimensional data cache form includes a time dimension, a sensor type dimension, a data attribute dimension and a value dimension; Step S202: perform feature classification operations on the four-dimensional data cache form, extract functional features of the sampled data from the data attribute dimension through a sliding window algorithm, group the functional features using a clustering analysis method, organize data with the same functional attributes into independent data streams, and generate data structure models of position data streams, load data streams, image data streams, and broken wire data streams.

[0055] Optionally, this embodiment designs an efficient data cache and processing solution for the collection and organization of multi-source heterogeneous data of bridge cranes. First, a four-dimensional data cache form is constructed in the field controller, and a multi-dimensional array storage structure is used to achieve efficient access and management of data. The time dimension uses a unified timestamp format with millisecond accuracy to ensure the temporal consistency of the data; the sensor type dimension records the data source information, including the sensor number and type identifier; the data attribute dimension stores the feature description of the sampled data; and the numerical dimension stores the actual measurement value.

[0056] The addition of the collection time mark adopts a high-precision clock synchronization mechanism. In this embodiment, a clock synchronization module is configured in each sensor node to synchronize with the master clock through the IEEE 1588 precision time protocol, and the clock deviation is controlled at the microsecond level. When collecting data, the timestamp of the current moment is written as a mark into the data packet header to ensure the accurate positioning of the data in the time dimension.

[0057] The data writing process uses a double buffer mechanism, setting up a write buffer and a storage buffer. When one buffer is writing data, the other buffer performs data storage operations, and the data throughput capacity is improved through parallel processing. During the writing process, the data is mapped to the corresponding position of the four-dimensional form according to the timestamp, and the index structure is used to accelerate data positioning.

[0058] In order to effectively manage and utilize cached data, this embodiment designs a feature extraction algorithm based on a sliding window. The window size is dynamically adjusted according to the time characteristics of different types of data. For example, a smaller time window is used for rapidly changing position data, while a larger time window is used for slowly changing broken wire data. In each window, the algorithm extracts multiple statistical features, including mean, variance, peak value, frequency component, etc., to form a feature vector.

[0059] The feature classification operation adopts a two-stage processing strategy. In the first stage, the principal component analysis method is used to reduce the dimension of the feature vector and retain the main feature components. In the second stage, the improved K-means clustering algorithm is used to cluster data with similar functional attributes into the same group by calculating the Euclidean distance between feature vectors. Inertia weights are introduced in the clustering process to ensure that data with similar time are more easily classified into the same category.

[0060] Based on the clustering results, this embodiment constructs the structural models of four data streams. The position data stream adopts a time-series linked list structure to support fast trajectory reconstruction; the load data stream uses a circular buffer structure to facilitate real-time monitoring and trend analysis; the image data stream adopts a frame sequence structure to support continuous processing of video streams; the broken wire data stream uses a state record table structure, which is suitable for statistical analysis of discrete events.

[0061] In actual application scenarios, when a bridge crane performs multi-point collaborative lifting operations, this embodiment can achieve real-time collection and effective organization of multi-source data. For example, when hoisting large equipment, the system simultaneously records the position and load data of multiple lifting points, realizes unified management of information through the data flow model, and provides data support for collaborative control. At the same time, the complete data organization structure also provides a reliable data foundation for subsequent status monitoring and fault diagnosis.

[0062] Through this data management solution, this embodiment significantly improves data processing efficiency, reduces storage space usage, and provides strong support for intelligent control and safety monitoring of bridge cranes. Especially under complex working conditions, the system can quickly respond to data requests, detect abnormal conditions in a timely manner, and ensure the safety and reliability of crane operations.

[0063] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 3 , and can also include the following: Step S301: configuring a multi-channel data acquisition module in the field control unit of the bridge crane, setting the sampling clock frequency of the position data stream and the load data stream to 50 Hz, collecting data at a time interval of 20 milliseconds through a timing trigger mechanism, setting the sampling clock frequency of the image data stream to 30 Hz, collecting data at a time interval of 33 milliseconds through a timing trigger mechanism, and calculating the cache capacity of the data stream according to the sampling clock frequency; Step S302: Apply wavelet transform to the data stream to remove high-frequency noise, use a sliding median filter to eliminate outliers, establish an anomaly detection model based on data statistical characteristics, mark data beyond the confidence interval as outliers and perform interpolation correction, use a timestamp alignment algorithm to correct the sampling delay between different data streams, and write the preprocessed data into the corresponding table item of the distributed database in time series.

[0064] Optionally, this embodiment designs a set of accurate data acquisition and preprocessing solutions, focusing on solving the problems of synchronous acquisition and data quality optimization of multi-source heterogeneous data. In terms of hardware configuration, a multi-channel data acquisition module is used, and each channel is equipped with an independent signal conditioning circuit and analog-to-digital converter to ensure the parallelism and real-time performance of data acquisition.

[0065] The setting of sampling frequency is based on in-depth theoretical analysis and experimental verification. For the position data stream and load data stream, considering the dynamic characteristics of the bridge crane, by analyzing the spectral characteristics of its mechanical movement and load changes, it is determined that the main frequency components of the signal do not exceed 25Hz. According to the sampling theorem, a sampling frequency of 50Hz is selected, which can not only fully retain the signal characteristics, but also avoid the waste of resources caused by excessive sampling. The image data stream adopts a sampling frequency of 30Hz, which is the best choice after weighing the image quality, storage space and processing power, and can clearly capture the dynamic scenes in the working area.

[0066] The timing trigger mechanism is implemented using a high-precision hardware timer. This embodiment configures an independent timer for each sampling channel and accurately controls the sampling interval through interrupts. The 20-millisecond sampling interval of position and load data is set by the frequency division coefficient of the timer to ensure the stability of the sampling process. The 33-millisecond interval of image acquisition also uses hardware timing, combined with the exposure synchronization signal of the image sensor, to achieve accurate image acquisition.

[0067] The calculation of cache capacity takes into account the characteristics of the data stream and the system response requirements. This embodiment adopts a two-layer cache structure. The first-level cache is used for real-time data acquisition, and the capacity is set to twice the number of sampling points of the sampling frequency; the second-level cache is used for data preprocessing, and the capacity is determined according to the processing delay and data throughput. For example, for a data stream sampled at 50Hz, the first-level cache can store data of 100 sampling points to ensure that the acquisition process will not lose data due to processing delays.

[0068] Data preprocessing adopts a multi-level optimization strategy. First, noise reduction is performed through wavelet transform. This embodiment selects the db4 wavelet basis function, decomposes the signal into 5 layers, and processes the high-frequency coefficients through the soft threshold method to effectively remove the measurement noise while maintaining the mutation characteristics of the signal. This is particularly important for detecting key events such as emergency braking of cranes.

[0069] The sliding median filter is used to eliminate outliers, and the window size is dynamically adjusted according to the data type. For position data, a 5-point window is used to fully retain the continuity of the motion trajectory; for load data, a 7-point window is used to better suppress the interference caused by load fluctuations.

[0070] The anomaly detection model is built based on the statistical characteristics of the data. This embodiment establishes an adaptive confidence interval by calculating the mean and standard deviation within the sliding time window. For data points outside the confidence interval, the cubic spline interpolation method is used to correct them to ensure that the corrected data conforms to physical laws and maintains time series continuity.

[0071] The timestamp alignment algorithm solves the synchronization problem between data streams with different sampling frequencies. This embodiment uses a linear interpolation method to align all data streams to a unified time base. For example, when it is necessary to align 30Hz image data with 50Hz position data, the precise position data corresponding to the image acquisition moment is calculated by interpolation to ensure the timing consistency of multi-source data.

[0072] In the database writing process, a batch writing strategy is adopted to organize the pre-processed data by time period, and the atomicity of writing is ensured through the transaction mechanism. The data table adopts a time partition structure, which is convenient for historical data query and improves the access efficiency of current data.

[0073] This embodiment significantly improves data quality in practical applications. For example, when lifting precision equipment, high-quality position and load data provide a reliable basis for precise control, and the noise-reduced image data provides clear visual feedback for operating area monitoring, ultimately achieving safe and efficient operation of the bridge crane.

[0074] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 4 , and can also include the following: Step S401: construct a distributed data reading module, divide the distributed database into a read and write buffer, establish a data index table in the read buffer, retrieve the storage address of the target data block from the data index table according to the timestamp and data type identifier, and load the target data block into the video memory buffer of the neural network computing unit in a batch processing manner; Step S402: construct a basic convolution layer in the neural network computing unit, input the image data stream into the basic convolution layer after data normalization, construct a candidate residual block configuration pool, the candidate residual block configuration pool contains residual block sets of different quantity combinations, use the network performance evaluation function as the objective function of the adaptive search, iteratively optimize the candidate residual block configuration based on the gradient descent method, and select the residual block quantity combination with the optimal objective function value to construct a feature extraction layer.

[0075] Optionally, this embodiment proposes an efficient distributed data processing and deep learning model construction scheme, which realizes the rapid reading and feature extraction of massive data for the intelligent monitoring scenario of bridge cranes. First, a partition management strategy is adopted in the data reading link to divide the distributed database into independent read and write buffers to avoid mutual interference between read and write operations. The read buffer adopts a multi-level index structure, including two dimensions of time index and data type index, to achieve rapid positioning of data.

[0076] The data index table is implemented using a B+ tree structure. Leaf nodes store the physical addresses of actual data blocks, and non-leaf nodes store index key values. The timestamp is used as the primary index key, and the data type identifier is used as the secondary index key to form a composite index structure. During the retrieval process, the corresponding time partition is first located by the timestamp, and then an accurate search is performed within the partition based on the data type identifier, which significantly reduces the time complexity of data retrieval.

[0077] Batch data loading uses a pre-reading mechanism to load the data blocks that may be used later into the video memory buffer in advance according to the timing characteristics of data access. This embodiment designs an adaptive batch size adjustment strategy, which dynamically adjusts the size of each batch of data by monitoring the GPU video memory usage and computing load, thereby ensuring computing efficiency and avoiding video memory overflow.

[0078] In terms of neural network design, this embodiment adopts a modular network structure. The basic convolution layer uses a standard 3×3 convolution kernel, combined with a batch normalization layer and a ReLU activation function to extract low-level features of the image. The data normalization process uses a batch normalization method to scale the image data to the [-1,1] interval to improve the training stability of the model.

[0079] The design of the candidate residual block configuration pool reflects the innovation of this embodiment. Each residual block contains two convolutional layers and a shortcut connection, and a feature extraction layer is formed by combining different numbers of residual blocks. The configuration pool contains a variety of combination schemes, such as [2,2,2], [3,4,6] and other configurations of different depths, each of which corresponds to different feature extraction capabilities and computational complexity.

[0080] The network performance evaluation function comprehensively considers multiple indicators, including feature extraction accuracy, computational latency, and resource consumption. The evaluation function adopts the form of weighted summation, and the weight coefficient is obtained by optimizing experimental data. In the crane operation scenario, special attention is paid to the real-time and accuracy of feature extraction, so these two indicators are given higher weights in the evaluation function.

[0081] The adaptive search process uses an improved gradient descent algorithm. First, the initial configuration is randomly selected and the objective function value is calculated. Then a local search is performed in the configuration space, and a neighborhood solution is generated by adding or deleting residual blocks, and the objective function value of the new configuration is calculated. If the new configuration is better, it is accepted; otherwise, the suboptimal solution is accepted with a certain probability to avoid falling into the local optimum.

[0082] This embodiment shows excellent performance in practical applications. Taking crane hoisting operation as an example, when it is necessary to monitor the image data of multiple cameras in real time, the distributed data reading module can quickly respond to data requests and ensure the continuous supply of data. The optimized neural network model can accurately identify key targets in the operating area, such as the position and posture of the hoisted object, and provide reliable visual perception support for the intelligent control of the crane.

[0083] Through this design, this embodiment significantly improves the data processing efficiency and the accuracy of feature extraction. Especially under complex working conditions, such as multi-machine collaborative operation or operation monitoring under severe weather conditions, the system shows good adaptability and reliability, providing a strong guarantee for the intelligent and safe operation of bridge cranes.

[0084] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 5 , and can also include the following: Step S501: constructing a neural network training module, setting the search range of the first group of residual blocks to 2 to 4, the search range of the second group of residual blocks to 3 to 5, the search range of the third group of residual blocks to 5 to 7, and the search range of the fourth group of residual blocks to 2 to 4, generating a combination matrix for the number of residual blocks in the search range, inputting each residual block number configuration in the combination matrix into the objective function calculation unit in sequence, and calculating the value of the objective function through the training set verification; Step S502: construct a three-layer convolution structure for each residual block, the first layer is a dimensionality reduction convolution layer with a 1×1 convolution kernel, which is used to reduce the number of feature map channels, the second layer is a standard convolution layer with a 3×3 convolution kernel, which is used to extract spatial features, and the third layer is a dimensionality increase convolution layer with a 1×1 convolution kernel, which is used to restore the number of feature map channels. A batch normalization layer and an activation function layer are added between the three-layer convolution structure, and the combination of the number of residual blocks with the optimal objective function value is determined as the final structure of the feature extraction layer.

[0085] Optionally, this embodiment elaborates on the search optimization and structural design scheme of the residual block in the neural network, and realizes the adaptive optimization of the network structure according to the feature extraction requirements of the bridge crane visual monitoring system. In the residual block search process, based on the in-depth analysis of the hierarchy of visual features, the network is divided into four functional groups, and the search range of each group is differentiated according to the complexity requirements of feature extraction.

[0086] The first group of residual blocks is mainly responsible for basic feature extraction, and the search range is set to 2 to 4. This is because the underlying features such as edges and textures are relatively simple and do not require too many calculation levels. The second group of residual blocks is responsible for mid-level feature combination, with a search range of 3 to 5, which is used to capture the local structural features of objects. The third group of residual blocks is the core processing unit of the network, with a search range of 5 to 7, responsible for extracting complex high-level semantic features. The fourth group of residual blocks is used for feature integration and optimization, with a search range of 2 to 4, to complete the dimensionality reduction and refinement of features.

[0087] The combination matrix is ​​generated recursively, traversing the possible values ​​in each search range to generate a complete configuration combination. For example, when the first group selects 2 residual blocks, the second group selects 3, the third group selects 5, and the fourth group selects 2, a set of configurations [2,3,5,2] is formed. This configuration method can balance the depth of feature extraction and computational efficiency in practical applications.

[0088] The design of the objective function takes into account multiple performance indicators. During the training set verification process, each configuration is fully trained and evaluated. The evaluation indicators include feature extraction accuracy, real-time performance, and computing resource consumption. In the crane operation scenario, special attention is paid to the model's detection accuracy of key targets (such as hoisted objects, obstacles, etc.), and whether the processing speed meets the requirements of real-time control.

[0089] The internal structure of the residual block adopts a three-layer convolution design to form an efficient feature extraction channel. The 1×1 dimensionality reduction convolution layer reduces the computational complexity by reducing the number of channels in the feature map, for example, compressing 256 channels to 64 channels. The 3×3 standard convolution layer extracts spatial features in the compressed feature space and effectively captures the shape and position information of the target. The 1×1 dimensionality increase convolution layer restores the feature map to the original number of channels to maintain the integrity of the information.

[0090] The insertion of the batch normalization layer solves the gradient vanishing problem in deep network training. This embodiment adds a batch normalization layer after each convolutional layer to normalize the feature distribution to a range of mean 0 and variance 1, thereby improving the stability of training. The activation function uses ReLU, which introduces nonlinear transformation while keeping the calculation simple, thereby enhancing the expressiveness of the model.

[0091] The final structure selection process of the feature extraction layer adopts a comprehensive scoring mechanism. First, the performance of each configuration is evaluated on the validation set and the objective function value is calculated. Then, the configuration with the best objective function value is selected as the final structure. This method ensures that the selected structure has the best comprehensive performance in practical applications.

[0092] This embodiment shows significant advantages in practical applications. Taking the precision lifting task of a crane as an example, the optimized network structure can accurately identify the position and posture of the hoisted object, and maintain stable detection performance even under complex background and lighting conditions. At the same time, through the coordination of dimensionality reduction and dimensionality increase convolution, the computational complexity is effectively controlled, so that the model can achieve real-time processing under limited computing resources, providing timely and reliable visual information support for the intelligent control of the crane.

[0093] Through this adaptive optimization design, this embodiment significantly improves the computational efficiency of the model while ensuring the accuracy of feature extraction. Especially in the scenario of simultaneous monitoring of multiple targets, the system demonstrates excellent real-time processing capabilities and robustness, providing reliable technical support for the intelligent upgrade of bridge cranes.

[0094] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 6 , and can also include the following: Step S601: construct a feature vector fusion module at the end of the neural network, flatten the feature map output by the feature extraction layer into a one-dimensional vector, map the one-dimensional vector into the position coordinates and confidence score of the target detection box through a three-layer fully connected network, the number of nodes in the first layer of the fully connected network is 1024, the number of nodes in the second layer is 512, and the number of nodes in the third layer is 256, and the coordinates of the detection box with the highest confidence are screened out through a non-maximum suppression algorithm; Step S602: read the load data stream and the broken wire data stream from the distributed database, compare the load data stream with the preset safe load threshold, generate a weight status mark, accumulate and count the broken wire signals in the broken wire data stream, establish a broken wire quantity statistics table, store the weight status mark and the broken wire quantity statistics table in the status monitoring database, and judge the equipment operation status according to the safety level threshold.

[0095] Optionally, this embodiment elaborates on the complete implementation scheme of feature vector fusion and state monitoring, and realizes the collaborative analysis of visual inspection results and equipment status data for the safety monitoring requirements of bridge cranes. The feature vector fusion module adopts a multi-level feature mapping structure to convert the two-dimensional feature map information into specific parameters required for target detection.

[0096] The feature map flattening process adopts row priority order to maintain the continuity of spatial features. The flattened one-dimensional vector contains key feature information such as the position and shape of the target. The fully connected network adopts the design idea of ​​layer-by-layer dimensionality reduction. The first layer of 1024 nodes is used to retain the rich expression of the original features, the second layer of 512 nodes is used for feature compression and screening, and the third layer of 256 nodes completes the final feature extraction and outputs the position coordinates (x, y, width, height) and confidence score of the target detection box.

[0097] In terms of the selection of node activation functions, the first two layers use the ReLU function to introduce nonlinear transformation, the last layer uses the Sigmoid function for the position coordinates to ensure that the output is within the valid range, and the Softmax function is used for normalization of the confidence score. This design ensures the effectiveness and reliability of the detection results.

[0098] The non-maximum suppression algorithm plays a key role in the selection of target detection frames. First, the detection frames are sorted in descending order according to the confidence score, and then the intersection-and-union ratio between the detection frames is calculated. When the intersection-and-union ratio exceeds the preset threshold, the detection frame with higher confidence is retained, which effectively solves the problem of multiple detections. In the crane operation scenario, this mechanism can accurately locate the position of the hoisted object and avoid interference caused by repeated detection.

[0099] The processing of the load data stream adopts a real-time comparison strategy. The safe load threshold is set according to the rated lifting capacity of the crane, taking into account the safety factor under different working conditions. When the load exceeds the threshold, an overload warning mark is generated. The weight status mark adopts a multi-level design, including safe, critical, overload and other states, which facilitates subsequent safety control decisions.

[0100] The processing of broken wire data stream focuses on the cumulative effect of wire rope damage. The broken wire signal is collected in real time by sensors, and the cumulative counter records the number of broken wires in each wire rope. The broken wire number statistics table is in matrix form, with rows representing wire ropes at different positions and columns representing time series, which is convenient for tracking the development trend of broken wires.

[0101] The condition monitoring database adopts a time series database structure, which supports efficient time series data storage and query. In addition to storing basic status data, the database also contains auxiliary information such as data timestamps and equipment numbers to facilitate subsequent data analysis and tracing.

[0102] The setting of the safety level threshold adopts a multi-dimensional evaluation method. It takes into account multiple indicators such as weight status and the number of broken wires, and sets different warning levels. For example, when the number of broken wires reaches 80% of the specification limit, even if the load is within the limit, an early warning signal will be triggered, prompting timely equipment maintenance.

[0103] This embodiment has shown significant safety monitoring effects in practical applications. Taking large-scale port lifting operations as an example, the feature vector fusion module can accurately locate and track hoisted objects, and combined with load and broken wire monitoring data, it can achieve all-round monitoring of equipment status. When abnormal situations occur, the system can issue warnings in time to prevent safety accidents.

[0104] Through this multi-source data fusion monitoring solution, this embodiment significantly improves the safety and reliability of crane operations. Especially in heavy-load and high-frequency operation scenarios, the system can effectively warn of potential safety hazards and provide reliable data support for the safe operation and preventive maintenance of equipment.

[0105] In one embodiment of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion of the present application, see Figure 7 , and can also include the following: Step S701: construct a trajectory analysis module, read the three-dimensional coordinate data of the hook in the position data stream, calculate the hook motion trajectory curve based on the cubic spline interpolation algorithm, perform real-time collision detection between the motion trajectory curve and the boundary surface of the safe operation area, calculate the minimum distance between the trajectory curve and the boundary surface by the vector projection method, and generate a hook position status mark; Step S702: Establish a multi-source information decision module, input the personnel detection results, weight status mark, broken wire quantity statistics and hook position status mark into the status evaluation function, generate an execution instruction code according to the safety evaluation rules, and send the execution instruction code to the execution unit of the crane controller through the field bus network, wherein the execution unit includes a motor driver, a brake and an alarm device.

[0106] Optionally, this embodiment elaborates on the implementation scheme of trajectory analysis and multi-source information decision-making in the intelligent safety control system of the crane, focusing on solving the trajectory planning and safety control problems during the lifting operation. The trajectory analysis module realizes intelligent monitoring and early warning of the hook movement through precise mathematical models and real-time calculations.

[0107] The collection of the hook's three-dimensional coordinate data adopts a high-frequency sampling method to obtain real-time position information through a distributed sensor network. The cubic spline interpolation algorithm constructs a smooth trajectory curve between discrete coordinate points, ensuring the continuity and differentiability of the curve. The algorithm uses cubic polynomials as the basis function, which ensures the smooth transition of the trajectory curve while ensuring the calculation efficiency and avoiding the appearance of mutation points.

[0108] The modeling of the boundary surface of the safe working area takes into account the fixed obstacles, dynamic equipment and no-go areas in the working environment. The boundary surface is represented by a piecewise three-dimensional grid, and each grid cell contains position and normal vector information. In the real-time collision detection process, the bounding box is first judged quickly, and then the vector projection method is used to calculate the precise distance.

[0109] When calculating the minimum distance, the vector projection method discretizes the trajectory curve into a dense point set, and calculates the vertical projection distance to the boundary surface for each point. When the distance is detected to be less than the safety threshold, an early warning signal is immediately generated. The hook position status mark contains multiple levels, reflecting the degree of proximity to the boundary, providing a basis for subsequent control decisions.

[0110] The multi-source information decision module adopts a hierarchical decision structure. The personnel detection results focus on the distribution and movement of personnel in the operating area. When a person is detected entering a dangerous area, a high-priority safety response is triggered. The weight status mark and the number of broken wires jointly determine the load capacity of the crane and affect the control parameters of the operating speed and braking force.

[0111] The state evaluation function uses a weighted scoring mechanism, and different safety factors are given different weights according to their importance. For example, when a person intrusion is detected and the hook trajectory predicts a possible collision, the system will immediately generate an emergency braking command. The evaluation rules are based on expert experience and safety regulations, covering common dangerous working conditions and emergency response plans.

[0112] The execution instruction code adopts a standardized format, including operation type, parameters and priority information. The fieldbus network adopts real-time Ethernet protocol to ensure reliable transmission and timely execution of instructions. The motor driver is responsible for adjusting the running speed and acceleration and deceleration characteristics, the brake realizes emergency braking and position holding, and the alarm device includes sound and light alarm and remote notification function.

[0113] This embodiment shows excellent safety control effects in practical applications. Taking the hoisting operation in a large workshop as an example, when the hook carrying heavy objects approaches equipment or buildings, the system can predict the potential collision risk in advance, automatically adjust the movement trajectory or issue a warning. At the same time, through the comprehensive analysis of multi-source information, the system can make accurate control decisions when safety hazards are detected.

[0114] Through this intelligent trajectory analysis and decision-making control scheme, this embodiment significantly improves the safety and intelligence level of crane operations. Especially in complex operating environments, the system can effectively prevent collision accidents, protect the safety of operators and equipment, and improve operating efficiency and automation. The real-time and reliability of the system have been verified by practical applications, providing strong technical support for the intelligent upgrade of cranes.

[0115] In order to effectively improve the perception ability and early warning accuracy of the system, the present application provides an embodiment of an intelligent safety monitoring device for a bridge crane based on multi-source data fusion for realizing all or part of the contents of the intelligent safety monitoring method for a bridge crane based on multi-source data fusion, see Figure 8 The intelligent safety monitoring device for bridge crane based on multi-source data fusion specifically includes the following contents: The data acquisition module 10 is used to install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to a timestamp; A data preprocessing module 20 is used to divide the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream and a broken wire data stream according to functional attributes, perform noise filtering and outlier correction on the position data stream, the load data stream, the image data stream and the broken wire data stream, and write the preprocessed data into a distributed database based on a time synchronization mechanism; The fusion monitoring module 30 is used to read the preprocessed data from the distributed database and input the image data stream into the convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates control instructions according to the detection results and sends them to the on-site execution system.

[0116] From the above description, it can be seen that the intelligent safety monitoring device for bridge cranes based on multi-source data fusion provided in the embodiment of the present application can realize multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The multi-source data streams are processed by frequency division sampling and time synchronization mechanism, and the convolutional neural network with adaptive residual block configuration is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as overweight of hanging objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception ability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0117] From the hardware level, in order to effectively improve the system's perception capability and early warning accuracy, the present application provides an embodiment of an electronic device for implementing all or part of the contents of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion, and the electronic device specifically includes the following contents: Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the intelligent safety monitoring device for bridge crane based on multi-source data fusion and related equipment such as core business system, user terminal and related database; the logic controller can be a desktop computer, tablet computer and mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the intelligent safety monitoring method for bridge crane based on multi-source data fusion and the embodiment of the intelligent safety monitoring device for bridge crane based on multi-source data fusion in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0118] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0119] In practical applications, part of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion can be executed on the electronic device side as described above, or all operations can be completed in the client device. The selection can be made based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0120] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0121] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0122] In one embodiment, the function of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion can be integrated into the central processor 9100. The central processor 9100 can be configured to perform the following control: Step S101: Install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Step S102: Divide the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream and broken wire data stream according to functional attributes, perform noise filtering and outlier correction on the position data stream, load data stream, image data stream and broken wire data stream, and write the pre-processed data into a distributed database based on a time synchronization mechanism; Step S103: read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0123] From the above description, it can be seen that the electronic device provided in the embodiment of the present application realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The frequency division sampling and time synchronization mechanism are used to process multi-source data streams, and the convolutional neural network configured with adaptive residual blocks is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of hanging objects, broken wire ropes and deviation of hook trajectory, and realizes all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception ability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0124] In another embodiment, the intelligent safety monitoring device for bridge cranes based on multi-source data fusion can be configured separately from the central processor 9100. For example, the intelligent safety monitoring device for bridge cranes based on multi-source data fusion can be configured as a chip connected to the central processor 9100, and the function of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion can be realized through the control of the central processor.

[0125] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0126] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0127] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0128] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0129] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0130] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0131] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0132] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0133] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps in the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the above-mentioned embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps in the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the above-mentioned embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: Install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Step S102: Divide the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream and broken wire data stream according to functional attributes, perform noise filtering and outlier correction on the position data stream, load data stream, image data stream and broken wire data stream, and write the pre-processed data into a distributed database based on a time synchronization mechanism; Step S103: read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0134] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The frequency division sampling and time synchronization mechanism are used to process multi-source data streams, and the convolutional neural network configured with adaptive residual blocks is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as overweight of hanging objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception ability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0135] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion are implemented. For example, the computer program / instruction implements the following steps: Step S101: Install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Step S102: Divide the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream and broken wire data stream according to functional attributes, perform noise filtering and outlier correction on the position data stream, load data stream, image data stream and broken wire data stream, and write the pre-processed data into a distributed database based on a time synchronization mechanism; Step S103: read the preprocessed data from the distributed database, and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on the network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on the training set, the output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

[0136] From the above description, it can be seen that the computer program product provided in the embodiment of the present application realizes multi-dimensional collection of weight, image, position and broken wire data by integrating lateral pressure sensors, intelligent cameras, displacement sensors and electromagnetic sensors. The frequency division sampling and time synchronization mechanism are used to process multi-source data streams, and the convolutional neural network configured with adaptive residual blocks is combined to realize accurate detection of personnel in dangerous areas. The system simultaneously monitors multi-dimensional safety risks such as excessive weight of hanging objects, broken wire ropes and deviation of hook trajectory, realizing all-round intelligent monitoring of the crane operation process. The present application effectively solves the problem of insufficient perception ability of traditional single monitoring solutions and improves the accuracy of safety warnings.

[0137] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An intelligent safety monitoring method for bridge cranes based on multi-source data fusion, characterized in that: The method comprises: Installing heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and writing the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to timestamps; Dividing the data in the multidimensional data cache matrix into position data stream, load data stream, image data stream and broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, load data stream, image data stream and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism; The preprocessed data is read from the distributed database, and the image data stream is input into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on a network performance evaluation index, and iteratively calculates the objective function value of different residual block quantity combinations on a training set, the output layer outputs the position coordinates of personnel in the dangerous area through a fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on a set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, and determines whether the motion trajectory of the hook exceeds the safety boundary in combination with the position data stream, and generates a control instruction based on the detection result and sends it to the on-site execution system.

2. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The weight data, image data, position data and broken wire data are written into a multidimensional data cache matrix according to timestamps, and the data in the multidimensional data cache matrix are divided into a position data stream, a load data stream, an image data stream and a broken wire data stream according to functional attributes, including: A four-dimensional data cache form is established in the field controller of the bridge crane, a collection time mark is added to the weight data, image data, position data and broken wire data, and the data is written into the corresponding time axis position in the four-dimensional data cache form according to the collection time mark, wherein the four-dimensional data cache form includes a time dimension, a sensor type dimension, a data attribute dimension and a value dimension; A feature classification operation is performed on the four-dimensional data cache form, the functional characteristics of the sampled data are extracted from the data attribute dimension through a sliding window algorithm, the functional characteristics are grouped using a clustering analysis method, and data with the same functional attributes are organized into independent data streams to generate data structure models of position data stream, load data stream, image data stream and broken wire data stream.

3. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The noise filtering and outlier correction of the position data stream, load data stream, image data stream and broken wire data stream, and writing the pre-processed data into a distributed database based on a time synchronization mechanism, include: A multi-channel data acquisition module is configured in the field control unit of the bridge crane, the sampling clock frequency of the position data stream and the load data stream is set to 50 Hz, and data is collected at a time interval of 20 milliseconds through a timing trigger mechanism, the sampling clock frequency of the image data stream is set to 30 Hz, and data is collected at a time interval of 33 milliseconds through a timing trigger mechanism, and the cache capacity of the data stream is calculated according to the sampling clock frequency; Wavelet transform is applied to the data stream to remove high-frequency noise, a sliding median filter is used to eliminate outliers, an anomaly detection model is established based on data statistical characteristics, data beyond the confidence interval is marked as outliers and interpolation correction is performed, a timestamp alignment algorithm is used to correct the sampling delay between different data streams, and the preprocessed data is written into the corresponding table item of the distributed database in time series.

4. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, including: Constructing a distributed data reading module, dividing the distributed database into a read-write buffer, establishing a data index table in the read buffer, retrieving the storage address of the target data block from the data index table according to the timestamp and data type identifier, and loading the target data block into the video memory buffer of the neural network computing unit in a batch processing manner; A basic convolution layer is built in the neural network computing unit, and the image data stream is input into the basic convolution layer after data normalization processing, and a candidate residual block configuration pool is constructed, wherein the candidate residual block configuration pool includes a residual block set of different quantity combinations, and a network performance evaluation function is used as the objective function of the adaptive search. The candidate residual block configuration is iteratively optimized based on the gradient descent method, and the residual block quantity combination with the optimal objective function value is selected to construct a feature extraction layer.

5. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The iterative calculation of the objective function values ​​of different combinations of residual block numbers on the training set includes: Constructing a neural network training module, setting the search range of the first group of residual blocks to 2 to 4, the search range of the second group of residual blocks to 3 to 5, the search range of the third group of residual blocks to 5 to 7, and the search range of the fourth group of residual blocks to 2 to 4, generating a combination matrix for the number of residual blocks in the search range, inputting each residual block number configuration in the combination matrix into the objective function calculation unit in sequence, and calculating the value of the objective function through the training set verification; A three-layer convolution structure is constructed for each residual block. The first layer is a dimensionality reduction convolution layer with a 1×1 convolution kernel, which is used to reduce the number of feature map channels. The second layer is a standard convolution layer with a 3×3 convolution kernel, which is used to extract spatial features. The third layer is a dimensionality increase convolution layer with a 1×1 convolution kernel, which is used to restore the number of feature map channels. A batch normalization layer and an activation function layer are added between the three-layer convolution structure, and the combination of the number of residual blocks with the optimal objective function value is determined as the final structure of the feature extraction layer.

6. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The output layer outputs the position coordinates of personnel in the dangerous area through the fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on the set threshold, and calculates the cumulative value of the number of broken wires in the broken wire data stream, including: A feature vector fusion module is constructed at the end of the neural network, the feature map output by the feature extraction layer is flattened into a one-dimensional vector, and the one-dimensional vector is mapped into the position coordinates and confidence score of the target detection box through a three-layer fully connected network, the number of nodes in the first layer of the fully connected network is 1024, the number of nodes in the second layer is 512, and the number of nodes in the third layer is 256, and the coordinates of the detection box with the highest confidence are screened out through a non-maximum suppression algorithm; The load data stream and the broken wire data stream are read from the distributed database, the load data stream is compared with the preset safe load threshold, a weight status mark is generated, the broken wire signals in the broken wire data stream are accumulated and counted, a broken wire quantity statistical table is established, the weight status mark and the broken wire quantity statistical table are stored in the status monitoring database, and the equipment operation status is judged according to the safety level threshold.

7. The intelligent safety monitoring method for bridge cranes based on multi-source data fusion according to claim 1 is characterized in that: The method of judging whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream and generating a control instruction according to the detection result and sending it to the on-site execution system includes: Constructing a trajectory analysis module, reading the three-dimensional coordinate data of the hook in the position data stream, calculating the hook motion trajectory curve based on the cubic spline interpolation algorithm, performing real-time collision detection between the motion trajectory curve and the boundary surface of the safe operation area, calculating the minimum distance between the trajectory curve and the boundary surface by the vector projection method, and generating a hook position status mark; A multi-source information decision-making module is established, and the personnel detection results, weight status marks, broken wire quantity statistics and hook position status marks are input into the state evaluation function. An execution instruction code is generated according to the safety assessment rules, and the execution instruction code is sent to the execution unit of the crane controller through the field bus network, wherein the execution unit includes a motor driver, a brake and an alarm device.

8. An intelligent safety monitoring device for bridge cranes based on multi-source data fusion, characterized in that: The device comprises: A data acquisition module, which is used to install heterogeneous sensors on the bridge crane to collect weight data, image data, position data and broken wire data, and write the weight data, image data, position data and broken wire data into a multidimensional data cache matrix according to a timestamp; A data preprocessing module, used for dividing the data in the multidimensional data cache matrix into a position data stream, a load data stream, an image data stream and a broken wire data stream according to functional attributes, performing noise filtering and outlier correction on the position data stream, the load data stream, the image data stream and the broken wire data stream, and writing the preprocessed data into a distributed database based on a time synchronization mechanism; A fusion monitoring module is used to read preprocessed data from the distributed database and input the image data stream into a convolutional neural network for personnel detection, wherein the feature extraction layer of the convolutional neural network determines the optimal residual block configuration through an adaptive search algorithm, the adaptive search algorithm constructs an objective function based on a network performance evaluation index, iteratively calculates the objective function value of different residual block quantity combinations on a training set, the output layer outputs the position coordinates of personnel in dangerous areas through a fully connected layer, detects the over-limit state of the hanging weight in the load data stream based on a set threshold, calculates the cumulative value of the number of broken wires in the broken wire data stream, determines whether the hook motion trajectory exceeds the safety boundary in combination with the position data stream, and generates control instructions based on the detection results and sends them to the on-site execution system.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the intelligent safety monitoring method for bridge crane based on multi-source data fusion as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent safety monitoring method for bridge cranes based on multi-source data fusion as described in any one of claims 1 to 7 are implemented.

Images