Anti-collision detection method, system and equipment for hoisting machinery and medium

Abstract

The invention discloses an anti-collision detection method, system, equipment and medium for hoisting machinery, and relates to the technical field of computer platform load balancing, and the method comprises the following steps: collecting multi-modal data of the hoisting machinery, and preprocessing the multi-modal data; fusing the collected multi-modal data through Kalman filtering, constructing an LSTM-GRU prediction model, and inputting the fused data into the LSTM-GRU prediction model to predict the moving trajectory of the hoisting machinery; and according to the trajectory prediction result, an anti-collision control instruction is dynamically generated through the AI edge computing device, and the PLC receives and executes the anti-collision control instruction. According to the method, on the basis of multi-modal data fusion, the crane operation state sensing precision is improved, on the basis of LSTM-GRU trajectory prediction, accurate future trajectory calculation is achieved, on the basis of AI edge calculation equipment, dynamic anti-collision control is achieved, and the intelligent level of anti-collision detection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control and safety detection of lifting machinery, and in particular to a method, system, equipment and medium for anti-collision detection of lifting machinery. Background Art

[0002] As key equipment in industrial production, logistics and transportation, and construction, cranes are widely used in ports, warehouses, construction sites, and large manufacturing companies. However, during operation, cranes are prone to collision accidents due to their complex operating environment, large load changes, and nonlinear operating trajectories, resulting in equipment damage, casualties, and economic losses. Therefore, how to achieve intelligent anti-collision detection for cranes and ensure safe operation has become a key technical issue that needs to be solved in the industry.

[0003] The anti-collision methods for cranes in the industry mainly include fixed sensor detection method, rule-based collision warning method and manual monitoring method. However, these methods have limited sensor detection range and are easily affected by occlusion, resulting in detection blind spots. Sensors have poor adaptability to ambient lighting, rainy and foggy weather, dust interference, etc., which affects detection accuracy. They can only passively sense the current state, cannot predict the trajectory in advance, and cannot avoid collisions during high-speed movement. Relying on fixed rules, they cannot adapt to dynamic changes under different working conditions. They cannot predict future trajectories in advance, and cannot effectively avoid the risk of sudden collisions under high-speed operation. There is a problem of false alarms. When cranes need to operate finely in narrow areas, they may frequently trigger alarms by mistake, affecting work efficiency. Relying on manual experience, there are misjudgments and delayed reactions. Operators have high work intensity and are prone to fatigue due to long-term work, leading to safety hazards. Cranes cannot be accurately controlled in complex environments, especially in scenarios where multiple cranes work together, which is prone to a series of problems such as misoperation leading to collisions. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing anti-collision detection method for lifting machinery has insufficient perception accuracy, weak trajectory prediction ability, unintelligent anti-collision strategy, low real-time performance, and how to achieve high-precision trajectory prediction and dynamic anti-collision control based on multimodal data fusion and artificial intelligence technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for anti-collision detection of lifting machinery, comprising collecting multimodal data of the lifting machinery and preprocessing the multimodal data.

[0007] The collected multimodal data are fused through Kalman filtering, and an LSTM-GRU prediction model is constructed. The fused data is input into the LSTM-GRU prediction model to predict the operation trajectory of the lifting machinery.

[0008] According to the trajectory prediction results, anti-collision control instructions are dynamically generated through the AI ​​edge computing device, and the PLC receives and executes the anti-collision control instructions.

[0009] Multimodal data fusion includes data fusion through Kalman filtering to eliminate measurement errors and improve state estimation accuracy.

[0010] Building the LSTM-GRU prediction model includes using LSTM to capture long-term trajectory trends and combining GRU to process short-term trajectory fluctuations to achieve short-term + long-term trajectory prediction fusion.

[0011] Based on time series prediction, the trajectory of the crane N time steps in the future is calculated in advance to determine the potential collision risk.

[0012] The trajectory smoothing optimization method is used to optimize the trajectory calculation.

[0013] As a preferred solution of the anti-collision detection method for lifting machinery described in the present invention, the multimodal data of the lifting machinery is synchronously collected through AI cameras, laser radars, absolute encoders, and inertial sensors, and the collected multimodal data of the lifting machinery is used as observation values ​​to construct a time series data set of the operation of the lifting machinery.

[0014] Multimodal data includes object category, location coordinates, reflection intensity, target distance, hook height, running speed, angular velocity, and angular velocity.

[0015] As a preferred solution of the anti-collision detection method for lifting machinery described in the present invention, the sensor data is aligned according to the highest frequency, and the low-frequency sensor data is filled by linear interpolation.

[0016] The aligned data is denoised based on particle filtering, and the denoised sensor data is used for outlier detection using the mean-standard deviation method. The abnormal part is removed and filled again using the linear interpolation method. The multimodal data preprocessing process is repeated until no outlier data is detected in the outlier detection process.

[0017] Particle filtering denoising includes simulating system states through multiple particles, dynamically estimating the true signal value, and eliminating measurement noise.

[0018] Outlier detection involves calculating the average of all data points, measuring the central tendency of the data, and using the standard deviation Measure the degree of data dispersion, judge the deviation of data points from the mean according to the degree of dispersion, and determine the data with too large deviation as outliers.

[0019] As a preferred solution of the anti-collision detection method for lifting machinery described in the present invention, wherein: based on Kalman filtering, fuse the preprocessed multi-modal sensing data, set the position, speed and acceleration information of the lifting machinery through the preprocessed multi-modal data, and use the set position, speed and acceleration information of the lifting machinery as input data, and predict the operating state of the lifting machinery according to Newton's kinematic equation.

[0020] Use the state transition model to input the state information of the previous moment, combine with the preprocessed acceleration, calculate the position, speed and acceleration of the crane at the next moment, and define the motion law.

[0021] Calculate the Kalman gain, and use the measurement data to correct the predicted state according to the Kalman gain to obtain the fused position information, speed information and acceleration information.

[0022] Calculating the Kalman gain includes inputting the Kalman gain calculation formula through the measurement matrix, measurement noise covariance, and covariance matrix of state estimation to obtain the Kalman gain.

[0023] The covariance matrix of state estimation measures the uncertainty of the predicted value and dynamically adjusts the confidence level of the prediction.

[0024] The measurement matrix maps the state variables to the measurement space and calculates the correction value.

[0025] The measurement noise covariance matrix is set according to the error characteristics of the sensor itself, reflects the uncertainty of the measurement data, and optimizes the accuracy of state correction.

[0026] As a preferred solution of the anti-collision detection method for lifting machinery described in the present invention, wherein: use the fused position information, speed information and acceleration information data as input, normalize the input position information, speed information and acceleration information data, and form a time series through the normalized position information, speed information and acceleration information data.

[0027] Introduce the traditional LSTM prediction model into GRU calculation to construct GRU layer and fully connected layer.

[0028] Use the time series as input, adapt to the immediate motion changes of the lifting machinery through the GRU layer, output the overall trajectory trend of the lifting machinery through the LSTM layer, combine the output results of the GRU layer and the LSTM layer for trajectory prediction calculation, and output the predicted trajectory points.

[0029] The historical position, velocity, and acceleration data are input into the prediction model. Based on the mean square error calculation, a trajectory smoothing penalty term is introduced to construct a mean square error loss function. The historical trajectory prediction results are input into the mean square error loss function to calculate the error between the historical predicted trajectory and the actual trajectory.

[0030] The fusion result of multimodal data is input into the LSTM-GRU prediction model to output the operation trajectory of the lifting machinery.

[0031] As a preferred solution of the anti-collision detection method for lifting machinery described in the present invention, the trajectory data output by the prediction model and the position information of the obstacle are used as input to calculate the minimum safe distance between the lifting machinery and the obstacle, and set the safety distance threshold according to the minimum safe distance.

[0032] According to the set safety distance threshold, a control strategy is generated and converted into control instruction data.

[0033] Setting the safety distance threshold includes sending an early warning signal when the distance from the crane to the obstacle is less than the warning distance.

[0034] When the distance between the crane and the obstacle is less than the braking distance, the braking operation is performed.

[0035] Generating a control strategy includes setting a control state according to a set safety distance threshold. When the distance between the crane and the obstacle is greater than the warning distance, the crane operates normally.

[0036] When the distance between the crane and the obstacle is between the braking distance and the warning distance, an audible and visual alarm will be given.

[0037] When the distance from the crane to the obstacle is equal to the braking distance, a deceleration command is sent to the PLC to reduce the operating speed of the crane.

[0038] When the distance between the crane and the obstacle is less than the braking distance, a braking command is sent to the PLC to stop the crane from working.

[0039] As a preferred solution of the anti-collision detection method for lifting machinery described in the present invention, the control instruction data is taken as input and input into the PLC control execution unit.

[0040] Under the normal operation instruction of the lifting machinery, the PLC controls the execution unit to maintain the original operating speed, does not trigger the alarm, does not adjust the inverter frequency, and does not trigger the braking system.

[0041] Under the sound and light alarm command, the PLC controls the execution unit to trigger the LED indicator to flash, the buzzer to sound, and the warning message to be displayed on the control panel or remote monitoring system. The crane speed will not be adjusted and the brake system will not be triggered.

[0042] Under the instruction to reduce the operating speed of the crane, the PLC controls the execution unit to adjust the speed, adjust the output frequency of the inverter, maintain the sound and light alarm, and do not trigger the brake system.

[0043] When the crane is stopped, the PLC controls the execution unit to cut off the power supply of the crane drive motor, trigger the brake system, perform emergency braking, maintain the sound and light alarm, and send a shutdown report.

[0044] Another object of the present invention is to provide a lifting machinery anti-collision detection system, which can solve the problems of insufficient perception accuracy, weak trajectory prediction ability, unintelligent anti-collision control strategy and low real-time performance in the current lifting machinery anti-collision detection technology through an AI edge computing solution based on multimodal data fusion and LSTM-GRU trajectory prediction.

[0045] As a preferred solution of the anti-collision detection system for lifting machinery described in the present invention, it includes: a data acquisition and preprocessing module, a data fusion and prediction module, and an instruction generation and execution module. The acquisition and preprocessing module is used to acquire multimodal data of the lifting machinery and preprocess the multimodal data. The data fusion and prediction module is used to fuse the collected multimodal data through Kalman filtering, build an LSTM-GRU prediction model, and input the fused data into the LSTM-GRU prediction model to predict the operation trajectory of the lifting machinery. The instruction generation and execution module is used to dynamically generate anti-collision control instructions through the AI ​​edge computing device according to the trajectory prediction results, and the PLC receives and executes the anti-collision control instructions.

[0046] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a step of a method for anti-collision detection of a lifting machinery.

[0047] A computer-readable storage medium stores a computer program, which implements the steps of a method for preventing collisions of lifting machinery when executed by a processor.

[0048] Beneficial effects of the present invention: The anti-collision detection method for crane machinery provided by the present invention is based on multimodal data fusion to improve the perception accuracy of the crane's operating status, based on LSTM-GRU trajectory prediction to achieve accurate future trajectory calculation, and based on AI edge computing equipment to achieve dynamic anti-collision control, which effectively improves the intelligence level of anti-collision detection. The present invention has achieved better results in terms of crane operation safety, trajectory prediction accuracy and anti-collision response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0050] Figure 1 An overall flow chart of a crane anti-collision detection method provided for the first embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0052] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a lifting machinery anti-collision detection method, comprising:

[0053] S1: Collect multimodal data of lifting machinery and preprocess the multimodal data.

[0054] The multimodal data of the lifting machinery is synchronously collected through AI cameras, lidars, absolute encoders, and inertial sensors. The collected multimodal data of the lifting machinery is used as observation values ​​to construct a time series data set of the lifting machinery operation.

[0055] Multimodal data includes object category, location coordinates, reflection intensity, target distance, hook height, running speed, angular velocity, and angular velocity.

[0056] Furthermore, the sensor data is aligned according to the highest frequency, and the low-frequency sensor data is filled by linear interpolation. A preferred solution of linear interpolation filling is: ; in, represents the interpolated data points, Indicates time The original data at the moment, Indicates time The original data at the moment, Indicates a timestamp.

[0057] The aligned data is denoised based on particle filtering, and the denoised sensor data is used for outlier detection using the mean-standard deviation method. The abnormal part is removed and filled again using the linear interpolation method. The multimodal data preprocessing process is repeated until no outlier data is detected in the outlier detection process.

[0058] Particle filtering denoising includes simulating system states through multiple particles, dynamically estimating true signal values, eliminating measurement noise, and improving data stability.

[0059] An optimal solution for particle filtering denoising is: ; in, represents the optimal state estimate after denoising, represents the number of particles, Indicates The state of a particle at the previous moment, represents the control input, represents the process noise, represents the state transfer equation, Indicates The weight of a particle.

[0060] Outlier detection involves calculating the average of all data points, measuring the central tendency of the data, and using the standard deviation Measures the degree of dispersion of data and determines the deviation of data points from the mean based on the degree of dispersion.

[0061] A preferred solution for outlier detection is: ; ; in, represents the mean of the sample data, represents the number of samples, Indicates data points, represents the standard deviation, Represents the deviation of each data point from the mean.

[0062] S2: The collected multimodal data is fused through Kalman filtering, and an LSTM-GRU prediction model is constructed. The fused data is input into the LSTM-GRU prediction model to predict the operation trajectory of the lifting machinery.

[0063] The preprocessed multimodal sensor data is fused based on Kalman filtering. The position, velocity and acceleration information of the lifting machinery are set through the preprocessed multimodal data. The set position, velocity and acceleration information of the lifting machinery are used as input data, and the operating state of the lifting machinery is predicted according to Newton's kinematic equations.

[0064] A preferred solution for predicting the operating status of a lifting machinery is: ; ; ; ; in, Indicates the current time step The state estimation vector, represents the state transfer matrix, represents the control input matrix, represents the process noise, Indicates the current time step The control input, represents the time step, , , Indicates the acceleration values ​​on three axes.

[0065] The state transfer model is used to input the state information of the previous moment, and combined with the preprocessed acceleration, the position, velocity and acceleration of the crane at the next moment are calculated to define the motion law.

[0066] The Kalman gain is calculated, and the predicted state is corrected using the measured data according to the Kalman gain to obtain the fused position information, velocity information, and acceleration information.

[0067] A preferred solution for state correction is: ; ; in, represents the Kalman gain matrix, represents the state covariance matrix, represents the measurement matrix, represents the measurement noise covariance matrix, represents the final estimated state vector, represents the measured value at the current time step, represents the projection of the predicted state into the measurement space, Represents the Kalman gain Measurement update.

[0068] Calculating the Kalman gain includes inputting a Kalman gain calculation formula through a measurement matrix, a measurement noise covariance, and a state estimation covariance matrix to obtain the Kalman gain.

[0069] Furthermore, a state transition model is adopted to input the state information of the previous moment, combined with the preprocessed acceleration, to calculate the position, velocity and acceleration of the crane at the next moment, to define the motion law, and to correct the predicted state according to the measured data.

[0070] The Kalman gain is calculated by inputting the Kalman gain calculation formula through the measurement matrix, the measurement noise covariance, and the covariance matrix of the state estimation.

[0071] The covariance matrix of the state estimate is used to measure the uncertainty of the predicted value and dynamically adjust the confidence level of the prediction.

[0072] The measurement matrix is ​​used to map the state variables into the measurement space to ensure that the calculated correction values ​​are consistent with the data provided by the sensors.

[0073] The measurement noise covariance matrix is ​​set according to the sensor's own error characteristics to reflect the uncertainty of the measurement data and optimize the accuracy of the state correction.

[0074] Furthermore, the fused position information, velocity information, and acceleration information data are taken as input, the input position information, velocity information, and acceleration information data are normalized, and the normalized position information, velocity information, and acceleration information data form a time series.

[0075] A preferred scheme for composing a time series is: ; in, Represents the time series data matrix input to LSTM-GRU, Indicates the current time step location information, Indicates the current time step Speed ​​information, Current time step The acceleration information, Indicates the length of the time window.

[0076] The traditional LSTM prediction model is introduced into GRU calculation, and the GRU layer and the fully connected layer are constructed.

[0077] A preferred solution for calculating the LSTM prediction model is: ; in, Indicates that LSTM is at time step The hidden state of represents the output gate of LSTM, Represents the memory cell state of LSTM, represents the activation of long-term trajectory information through the hyperbolic tangent function, Represents element-wise multiplication.

[0078] A preferred solution for introducing GRU calculation is: ; in, Indicates that GRU is at time step The hidden state of represents the update gate of GRU, represents the bias term of GRU, represents the input data of the current time step, represents the weight matrix of GRU, Represents the reset gate of GRU.

[0079] The time series is used as input, the GRU layer is used to adapt to the instant motion changes of the lifting machinery, the LSTM layer is used to output the overall trajectory trend of the lifting machinery, and the output results of the GRU layer and the LSTM layer are combined to perform trajectory prediction calculation and output the predicted trajectory points.

[0080] A preferred solution for combining the GRU layer and the LSTM layer is: ; in, Represents the predicted trajectory data for the next moment, represents the weight matrix of the output layer, represents the LSTM-GRU combination scaling factor, represents the long-term trajectory features calculated by LSTM, Represents the short-term trajectory features calculated by GRU, Represents the bias term of the output layer.

[0081] A preferred solution for trajectory prediction is: ; in, Indicates the predicted future The trajectory data of time steps, Indicates that based on input Predict future trajectories.

[0082] The historical position, velocity, and acceleration data are input into the prediction model. Based on the mean square error calculation, a trajectory smoothing penalty term is introduced to construct a mean square error loss function. The historical trajectory prediction results are input into the mean square error loss function to calculate the error between the historical predicted trajectory and the actual trajectory.

[0083] An optimal solution for constructing the mean square error loss function is: ; in, represents the total loss function, represents the real future trajectory data, Represents: Future trajectory data predicted by LSTM-GRU, represents the number of training data samples, represents the smoothing penalty coefficient, represents the second-order derivative of the predicted trajectory, Indicates the time window length for smoothing calculation.

[0084] S3: Based on the trajectory prediction results, anti-collision control instructions are dynamically generated through the AI ​​edge computing device, and the PLC receives and executes the anti-collision control instructions.

[0085] The trajectory data output by the prediction model and the location information of the obstacle are used as input to calculate the minimum safe distance between the lifting machinery and the obstacle, and the safety distance threshold is set according to the minimum safe distance.

[0086] A preferred method for calculating the minimum safe distance between a lifting machine and an obstacle is: ; in, represents the crane and the obstacle at the time step The Euclidean distance of , , Denotes the crane at time step The three-dimensional coordinate position of , , Indicates that the obstacle is at the time step The three-dimensional coordinate position of .

[0087] According to the set safety distance threshold, a control strategy is generated and converted into control instruction data.

[0088] Furthermore, setting the safety distance threshold includes sending a warning signal when the distance from the crane to the obstacle is less than the warning distance.

[0089] When the distance between the crane and the obstacle is less than the braking distance, the braking operation is performed.

[0090] Generating a control strategy includes setting a control state according to a set safety distance threshold. When the distance between the crane and the obstacle is greater than the warning distance, the crane operates normally.

[0091] When the distance between the crane and the obstacle is between the braking distance and the warning distance, an audible and visual alarm will be given.

[0092] When the distance from the crane to the obstacle is equal to the braking distance, a deceleration command is sent to the PLC to reduce the operating speed of the crane.

[0093] When the distance between the crane and the obstacle is less than the braking distance, a braking command is sent to the PLC to stop the crane from working.

[0094] Furthermore, the control instruction data is taken as input and input into the PLC control execution unit.

[0095] Under the normal operation instruction of the lifting machinery, the PLC controls the execution unit to maintain the original operating speed, does not trigger the alarm, does not adjust the inverter frequency, and does not trigger the braking system.

[0096] Under the sound and light alarm command, the PLC controls the execution unit to trigger the LED indicator to flash, the buzzer to sound, and the warning message to be displayed on the control panel or remote monitoring system. The crane speed will not be adjusted and the brake system will not be triggered.

[0097] Under the instruction to reduce the operating speed of the crane, the PLC controls the execution unit to adjust the speed, adjust the output frequency of the inverter, maintain the sound and light alarm, and do not trigger the brake system.

[0098] A preferred solution for adjusting the inverter output frequency is: ; in, Indicates the new inverter output frequency of the crane. Indicates the current inverter output frequency of the crane. Indicates the speed adjustment factor.

[0099] When the crane is stopped, the PLC controls the execution unit to cut off the power supply of the crane drive motor, trigger the brake system, perform emergency braking, maintain the sound and light alarm, and send a shutdown report.

[0100] Embodiment 2 is an embodiment of the present invention, which provides a method for detecting anti-collision of a lifting machinery. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0101] In order to verify the effectiveness of the crane anti-collision detection method based on LSTM-GRU trajectory prediction of the present invention, this experiment was tested in an indoor crane operation environment of a large manufacturing plant. The experimental site includes multiple fixed obstacles, moving vehicles, manual operators, and multiple cranes operating in parallel. The purpose of the experiment is to evaluate the system's anti-collision detection accuracy, trajectory prediction accuracy, and control response speed under different working conditions.

[0102] First, the equipment and sensor configuration uses a bridge crane with a maximum lifting load of 10 tons and an operating speed of 1~3m / s.

[0103] The sensor uses an AI camera for target recognition with a frame rate of 30FPS. The laser radar detects the distance to obstacles with a scanning frequency of 10Hz. The absolute encoder records the hook position with a resolution of 0.01m. The inertial sensor IMU measures angular velocity with an acceleration accuracy of 0.01m / s².

[0104] The experiment first collects the operation data of the crane and preprocesses the data. Synchronize multimodal data, including crane position, speed, acceleration, obstacle distance and other information. Use particle filtering for denoising to reduce sensor errors and improve data quality. Use linear interpolation to fill low-frequency sensor data to ensure data time synchronization. Perform outlier detection based on the mean-standard deviation method, remove abnormal data and re-interpolate and fill.

[0105] Kalman filtering is used to fuse the data and estimate the real-time status of the crane. The fused data is input into the LSTM-GRU trajectory prediction model to predict the trajectory of the crane in the next 5 seconds. Trajectory smoothing optimization is used to ensure a smooth prediction curve and reduce mutation errors.

[0106] The safe distance between the crane and the obstacle is calculated, and the warning threshold is set to 3m, the deceleration threshold to 2m, and the braking threshold to 1m. When the predicted trajectory is about to approach the obstacle, the AI ​​computing device generates an anti-collision control instruction. The key indicators such as trajectory error, false alarm rate, control response time, and collision rate during the experiment are collected. The experimental data are shown in Table 1.

[0107] Table 1 Experimental data table Parameter name Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Subject 6 True track deviation (m) 0.12 0.08 0.18 0.14 0.10 0.20 Prediction trajectory deviation (m) 0.15 0.10 0.21 0.16 0.13 0.23 False alarm rate (%) 2.5 2.0 3.1 2.5 2.2 3.5 False positive rate (%) 1.7 1.5 2.0 1.8 1.6 2.5 Control response time (ms) 140 135 145 138 133 150 Number of collisions 0 0 0 0 0 0

[0108] Embodiment 3 is an embodiment of the present invention, which provides a lifting machinery anti-collision detection system, including a data acquisition and preprocessing module 100, a data fusion and prediction module 200, and an instruction generation and execution module 300.

[0109] Wherein S4: the collection and preprocessing module 100 is used to collect multimodal data of the lifting machinery and preprocess the multimodal data.

[0110] It should also be noted that the data acquisition and preprocessing module 100 is responsible for the state perception of the lifting machinery, and collects data from sensors such as AI cameras, laser radars, absolute encoders, inertial sensors, etc. in real time, including position information, speed information, and acceleration information. In order to improve the data quality, the module will perform time alignment, noise removal, outlier detection, and data completion on the collected multimodal data to ensure the reliability and consistency of the input data. The output of the module is high-quality preprocessed sensor data, which is passed to the data fusion and prediction module 200.

[0111] S5: The data fusion and prediction module 200 is used to fuse the collected multimodal data through Kalman filtering, build an LSTM-GRU prediction model, and input the fused data into the LSTM-GRU prediction model to predict the operation trajectory of the lifting machinery.

[0112] It should also be noted that the data fusion and prediction module 200 is responsible for fusing the pre-processed multimodal data and predicting the operation trend of the crane in advance based on the trajectory prediction algorithm. The sensor data is fused using Kalman filtering to eliminate measurement errors and build a unified state estimation model. The LSTM-GRU prediction model is used to input the fused data into the prediction model, and the future trajectory points are calculated through time series analysis to determine the motion state of the crane in the future time step. The output of the module is the predicted trajectory data, which is passed to the instruction generation and execution module 300.

[0113] S6: The instruction generation and execution module 300 is used to dynamically generate anti-collision control instructions through the AI ​​edge computing device according to the trajectory prediction results, and the PLC receives and executes the anti-collision control instructions.

[0114] It should also be noted that the command generation and execution module 300 is responsible for making intelligent anti-collision decisions based on trajectory prediction data and controlling the operation status of the crane. The AI ​​edge computing device analyzes the predicted trajectory and calculates the minimum safe distance between the crane and the obstacle. According to the safety threshold, the system dynamically generates anti-collision control instructions, and the PLC executes the corresponding operations.

[0115] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0116] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0117] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk case (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0118] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logical function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting collision prevention of a lifting machinery, characterized in that: include: Collect multimodal data of lifting machinery and pre-process the multimodal data; The collected multimodal data is fused through Kalman filtering to build an LSTM-GRU prediction model, and the fused data is input into the LSTM-GRU prediction model to predict the operation trajectory of the lifting machinery; According to the trajectory prediction results, the AI ​​edge computing device dynamically generates anti-collision control instructions, and the PLC receives and executes the anti-collision control instructions; Multimodal data fusion includes data fusion through Kalman filtering to eliminate measurement errors and improve state estimation accuracy; The construction of LSTM-GRU prediction model includes using LSTM to capture long-term trajectory trends and combining GRU to process short-term trajectory fluctuations to achieve short-term + long-term trajectory prediction fusion; Based on time series prediction, the crane’s trajectory for N future time steps is calculated in advance to determine potential collision risks; The trajectory smoothing optimization method is used to optimize the trajectory calculation.

2. The anti-collision detection method for lifting machinery according to claim 1, characterized in that: The collecting of multimodal data of lifting machinery includes: The multimodal data of the lifting machinery is collected synchronously through AI cameras, lidars, absolute encoders, and inertial sensors. The collected multimodal data of the lifting machinery is used as observation values ​​to construct a time series data set of the lifting machinery operation. Multimodal data includes object category, location coordinates, reflection intensity, target distance, hook height, running speed, angular velocity, and angular velocity.

3. The anti-collision detection method for lifting machinery according to claim 1 or 2, characterized in that: The preprocessing of the multimodal data includes: Align the sensor data according to the highest frequency, and use linear interpolation to fill in the low-frequency sensor data; The aligned data is denoised based on particle filtering, and the sensor data after denoising is detected by the mean-standard deviation method. The abnormal part is removed and filled again using the linear interpolation method. The multimodal data preprocessing process is repeated until no abnormal value data is detected in the outlier detection process; Particle filter denoising includes simulating the system state through multiple particles, dynamically estimating the true signal value, and eliminating measurement noise; Outlier detection involves calculating the average of all data points, measuring the central tendency of the data, and using the standard deviation Measure the degree of dispersion of the data, determine the deviation of the data points from the mean based on the degree of dispersion, and identify data with excessive deviation as outliers.

4. The anti-collision detection method for lifting machinery according to claim 3, characterized in that: The multimodal data fusion includes: Based on Kalman filtering, the pre-processed multi-modal sensor data is fused, and the position, speed and acceleration information of the lifting machinery are set through the pre-processed multi-modal data. The set position, speed and acceleration information of the lifting machinery are used as input data, and the operation state of the lifting machinery is predicted according to Newton's kinematic equations; The state transition model is used to input the state information of the previous moment, and combined with the preprocessed acceleration, the position, speed and acceleration of the crane at the next moment are calculated to define the motion law; Calculate the Kalman gain, and use the measured data to correct the predicted state according to the Kalman gain to obtain the fused position information, velocity information, and acceleration information; Calculating the Kalman gain includes inputting a Kalman gain calculation formula through a measurement matrix, a measurement noise covariance, and a state estimation covariance matrix to obtain the Kalman gain; The covariance matrix of the state estimate measures the uncertainty of the predicted value and dynamically adjusts the confidence level of the prediction; The measurement matrix maps the state variables to the measurement space and calculates the correction values; The measurement noise covariance matrix is ​​set according to the sensor's own error characteristics to reflect the uncertainty of the measurement data and optimize the accuracy of state correction.

5. The anti-collision detection method for lifting machinery according to claim 1, 2 or 4, characterized in that: The construction of the LSTM-GRU prediction model includes: The fused position information, velocity information, and acceleration information data are used as input, and the input position information, velocity information, and acceleration information data are normalized, and the normalized position information, velocity information, and acceleration information data are used to form a time series; Introduce the traditional LSTM prediction model into GRU calculation and construct the GRU layer and the fully connected layer; Take the time series as input, adapt to the instant movement changes of the lifting machinery through the GRU layer, output the overall trajectory trend of the lifting machinery through the LSTM layer, combine the output results of the GRU layer and the LSTM layer to perform trajectory prediction calculation, and output the predicted trajectory points; The historical position, velocity, and acceleration data are input into the prediction model. Based on the mean square error calculation, a trajectory smoothing penalty term is introduced to construct a mean square error loss function. The historical trajectory prediction results are input into the mean square error loss function to calculate the error between the historical predicted trajectory and the actual trajectory. The fusion result of multimodal data is input into the LSTM-GRU prediction model to output the operation trajectory of the lifting machinery.

6. The anti-collision detection method for lifting machinery according to claim 5, characterized in that: The method of dynamically generating anti-collision control instructions by using an AI edge computing device includes: The trajectory data output by the prediction model and the location information of the obstacle are used as input to calculate the minimum safe distance between the lifting machinery and the obstacle, and the safety distance threshold is set according to the minimum safe distance; Generate a control strategy based on the set safety distance threshold and convert the control strategy into control instruction data; Setting the safety distance threshold includes sending a warning signal when the distance between the crane and the obstacle is less than the warning distance; When the distance between the crane and the obstacle is less than the braking distance, the braking operation is performed; Generating a control strategy includes setting the control state according to the set safety distance threshold, and when the distance between the crane and the obstacle is greater than the warning distance, the crane operates normally; When the distance between the crane and the obstacle is between the braking distance and the warning distance, an audible and visual alarm will be given; When the distance between the crane and the obstacle is equal to the braking distance, a deceleration command is sent to the PLC to reduce the crane's operating speed; When the distance between the crane and the obstacle is less than the braking distance, a braking command is sent to the PLC to stop the crane from working.

7. The anti-collision detection method for lifting machinery according to claim 1, 2, 4 or 6, characterized in that: The PLC receives and executes anti-collision control instructions including: The control instruction data is taken as input and input into the PLC control execution unit; Under normal operation instructions of the hoisting machinery, the PLC controls the execution unit to maintain the original operating speed, does not trigger the alarm, does not adjust the frequency of the inverter, and does not trigger the brake system; Under the sound and light alarm command, the PLC controls the execution unit to trigger the LED indicator to flash, the buzzer to sound, and the warning message to be displayed on the control panel or remote monitoring system, without adjusting the crane speed and triggering the brake system; Reduce the crane's operating speed. The PLC controls the execution unit to adjust the speed, adjust the inverter output frequency, maintain the sound and light alarm, and do not trigger the brake system. When the crane is stopped, the PLC controls the execution unit to cut off the power supply of the crane drive motor, trigger the brake system, perform emergency braking, maintain the sound and light alarm, and send a shutdown report.

8. A system using the hoisting machinery anti-collision detection method according to any one of claims 1 to 7, characterized in that: It includes a data collection and preprocessing module (100), a data fusion and prediction module (200), and an instruction generation and execution module (300); The acquisition and preprocessing module (100) is used to acquire multimodal data of the lifting machinery and preprocess the multimodal data; The data fusion and prediction module (200) is used to fuse the collected multi-modal data through Kalman filtering, build an LSTM-GRU prediction model, and input the fused data into the LSTM-GRU prediction model to predict the operation trajectory of the lifting machinery; The instruction generation and execution module (300) is used to dynamically generate anti-collision control instructions through an AI edge computing device according to the trajectory prediction result, and the PLC receives and executes the anti-collision control instructions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the lifting machinery anti-collision detection method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lifting machinery anti-collision detection method according to any one of claims 1 to 7 are implemented.

Images