Tyre crane anti-collision and identification method fusing GNSS differential data

Abstract

The invention relates to a tire crane anti-collision and identification method fused with GNSS differential data, which realizes highly automatic and accurately controlled tire crane operation by installing a sensor on a tire crane, improves the working efficiency and safety, acquires GNSS signals, obtains the GNSS differential data and lane line information after processing, and improves the accuracy of the tire crane anti-collision and identification method. And the received data of the three-dimensional laser radar and the binocular camera are filtered and denoised, so that the positioning precision and the data quality are improved, and a reliable basis is provided for subsequent data processing. The deep learning technology is used for processing data collected by the binocular camera, obstacle and lane line information is extracted, the dynamic model is combined for predicting the motion state of the tire crane, and the prediction precision and the environment sensing capacity of the system are improved. Through a three-dimensional visual image identification technology and point cloud data processing, the position and the type of an obstacle are identified, and an early warning is given out when a potential collision is detected, so that the safety is improved, and accidents are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of tire crane monitoring, and in particular relates to a tire crane anti-collision and identification method integrating GNSS differential data. Background Art

[0002] Due to its strong mobility and wide adaptability, rubber-tyred cranes are widely used in container loading and unloading operations at ports and docks. They can move flexibly in the dock area to stack and load and unload containers.

[0003] After searching, a tire crane anti-collision control system based on a GNSS positioning system was disclosed with a publication number of CN101464688B, and the publication date was 2011-05-18. The product uses components such as a GNSS reference station subsystem, a TOS interface subsystem, a GNSS mobile station subsystem, and a center subsystem. Although it can achieve anti-collision control based on GNSS positioning, it is easily affected by satellite signal interference and multipath effects because it relies on GNSS positioning technology, resulting in insufficient positioning accuracy and reliability under complex terrain and variable working conditions.

[0004] In addition, the prior art still has the following problems: 1. High dependence on manual operation: The existing tire crane correction mainly relies on manual operation of the driver. The driver needs to keep a close eye on the position of the tire crane in the lane at all times, which is labor-intensive, low-efficiency, and prone to safety accidents due to misjudgment or improper operation; Poor environmental adaptability: Ground marking and image recognition correction methods cannot work normally in rainy and snowy weather. Line defects have a great impact on recognition. Infrared or photoelectric ranging correction methods are seriously interfered by external light and are difficult to correctly reflect the actual position of the equipment.

[0005] 2. Technical limitations: The obstacles that can be protected by the collision avoidance method based on two-dimensional and three-dimensional lidar are limited. They must be high enough and within the detection plane. They are also easily affected by noise points, have poor environmental adaptability, and have frequent false alarms.

[0006] 3. Sensor fusion problem: Although there is a three-dimensional collision avoidance method for tire cranes based on the fusion of three-dimensional lidar and binocular camera, how to effectively fuse the two sensor data to improve the degree of automation and accuracy of detection is still a technical challenge.

[0007] 4. False alarm and missed detection problems: The data features provided by a single sensor are single, resulting in high false detection and missed detection rates, which reduces the applicability and accuracy of the tire crane's anti-collision function.

[0008] The above problems show that the current tire crane anti-collision and control products on the market are difficult to effectively respond to the new requirements for high reliability and real-time response under complex terrain and variable working conditions. Therefore, the present invention provides a tire crane anti-collision and identification method integrating GNSS differential data to overcome these shortcomings and provide a new solution that is more intelligent, efficient and adaptable to variable environments. Summary of the invention

[0009] The present invention provides a tire crane anti-collision and identification method integrating GNSS differential data, which solves the problems of unstable performance, complex operation, slow response speed, insufficient adaptability and reliability of existing tire crane anti-collision products under complex terrain and variable working conditions. The present invention aims to provide a new solution that is more intelligent, efficient and adaptable to variable environments, so as to effectively improve the safety and reliability of tire cranes under complex terrain and variable working conditions.

[0010] The technical solution adopted by the present invention to solve the above technical problems is: a tire crane anti-collision and identification method integrating GNSS differential data, comprising the following steps: S1: System initialization and sensor integration: installing and integrating GNSS receiver, 3D LiDAR, binocular camera and IMU on the RTG, and ensuring they are synchronized with the RTG control system; S2: Data collection and preprocessing: collect GNSS signals, obtain GNSS differential data and lane information after processing, and filter and denoise the received data from 3D laser radar and binocular camera; S3: Sensor data fusion and processing: using deep learning technology to process the data collected by the binocular camera, extracting obstacle and lane line information, and combining the dynamic model to predict the movement state of the tire crane to improve the prediction accuracy of the system; S4: Collision detection and warning, through 3D visual image recognition technology and point cloud data processing, it identifies the location and type of obstacles, issues a warning when a potential collision is detected, and stores the image and location information of the obstacle target for subsequent analysis; S5: Automatic deviation correction control, automatically adjusts the driving direction and speed of the tire crane to correct the deviation based on GNSS differential data and lane line information; S6: Dynamic path planning, simulating the movement state and surrounding environment of the tire crane, and dynamically planning the optimal driving path; S7: System feedback and optimization, collect system operation data, optimize the system, and improve response speed and accuracy. At the same time, use AI fusion methods to reduce the number of sensors, reduce the time required for automatic box placement, and increase the success rate of one-time box placement; S8: Emergency handling: When the system detects an emergency, it automatically activates the emergency braking system to prevent a collision, and reassesses the environment and plans a new safe path after the emergency is handled.

[0011] Preferably, in S1, the following steps are specifically included: S1.1: The GNSS receiver is installed on the top of the tire crane, the 3D laser radar and the binocular camera are installed on the front and side of the tire crane respectively, and the IMU is installed at the center of the tire crane; S1.2: Use a programmable controller (PLC) to coordinate the data of each sensor to achieve automated control. The PLC is connected to each sensor through wired or wireless communication to ensure real-time data transmission.

[0012] Preferably, in S2, the following steps are specifically included: S2.1: Collect GNSS signals and achieve centimeter-level positioning in combination with ground base stations. The ground base station is connected to the GNSS receiver on the tire crane through wireless communication to provide GNSS differential data and lane line information in real time; S2.2: Filter and denoise the data collected by the 3D laser radar and the binocular camera. The filtering algorithm is as follows: ; Among them, is the state vector. For a 3D laser radar, it mainly includes distance and angle information. For a binocular camera, it mainly includes the position and speed of the target in the image. is at the time step The a posteriori state estimate is the optimal estimate of the system state after obtaining the actual measurement value at time k; is at the time step The prior state estimate is the state predicted based on the state estimate at time k-1 and the system model before the actual measurement value at time k is obtained; K k Adjustment factor, used to determine the weight between the measured value and the predicted value; Z k is the actual measurement value at time step k, obtained from the data collected by the 3D laser radar and the stereo camera; The observation matrix maps the state variables to the measurement space and determines how the state variables affect the measurement values.

[0013] Preferably, in S3, it specifically includes: The video data collected by the binocular camera is processed using deep learning video detection and segmentation technology to extract obstacles and lane line information in the environment. Deep learning specifically uses the YOLO detection model, and its loss function is: ; Among them, L in the above formula is the total loss. The smaller the value of the total loss L is, the closer the obstacles and lane lines predicted by the model are to the actually marked obstacles and lane lines, and the better the performance of the model is. is a weight parameter used to balance the contribution of coordinate loss in the total loss; it controls the degree of influence of the accuracy of bounding box position prediction on the overall loss; Indicates the number of bounding boxes predicted by each grid unit in the target detection task. Different grid units correspond to different numbers of predicted bounding boxes, and different numbers of bounding boxes correspond to dynamically changing scenes and obstacles. B Represents the maximum number of bounding boxes that can be predicted in each grid cell to cover multiple obstacles and lane lines that may appear. B The value of needs to be large enough; is an indicator function, when the grid cell and bounding box When there is a target in , its value is 1, otherwise it is 0, so that only the bounding box containing obstacle and lane line information contributes to the loss function; and Represents the true value of the x-coordinate and y-coordinate of the center of the bounding box respectively; and Represents the predicted values ​​of the x-coordinate and y-coordinate of the center of the bounding box respectively; and Represent the true values ​​of the width and height of the bounding box, respectively. and Help the model understand the size of obstacles and lane lines; and Represent the predicted values ​​of the width and height of the bounding box respectively; is another weight parameter that identifies areas without obstacles or lane lines; The true value representing the confidence of the object existing in the bounding box; The predicted value representing the confidence level of the object in the bounding box; Indicates that the bounding box belongs to the category c The true value of the probability of Indicates that the bounding box belongs to the category c The predicted value of the probability; classes is the total number of obstacle and lane line information categories.

[0014] Preferably, in S3, it further includes: The dynamic model technology is combined to model and predict the motion state of the tire crane, predict the future position and driving trajectory of the tire crane, and improve the prediction accuracy of the system. The GRU model is specifically used, and its equation is as follows: ; Among them, To update the value of the gate, it determines how much of the previous hidden state The hidden state that will be retained until the current moment middle; is the weight matrix of the update gate, is the bias term of the update gate, and σ is the sigmoid activation function, which limits the output value between 0 and 1; To reset the value of the gate, it determines how much of the previous hidden state will be reset to 0; is the weight matrix of the reset gate, is the bias term for the reset gate; is the value of the candidate hidden state, which is a candidate for a new hidden state and is used partially or completely to update the hidden state at the current moment ; is the weight matrix of the candidate hidden states, is the bias term for the candidate hidden state; tanh is the hyperbolic tangent activation function, which limits the output value to between -1 and 1; the symbol ⊙ represents the Hadamard product (element-wise multiplication); According to the value of the update gate To update the hidden state at the current moment , is the hidden state vector at time step t, which represents the internal state of the tire crane, including the predicted position and speed information of the tire crane; is the input vector at time step t, which contains the current speed, acceleration, and direction information of the tire crane.

[0015] Preferably, in S4, the following steps are specifically included: S4.1: Solve the obstacle avoidance and warning problems of large mechanical equipment such as tire cranes through 3D visual image recognition technology. After the data collected by the 3D laser radar and the binocular camera are fused, a 3D point cloud map is generated. The location and type of obstacles are identified through point cloud clustering and segmentation algorithms. The point cloud clustering uses the DBSCAN algorithm, and its parameters include the minimum number of points. and neighborhood radius minPts; S4.2: Realize the storage of obstacle target alarm images to provide traceability for accident analysis. The image and location information of the obstacle target are stored in the on-board memory for post-analysis and troubleshooting.

[0016] Preferably, in S5, the following steps are specifically included: S5.1: Automatically adjust the travel direction and speed of the tire crane according to the GNSS differential data and lane line information to correct the deviation. The central processor calculates the current position and deviation of the tire crane according to the GNSS differential data and lane line information, and adjusts the travel direction and speed of the tire crane through the PID controller; S5.2: Use BP neural network to identify the images collected by the binocular camera, simplify them into discrete points, and perform straight line detection through the hybrid Hough transform and least squares method to formulate a correction strategy.

[0017] Preferably, in S6, specifically including: Predict and simulate the trajectory and position of the box, dynamically plan the optimal path, simulate the movement state and surrounding environment of the tire crane in real time, and calculate the optimal driving path through the optimization algorithm. The cost function is as follows: ; in, The total cost of the node, which means the total cost of the node from the starting point to the target point. The estimated cost of, in the scenario of digital twin simulating a tire crane, this represents the total cost from the current position of the tire crane to the target position; In the above formula is the actual cost from the starting point to the node, which is specifically the actual driving distance or time cost of the tire crane moving from the initial position to the current node position; In the above formula The estimated cost from the node to the target point is a heuristic estimate used to predict the shortest path cost from the current node to the target node. It is based on the shortest estimated distance from the current position of the tire crane to the target position, calculated using Euclidean distance or Manhattan distance. In the above formula Represents the current node. Each node represents the state of the tire crane at a specific location and time, including its position coordinates, speed, and direction.

[0018] Preferably, in S7, the following steps are specifically included: S7.1: Collect system operation data, optimize the system, and improve the system's response speed and accuracy. The data collection module records sensor data and control instructions in real time, and optimizes the algorithm through big data analysis to improve the system's performance and reliability; S7.2: Use AI fusion methods to reduce the number of spreader sensors, reduce the time required for automatic box placement, and increase the success rate of one-time box placement. AI fusion methods improve the system's perception ability and decision-making accuracy through multi-modal data fusion.

[0019] Preferably, in S8, the following steps are specifically included: S8.1: When the system detects an emergency, the emergency braking system is automatically activated to prevent collision. The emergency braking system includes a brake control unit and a brake. The brake control unit quickly activates the brake according to the instructions of the central processor to ensure the safety of the tire crane in an emergency. S8.2: After handling the emergency, the system will re-evaluate the environment and plan a new safe path. The re-evaluation module re-collects environmental data and calculates a new driving path to ensure the safety and efficiency of the tire crane.

[0020] Beneficial effects of the present invention: The present invention realizes the automation, intelligence and safety of the operation of the tire crane by integrating multiple sensors, applying deep learning technology, dynamic models, optimization algorithms, etc. Specifically, the method improves the positioning accuracy, data quality, prediction accuracy, obstacle detection capability, path planning efficiency, system response speed and accuracy, as well as the safety and emergency response capability in emergency situations of the tire crane. Through the integrated application of these technologies, the operating efficiency and safety of the tire crane are significantly improved, the operating cost is reduced, and the reliability and economy of the system are enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0023] It should be noted that if the description and claims of the present application and the above-mentioned drawings relate to the terms "first", "second", etc., they are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, if the terms "including" and "having" and any of their variations are involved, it is intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0024] In this application, if the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inside", "outside", "middle", "vertical", "horizontal", "lateral", "longitudinal", etc. are involved, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings. These terms are mainly used to better describe the present application and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation.

[0025] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to specific circumstances.

[0026] In addition, in this application, the terms "installed", "set", "provided with", "connected", "connected", "socketed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection, or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, elements or components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] The embodiment of the present invention provides a tire crane anti-collision and identification method integrating GNSS differential data, which solves the problems of unstable performance, complex operation, slow response speed, insufficient adaptability and reliability of existing tire crane anti-collision products under complex terrain and variable working conditions. The present invention aims to provide a new solution that is more intelligent, efficient and adaptable to variable environments, so as to effectively improve the safety and reliability of tire cranes under complex terrain and variable working conditions.

[0029] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] See also Figure 1 A tire crane anti-collision and identification method integrating GNSS differential data comprises the following steps: S1: System initialization and sensor integration, installing and integrating GNSS receivers, 3D LiDAR, binocular cameras and IMU on the RTG, and ensuring they are synchronized with the RTG control system; specifically including: S1.1: The GNSS receiver is installed on the top of the tire crane, the 3D laser radar and the binocular camera are installed on the front and side of the tire crane respectively, and the IMU is installed at the center of the tire crane; S1.2: Use a programmable controller (PLC) to coordinate the data of each sensor to achieve automatic control. The PLC is connected to each sensor through wired or wireless communication to ensure real-time data transmission; In this embodiment, by installing and integrating a GNSS receiver, a three-dimensional lidar, a binocular camera and an IMU on the tire crane and ensuring that they are synchronized with the control system of the tire crane, highly automated and precisely controlled tire crane operations are achieved, thereby improving operating efficiency and safety.

[0031] S2: Data collection and preprocessing: collect GNSS signals, obtain GNSS differential data and lane information after processing, and filter and denoise the received 3D laser radar and binocular camera data; specifically includes the following steps: S2.1: Collect GNSS signals and achieve centimeter-level positioning in combination with ground base stations. The ground base station is connected to the GNSS receiver on the tire crane through wireless communication to provide GNSS differential data and lane line information in real time; S2.2: Filter and denoise the data collected by the 3D laser radar and the binocular camera. The filtering algorithm is as follows: ; Among them, is the state vector. For a 3D laser radar, it mainly includes distance and angle information. For a binocular camera, it mainly includes the position and speed of the target in the image. is at the time step The a posteriori state estimate is the optimal estimate of the system state after obtaining the actual measurement value at time k; is at the time step The prior state estimate is the state predicted based on the state estimate at time k-1 and the system model before the actual measurement value at time k is obtained; K k Adjustment factor, used to determine the weight between the measured value and the predicted value; Z k is the actual measurement value at time step k, obtained from the data collected by the 3D laser radar and the stereo camera; The observation matrix maps the state variables to the measurement space and determines how the state variables affect the measurement values.

[0032] In this embodiment, this equation combines the predicted state estimate with the actual measurement value to produce a more accurate state estimate and quantify the uncertainty of the estimate. In this way, the noise in the 3D laser radar and binocular camera data can be effectively reduced, providing more reliable data for subsequent processing. GNSS signals are collected and combined with ground base stations to achieve centimeter-level positioning, which improves the accuracy of positioning and provides accurate basic data for subsequent path planning and obstacle detection. The data collected by the 3D laser radar and binocular camera are filtered and denoised, which reduces the noise in the data, improves the reliability of the data, and provides high-quality input for subsequent data processing.

[0033] S3: Sensor data fusion and processing, using deep learning technology to process the data collected by the binocular camera, extracting obstacle and lane line information, and combining the dynamic model to predict the movement state of the tire crane to improve the prediction accuracy of the system; specifically including: The video data collected by the binocular camera is processed using deep learning video detection and segmentation technology to extract obstacles and lane line information in the environment. Deep learning specifically uses the YOLO detection model, and its loss function is: ; Among them, L in the above formula is the total loss. The smaller the value of the total loss L is, the closer the obstacles and lane lines predicted by the model are to the actually marked obstacles and lane lines, and the better the performance of the model is. is a weight parameter used to balance the contribution of coordinate loss in the total loss; it controls the degree of influence of the accuracy of bounding box position prediction on the overall loss; Indicates the number of bounding boxes predicted by each grid unit in the target detection task. Different grid units correspond to different numbers of predicted bounding boxes, and different numbers of bounding boxes correspond to dynamically changing scenes and obstacles. B Represents the maximum number of bounding boxes that can be predicted in each grid cell to cover multiple obstacles and lane lines that may appear. B The value of needs to be large enough; is an indicator function, when the grid cell and bounding box When there is a target in , its value is 1, otherwise it is 0, so that only the bounding box containing obstacle and lane line information contributes to the loss function; and Represents the true value of the x-coordinate and y-coordinate of the center of the bounding box respectively; and Represents the predicted values ​​of the x-coordinate and y-coordinate of the center of the bounding box respectively; and Represent the true values ​​of the width and height of the bounding box, respectively. and Help the model understand the size of obstacles and lane lines; and Represent the predicted values ​​of the width and height of the bounding box respectively; is another weight parameter that identifies areas without obstacles or lane lines; The true value representing the confidence of the object existing in the bounding box; The predicted value representing the confidence level of the object in the bounding box; Indicates that the bounding box belongs to the category c The true value of the probability of Indicates that the bounding box belongs to the category c The predicted value of the probability; classes is the total number of obstacle and lane line information categories.

[0034] In this embodiment, deep learning video detection and segmentation technology is used to process the video data collected by the binocular camera to extract obstacle and lane line information in the environment, thereby improving the accuracy of obstacle detection and the ability to recognize lane lines. The YOLO detection model is used to optimize the model performance through a specific loss function, thereby improving the accuracy and efficiency of target detection.

[0035] In step S3, it also includes: The dynamic model technology is combined to model and predict the motion state of the tire crane, predict the future position and driving trajectory of the tire crane, and improve the prediction accuracy of the system. The GRU model is specifically used, and its equation is as follows: ; Among them, To update the value of the gate, it determines how much of the previous moment's hidden state The hidden state that will be retained until the current moment middle; is the weight matrix of the update gate, is the bias term of the update gate, and σ is the sigmoid activation function, which limits the output value between 0 and 1; To reset the value of the gate, it determines how much of the previous hidden state will be reset to 0; is the weight matrix of the reset gate, is the bias term for the reset gate; is the value of the candidate hidden state, which is a candidate for a new hidden state and is used partially or completely to update the hidden state at the current moment ; is the weight matrix of the candidate hidden states, is the bias term for the candidate hidden state; tanh is the hyperbolic tangent activation function, which limits the output value to between -1 and 1; the symbol ⊙ represents the Hadamard product (element-wise multiplication); According to the value of the update gate To update the hidden state at the current moment , is the hidden state vector at time step t, which represents the internal state of the tire crane, including the predicted position and speed information of the tire crane; is the input vector at time step t, which contains the current speed, acceleration, and direction information of the tire crane.

[0036] In this embodiment, the motion state of the tire crane is modeled and predicted in combination with dynamic model technology, which improves the prediction accuracy of the system and makes the operation of the tire crane safer and more efficient. The GRU model is used to effectively process sequence data through the update gate and reset gate mechanism, thereby improving the accuracy of the prediction of the motion state of the tire crane.

[0037] S4: Collision detection and warning, through 3D visual image recognition technology and point cloud data processing, identify the location and type of obstacles, and issue a warning when a potential collision is detected, while storing the image and location information of the obstacle target for subsequent analysis; specifically includes the following steps: S4.1: Solve the obstacle avoidance and warning problems of large mechanical equipment such as tire cranes through 3D visual image recognition technology. After the data collected by the 3D laser radar and the binocular camera are fused, a 3D point cloud map is generated. The location and type of obstacles are identified through point cloud clustering and segmentation algorithms. The point cloud clustering uses the DBSCAN algorithm, and its parameters include the minimum number of points. and neighborhood radius minPts; S4.2: Realize the storage of obstacle target alarm images to provide traceability for accident analysis. The image and location information of the obstacle target are stored in the on-board memory for post-analysis and troubleshooting.

[0038] In this embodiment, the obstacle avoidance and warning problems of large mechanical equipment such as tire cranes are solved by three-dimensional visual image recognition technology, which improves the accuracy of obstacle detection and the timeliness of warning. The DBSCAN algorithm is used for point cloud clustering and segmentation, which improves the ability to identify the location and type of obstacles and provides reliable data support for collision warning.

[0039] S5: Automatic deviation correction control, automatically adjusts the driving direction and speed of the tire crane according to GNSS differential data and lane line information to correct the deviation; specifically includes the following steps: S5.1: Automatically adjust the travel direction and speed of the tire crane according to the GNSS differential data and lane line information to correct the deviation. The central processor calculates the current position and deviation of the tire crane according to the GNSS differential data and lane line information, and adjusts the travel direction and speed of the tire crane through the PID controller; S5.2: Use BP neural network to identify the images collected by the binocular camera, simplify them into discrete points, and perform straight line detection through the hybrid Hough transform and least squares method to formulate a correction strategy.

[0040] In this embodiment, the driving direction and speed of the tire crane are automatically adjusted according to the GNSS differential data and lane line information to correct the deviation, thereby improving the automation level and accuracy of the operation. The images collected by the binocular camera are recognized by the BP neural network, simplified into discrete points, and straight line detection is performed by hybrid Hough transform and least square method, thereby improving the accuracy and efficiency of the deviation correction strategy.

[0041] S6: Dynamic path planning, simulating the movement state and surrounding environment of the tire crane, and dynamically planning the optimal driving path; specifically including: Predict and simulate the trajectory and position of the box, dynamically plan the optimal path, simulate the movement state and surrounding environment of the tire crane in real time, and calculate the optimal driving path through the optimization algorithm. The cost function is as follows: ; in, The total cost of the node, which means the total cost of the node from the starting point to the target point. The estimated cost of, in the scenario of digital twin simulating a tire crane, this represents the total cost from the current position of the tire crane to the target position; In the above formula is the actual cost from the starting point to the node, which is specifically the actual driving distance or time cost of the tire crane moving from the initial position to the current node position; In the above formula The estimated cost from the node to the target point is a heuristic estimate used to predict the shortest path cost from the current node to the target node. It is based on the shortest estimated distance from the current position of the tire crane to the target position, calculated using Euclidean distance or Manhattan distance. In the above formula Represents the current node. Each node represents the state of the tire crane at a specific location and time, including its position coordinates, speed, and direction.

[0042] In this embodiment, the trajectory and position of the box are predicted and simulated, and the optimal path is dynamically planned, which improves the efficiency and adaptability of path planning, making the operation of the tire crane safer and more efficient. The optimal driving path is calculated by the optimization algorithm, which improves the accuracy and efficiency of path planning.

[0043] S7: System feedback and optimization, collect system operation data, optimize the system, and improve response speed and accuracy. At the same time, use AI fusion methods to reduce the number of sensors, reduce the time required for automatic box placement, and increase the success rate of one-time box placement; specifically, the following steps are included: S7.1: Collect system operation data, optimize the system, and improve the system's response speed and accuracy. The data collection module records sensor data and control instructions in real time, and optimizes the algorithm through big data analysis to improve the system's performance and reliability; S7.2: Use AI fusion methods to reduce the number of spreader sensors, reduce the time required for automatic box placement, and increase the success rate of one-time box placement. AI fusion methods improve the system's perception ability and decision-making accuracy through multi-modal data fusion.

[0044] S8: Emergency handling: When the system detects an emergency, it automatically activates the emergency braking system to prevent a collision, and reassesses the environment and plans a new safe path after the emergency is handled; specifically, it includes the following steps: S8.1: When the system detects an emergency, the emergency braking system is automatically activated to prevent collision. The emergency braking system includes a brake control unit and a brake. The brake control unit quickly activates the brake according to the instructions of the central processor to ensure the safety of the tire crane in an emergency. S8.2: After handling the emergency, the system will re-evaluate the environment and plan a new safe path. The re-evaluation module re-collects environmental data and calculates a new driving path to ensure the safety and efficiency of the tire crane.

[0045] In this embodiment, when the system detects an emergency, the emergency braking system is automatically activated to prevent a collision, and the environment is reassessed and a new safe path is planned after the emergency is handled, thereby enhancing the safety and emergency response capabilities of the system.

[0046] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A tire crane anti-collision and identification method integrating GNSS differential data, characterized in that: The following steps are involved: S1: System initialization and sensor integration: installing and integrating GNSS receiver, 3D LiDAR, binocular camera and IMU on the tire crane; S2: Data collection and preprocessing: collect GNSS signals, obtain GNSS differential data and lane information after processing, and filter and denoise the received data from 3D laser radar and binocular camera; S3: Sensor data fusion and processing, using deep learning technology to process the data collected by the binocular camera, extract obstacle and lane line information, and combine the dynamic model to predict the movement state of the tire crane; S4: Collision detection and warning, through 3D visual image recognition technology and point cloud data processing, it identifies the location and type of obstacles, issues a warning when a potential collision is detected, and stores the image and location information of the obstacle target; S5: Automatic deviation correction control, automatically adjusts the travel direction and speed of the tire crane according to GNSS differential data and lane line information; S6: Dynamic path planning, simulating the movement state and surrounding environment of the tire crane, and dynamically planning the optimal driving path; S7: System feedback and optimization, collect system operation data, optimize the system, and use AI fusion methods to reduce the number of sensors; S8: Emergency handling: When the system detects an emergency, it automatically activates the emergency braking system to prevent a collision, and reassesses the environment and plans a new safe path after the emergency is handled.

2. A tire crane anti-collision and identification method integrating GNSS differential data according to claim 1, characterized in that: In S1, the following steps are specifically included: S1.1: The GNSS receiver is installed on the top of the tire crane, the 3D laser radar and the binocular camera are installed on the front and side of the tire crane respectively, and the IMU is installed at the center of the tire crane; S1.2: Use a programmable controller PLC to coordinate the data of each sensor, and the PLC is connected to each sensor through wired or wireless communication.

3. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1 is characterized in that: In S2, the following steps are specifically included: S2.1: Collect GNSS signals and achieve centimeter-level positioning in combination with ground base stations. The ground base station is connected to the GNSS receiver on the tire crane through wireless communication to provide GNSS differential data and lane line information in real time; S2.2: Filter and denoise the data collected by the 3D laser radar and the binocular camera. The filtering algorithm is as follows: ； Among them, is the state vector. For a 3D laser radar, it mainly includes distance and angle information. For a binocular camera, it mainly includes the position and speed of the target in the image. is at the time step The a posteriori state estimate is the optimal estimate of the system state after obtaining the actual measurement value at time k; is at the time step The prior state estimate is the state predicted based on the state estimate at time k-1 and the system model before the actual measurement value at time k is obtained; K k Adjustment factor, used to determine the weight between the measured value and the predicted value; Z k is the actual measurement value at time step k, obtained from the data collected by the 3D laser radar and the stereo camera; The observation matrix maps the state variables to the measurement space and determines how the state variables affect the measurement values.

4. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1 is characterized in that: In S3, it specifically includes: The video data collected by the binocular camera is processed using deep learning video detection and segmentation technology to extract obstacles and lane line information in the environment. Deep learning specifically uses the YOLO detection model, and its loss function is: ； Among them, L in the above formula is the total loss. The smaller the value of the total loss L is, the closer the obstacles and lane lines predicted by the model are to the actually marked obstacles and lane lines, and the better the performance of the model is. is a weight parameter used to balance the contribution of coordinate loss in the total loss; Indicates the number of bounding boxes predicted by each grid unit in the target detection task. Different grid units correspond to different numbers of predicted bounding boxes, and different numbers of bounding boxes correspond to dynamically changing scenes and obstacles. B Represents the maximum number of bounding boxes that can be predicted in each grid cell to cover multiple obstacles and lane lines that may appear. B The value of needs to be large enough; is an indicator function, when the grid cell and bounding box When there is a target in , its value is 1, otherwise it is 0, so that only the bounding box containing obstacle and lane line information contributes to the loss function; and Represents the true value of the x-coordinate and y-coordinate of the center of the bounding box respectively; and Represents the predicted values ​​of the x-coordinate and y-coordinate of the center of the bounding box respectively; and Represent the true values ​​of the width and height of the bounding box, respectively. and Help the model understand the size of obstacles and lane lines; and Represent the predicted values ​​of the width and height of the bounding box respectively; is another weight parameter that identifies areas without obstacles or lane lines; The true value representing the confidence of the object existing in the bounding box; The predicted value representing the confidence level of the object in the bounding box; Indicates that the bounding box belongs to the category c The true value of the probability of Indicates that the bounding box belongs to the category c The predicted value of the probability; classes is the total number of obstacle and lane line information categories.

5. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 4 is characterized in that: In S3, it also includes: The dynamic model technology is combined to model and predict the motion state of the tire crane, and predict the future position and driving trajectory of the tire crane. The GRU model is specifically used, and its equation is as follows: ； Among them, To update the value of the gate, it determines how much of the previous moment's hidden state The hidden state that will be retained until the current moment middle; is the weight matrix of the update gate, is the bias term of the update gate, and σ is the sigmoid activation function, which limits the output value between 0 and 1; To reset the value of the gate, it determines how much of the previous hidden state will be reset to 0; is the weight matrix of the reset gate, is the bias term for the reset gate; is the value of the candidate hidden state, which is a candidate for a new hidden state and is used partially or completely to update the hidden state at the current moment ; is the weight matrix of the candidate hidden states, is the bias term of the candidate hidden state. tanh is the hyperbolic tangent activation function, which limits the output value to between -1 and 1. The symbol ⊙ represents the Hadamard product. According to the value of the update gate To update the hidden state at the current moment , is the hidden state vector at time step t, which represents the internal state of the tire crane, including the predicted position and speed information of the tire crane; is the input vector at time step t, which contains the current speed, acceleration, and direction information of the tire crane.

6. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1, characterized in that: In S4, the following steps are specifically included: S4.1: After the data collected by the 3D laser radar and the binocular camera are fused, a 3D point cloud is generated. The location and type of obstacles are identified through point cloud clustering and segmentation algorithms. The point cloud clustering uses the DBSCAN algorithm, and its parameters include the minimum number of points and neighborhood radius minPts; S4.2: Realize obstacle target alarm image storage. The image and position information of the obstacle target are stored in the vehicle memory for subsequent analysis and troubleshooting.

7. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1, characterized in that: In S5, the following steps are specifically included: S5.1: Automatically adjust the travel direction and speed of the tire crane according to the GNSS differential data and lane line information; the central processor calculates the current position and deviation of the tire crane according to the GNSS differential data and lane line information, and adjusts the travel direction and speed of the tire crane through the PID controller; S5.2: Use BP neural network to identify the images collected by the binocular camera, simplify them into discrete points, and perform straight line detection through the hybrid Hough transform and least squares method to formulate a correction strategy.

8. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1, characterized in that: In S6, it specifically includes: Predict and simulate the trajectory and position of the box, dynamically plan the optimal path, simulate the movement state and surrounding environment of the tire crane in real time, and calculate the optimal driving path through the optimization algorithm. The cost function is as follows: ； in, The total cost of the node, which means the total cost of the node from the starting point to the target point. The estimated cost of, in the scenario of digital twin simulating a tire crane, this represents the total cost from the current position of the tire crane to the target position; In the above formula is the actual cost from the starting point to the node, which is specifically the actual driving distance or time cost of the tire crane moving from the initial position to the current node position; In the above formula The estimated cost from the node to the target point is a heuristic estimate used to predict the shortest path cost from the current node to the target node. It is based on the shortest estimated distance from the current position of the tire crane to the target position, calculated using Euclidean distance or Manhattan distance. In the above formula Represents the current node. Each node represents the state of the tire crane at a specific location and time, including its position coordinates, speed, and direction.

9. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1, characterized in that: In S7, the following steps are specifically included: S7.1: Collect system operation data and optimize the system; the data collection module records sensor data and control instructions in real time, and optimizes the algorithm through big data analysis; S7.2: Use AI fusion method to reduce the number of spreader sensors; AI fusion method is through multimodal data fusion.

10. The tire crane anti-collision and identification method integrating GNSS differential data according to claim 1, characterized in that: In S8, the following steps are specifically included: S8.1: When the system detects an emergency, the emergency braking system is automatically activated. The emergency braking system includes a brake control unit and a brake. The brake control unit quickly activates the brake according to the instructions of the central processor; S8.2: After handling the emergency, the system will re-evaluate the environment and plan a new safe path. The re-evaluation module calculates the new driving path by re-collecting environmental data.

Images