AGV real-time anti-collision early warning method and device based on deep learning

Abstract

The invention relates to the technical field of AGV real-time anti-collision, and particularly discloses an AGV real-time anti-collision early warning method and device based on deep learning, and the method comprises the steps: collecting road data through a data collection device, and carrying out the marking of the road data through a marking tool; marking the road surface data by using a marking tool, wherein the road surface data comprise road signs, road lines, sidewalks and the like; the road segmentation model is improved based on the improved PSPNet road segmentation model, and the model performance is improved; carrying out real-time calculation on the AGV driving road safety area by using a pixel-level-based road area calculation method; carrying out real-time discrimination on AGV driving road safety by using a safety discrimination model based on pressure change, and carrying out decision making; the AGV real-time anti-collision early warning device based on deep learning is used for carrying out hardware deployment support on the AGV real-time anti-collision early warning method based on deep learning, so that the method is applied.

Description

Technical Field

[0001] The present invention relates to the field of AGV real-time collision avoidance technology, and in particular to a deep learning-based AGV real-time collision avoidance warning method and device. Background Art

[0002] As a key technology in modern industry and logistics, automatic guided vehicles (AGVs) are widely used in various fields to improve transportation efficiency and work safety. In industrial automation, AGVs play an important role from raw material transportation to finished product distribution. They can drive autonomously according to preset paths and programs, perform material handling, assembly and robot-assisted production line operations, thereby greatly improving production efficiency and factory operation flexibility. In warehousing and logistics management, AGVs are used for automated cargo handling and in-warehouse operations. They can operate safely in narrow passages and high-density storage areas, reduce the time and error rate of manual operations, and improve the accuracy and speed of logistics operations. In addition to the industrial field, AGVs also have important applications in medical facilities and service industries. In hospitals, they are used to safely transport drugs, medical equipment and medical records, reduce personnel contact and infection risks, and improve hospital operation efficiency and service quality. In the field of electronic manufacturing, AGVs are widely used in semiconductor production lines and electronic product assembly plants, and through automated transportation and assembly processes, the flexibility and response speed of production lines are improved. The application of AGV is not limited to the traditional industrial manufacturing and logistics fields. They are becoming a key component of modern factories and warehouses, driving the innovation of intelligent manufacturing and logistics management. With the continuous development of technology and the expansion of application scenarios, AGV will continue to play an important role in automation, safety and efficiency improvement, and become an important driving force for the intelligent development of industry and logistics in the future.

[0003] With the increase in the number and application scenarios of AGVs, how to effectively avoid collisions has become a technical challenge that needs to be solved urgently. The importance of anti-collision warning technology is self-evident, and it is directly related to the safety and reliability of AGVs in complex environments. Traditional anti-collision systems usually rely on devices such as laser radar, infrared sensors or ultrasonic sensors to detect surrounding obstacles and avoid paths. Laser radar is a sensor technology based on optical principles. When laser radar works, it determines the position and distance of surrounding objects by emitting a laser beam and measuring the time it takes for it to reflect back. The working principle is similar to sonar, but it uses light pulses. Laser radar can provide high-precision environmental maps and obstacle location information. Its accuracy and detection distance are usually high, and it is suitable for precise navigation and collision avoidance operations in complex environments. Infrared sensors use the principle of infrared radiation to detect the existence and distance of objects. Infrared sensors emit infrared rays and measure the reflected signals to determine the distance and position of surrounding objects. They are usually cheaper and easier to integrate, and are suitable for close-range obstacle detection. However, infrared sensors are greatly affected by ambient light interference, and their detection range and accuracy are relatively limited compared to laser radar and ultrasonic sensors. Ultrasonic sensors are sensors that use the reflection of ultrasound to measure distance. Ultrasonic sensors emit ultrasonic pulses and measure the time it takes for the pulses to reflect back, thereby calculating the distance between the object and the sensor. The advantages of ultrasonic sensors are low cost and low energy consumption, and they are suitable for close-range obstacle detection and collision avoidance. However, their accuracy and detection range are usually lower than those of lidar and infrared sensors, and are greatly affected by the shape and surface characteristics of the object. However, these systems often fail to provide sufficient real-time response and accurate collision avoidance strategies when facing dynamic, dense or irregular working environments, which may lead to collision accidents. In recent years, AGV real-time collision avoidance warning systems based on deep learning have become the forefront of technology. Deep learning technology enables AGV to perceive the environment more intelligently, predict the movement trajectory of obstacles, and realize real-time decision-making and path planning through large-scale data learning and pattern recognition. Compared with traditional static rules, deep learning systems can dynamically adjust the driving path, effectively avoid potential collision risks, and improve the safety and reliability of the system.

[0004] The AGV real-time anti-collision warning system based on deep learning uses deep neural networks (DNN) as its core technology to achieve high-level perception and intelligent decision-making capabilities for complex environments by processing and analyzing multi-sensor data. In the AGV real-time anti-collision warning system, the application of deep learning mainly includes: data acquisition and preprocessing, feature extraction and learning, obstacle recognition and prediction, and real-time decision-making and path planning. Data acquisition and preprocessing mainly refers to the system obtaining environmental data through a variety of sensors. These sensors provide detailed information about the position, speed, size, and direction of movement of surrounding objects, which is used to construct input data. Feature extraction and learning: After preprocessing, the acquired sensor data is fed into the deep learning model. The deep learning model consists of multiple neural network layers, each of which is responsible for converting the input data into a higher-level feature representation. Through multiple iterations of training, the model can automatically learn and recognize complex features in the environment, such as the shape, movement trajectory, and possible interaction mode of different objects. Obstacle recognition and prediction: The trained deep learning model can accurately identify and classify obstacles in the surrounding environment, such as other vehicles, personnel, or fixed objects. More importantly, the model can predict the future movement trend and possible behavior of obstacles, thereby warning of potential collision risks in advance. Real-time decision-making and path planning: Based on the information output by the deep learning model, the AGV can adjust its driving path and speed in real time to avoid collisions with identified obstacles. This capability enables the AGV to navigate safely in complex, dynamically changing environments while maximizing transportation efficiency and work efficiency. The AGV real-time collision avoidance warning system based on deep learning has obvious advantages over traditional sensor-based collision avoidance methods. The deep learning model can learn and understand complex environmental characteristics from a large amount of data, and can identify and predict various types of obstacles and their behaviors, including stationary objects and dynamically moving objects. Compared with traditional sensor technology, deep learning can provide more accurate and reliable obstacle identification and prediction capabilities. This accuracy enables the AGV to more accurately assess the risk of collision and take effective collision avoidance measures. The deep learning model enables rapid data processing and decision generation, allowing the AGV to respond quickly to real-time environmental changes and adjust its driving strategy to ensure safety and efficiency. In summary, the deep learning-based AGV real-time collision avoidance warning system provides advanced and reliable solutions for the fields of industrial automation and logistics management through its high-level environmental perception capabilities and precise decision-making capabilities, and promotes the development of AGV systems towards higher intelligence and safety.

[0005] Based on this technical background, the present invention studies a real-time anti-collision warning method and device for AGV based on deep learning. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a real-time anti-collision warning method and device for AGV based on deep learning. The method uses a deep learning framework to build a road segmentation model based on PSPNet. Through deep learning technology, high-precision image semantic segmentation can be achieved, and roads can be accurately identified and classified, thereby providing more reliable warning information.

[0007] In order to achieve the above object, a first aspect of the present invention provides an AGV real-time anti-collision warning method based on deep learning, comprising:

[0008] Collect and annotate AGV datasets;

[0009] Improve the PSPNet road segmentation model to improve the performance of the model;

[0010] The pixel-level road area calculation method is used to calculate the safe area of ​​the AGV driving road in real time;

[0011] The safety judgment model based on pressure change is used to judge the road safety of AGV in real time and make decisions.

[0012] The second aspect of the present invention provides an AGV real-time anti-collision warning device based on deep learning adopted by the above method, including a drive motor, a shock absorbing device, a walking wheel, a camera, a lifting device, a frame, a battery and a pressure sensor.

[0013] The beneficial effects of the present invention include:

[0014] (1) The AGV real-time anti-collision warning method based on deep learning proposed in the present invention uses a deep learning framework to build a road segmentation model based on the improved PSPNet. Through deep learning technology, high-precision image semantic segmentation can be achieved, and roads can be accurately identified and classified, thereby providing more reliable warning information.

[0015] (2) The deep learning-based real-time anti-collision warning method for AGV proposed in this invention, PSPNet, can dynamically adjust the strategy in real time to optimize the input and parameter settings of the model according to the changes in the actual environment and the feedback of sensor data. This adaptive capability enables AGV to quickly respond to changes in complex traffic scenarios, thereby improving the flexibility and response capabilities of the system.

[0016] (3) Compared with traditional lidar, ultrasonic sensor and infrared sensor technologies, the deep learning-based AGV real-time anti-collision warning method proposed in the present invention has higher accuracy, wider environmental perception capabilities, stronger adaptability and intelligence level. These advantages enable AGV to operate more safely and reliably in complex traffic environments, bringing new development opportunities and technological progress to smart logistics and industrial automation.

[0017] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings.

[0019] Figure 1 This is a flow chart of the deep learning-based AGV real-time anti-collision warning method proposed in the present invention.

[0020] Figure 2 This is a schematic diagram of the hardware system of AGV real-time collision avoidance warning in a specific implementation of the AGV real-time collision avoidance warning method based on deep learning proposed by the present invention.

[0021] Figure 3 This is a flow chart of improving the road segmentation performance of the model in a specific implementation of the deep learning-based AGV real-time anti-collision warning method proposed in the present invention.

[0022] Figure 4 This is a schematic diagram of introducing the CBAM attention mechanism in a specific implementation of the deep learning-based AGV real-time anti-collision warning method proposed in the present invention.

[0023] Description of reference numerals:

[0024] 1-drive motor, 2-shock absorber, 3-travel wheel, 4-camera, 5-lifting device, 6. frame, 7-battery, 8-pressure sensor, 9-edge computing platform. DETAILED DESCRIPTION

[0025] The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0026] The present invention provides an AGV real-time anti-collision warning method based on deep learning, such as Figure 1 As shown, including:

[0027] Collect and annotate AGV datasets;

[0028] Improve the PSPNet road segmentation model to improve model performance;

[0029] The pixel-level road area calculation method is used to calculate the safe area of ​​the AGV driving road in real time;

[0030] The safety judgment model based on pressure change is used to judge the road safety of AGV in real time and make decisions.

[0031] In the present invention, a deep learning framework is used to build a road segmentation model based on the improved PSPNet. Through deep learning technology, high-precision image semantic segmentation can be achieved, and roads can be accurately identified and classified, thereby providing more reliable warning information.

[0032] In the present invention, collecting the AGV data set may include:

[0033] In actual road scenarios, cameras or sensor devices are used to record the road conditions of AGVs in real time to obtain AGV datasets.

[0034] Using computer simulation software, different road scenes and obstacles are generated in a virtual environment to obtain labeled image data as AGV data sets;

[0035] Use the existing public dataset as the AGV dataset. The public dataset marks the semantic information of roads, vehicles, and pedestrians.

[0036] Computer simulation software includes CARLA and Gazebo;

[0037] Existing public datasets include Cityscapes and KITTI.

[0038] According to the present invention, collecting and annotating AGV data sets includes:

[0039] Collecting AGV data sets using a data collection device;

[0040] Use annotation tools to annotate the collected AGV dataset;

[0041] The annotated AGV dataset includes road signs, lane lines, and sidewalks;

[0042] Annotation of the AGV dataset includes:

[0043] The boundaries and shapes of the road areas in the AGV dataset images are annotated to distinguish roads from other objects.

[0044] Improvements to the PSPNet road segmentation model include:

[0045] Use deep convolutional neural networks to train the labeled AGV dataset to achieve accurate segmentation of roads and obstacles;

[0046] Introducing CBAM or SE attention mechanism to enhance the model's ability to recognize small targets in complex road scenes;

[0047] Combined with multi-scale feature extraction technology, it is used to improve the detection performance of road elements of different scales;

[0048] Use data augmentation technology to expand the training data set to enhance the robustness and generalization ability of the model;

[0049] Post-process the model output, including morphological operations and conditional random fields, to optimize the boundaries and details of the segmentation results;

[0050] Through online learning mechanisms, the model is continuously updated to adapt to new scenarios and environmental changes.

[0051] like Figure 4 As shown in the figure, the CBAM attention mechanism is introduced into the backbone feature extraction network to improve the model's road segmentation performance.

[0052] According to the present invention, the real-time calculation of the safe area of ​​the AGV driving road using the pixel-level road area calculation method includes:

[0053] Calculate the pixel area of ​​the target road based on the segmented target area;

[0054] The real area of ​​the target road is inverted through the mapping model between the pixel area and the real road surface.

[0055] According to the present invention, a safety judgment model based on pressure change is used to judge the road safety of AGV in real time, and the decision-making includes:

[0056] The braking area of ​​the AGV under the current load is calculated by the pressure of the load on the AGV. The braking area under different loads is obtained through the mapping model between load and braking area, where the braking area includes the warning area and the safety area.

[0057] After the road is segmented, the segmented area is calculated using the segmented images;

[0058] When the segmented area is smaller than the set warning area, the AGV starts to slow down, otherwise it drives normally;

[0059] When the segmented area is smaller than the set safety area, the AGV starts braking, otherwise it drives normally.

[0060] In the present invention, the improved PSPNet can dynamically adjust the strategy in real time to optimize the input and parameter settings of the model according to the changes in the actual environment and the feedback of sensor data. This adaptive capability enables the AGV to quickly respond to changes in complex traffic scenarios, thereby improving the flexibility and response capabilities of the system.

[0061] In the present invention, the road segmentation model can be trained using the labeled AGV dataset, and the specific methods include:

[0062] Data loading: Load the labeled AGV dataset into the training environment;

[0063] Loss function definition: Define a loss function suitable for semantic segmentation tasks;

[0064] Optimizer selection: select the appropriate optimization algorithm to optimize and update the parameters of the road segmentation model;

[0065] The loss function is the cross entropy loss function;

[0066] The optimizer used in the optimization algorithm is the Adam optimizer.

[0067] According to the present invention, model verification of the trained road segmentation model includes:

[0068] Model reasoning: Use the data in the validation set to reason about the road segmentation model and generate prediction results;

[0069] Evaluation indicators: Calculate the accuracy, recall, and IoU between the prediction results and the standard answers to evaluate the performance of the road segmentation model;

[0070] Deploying the road segmentation model into an actual AGV system involves:

[0071] Model conversion: convert the road segmentation model after model verification into a format suitable for the target hardware and software environment;

[0072] Integration into AGV system: Integrate the road segmentation model into the AGV software platform to obtain image data in real time and make predictions;

[0073] Interface definition: define the interface and communication protocol between the road segmentation model and other system components;

[0074] Other system components include sensors and controllers.

[0075] In the present invention, the deployed road segmentation model can be tested and verified at the system level, and the specific methods include:

[0076] Functional testing: Verify whether the AGV system can capture image data, perform road segmentation and obstacle recognition in real time;

[0077] Performance testing: Evaluate the response speed, accuracy and robustness of the AGV system in different scenarios;

[0078] Safety test: Test the emergency response and safety control capabilities of the AGV system in the face of emergencies.

[0079] Compared with traditional lidar, ultrasonic sensor and infrared sensor technologies, the present invention has higher accuracy, wider environmental perception capabilities, stronger adaptability and intelligence level. These advantages enable AGV to operate more safely and reliably in complex traffic environments, bringing new development opportunities and technological progress to smart logistics and industrial automation.

[0080] The present invention also provides an AGV real-time anti-collision warning device based on deep learning adopted by the method described in the claim above, such as Figure 2 As shown, it includes a driving motor 1, a shock absorbing device 2, a walking wheel 3, a camera 4, a lifting device 5, a frame 6, a battery 7, a pressure sensor 8, and an edge computing platform 9.

[0081] The present invention will be described in more detail below by way of examples.

[0082] Embodiment 1:

[0083] This embodiment provides an AGV real-time anti-collision warning method based on deep learning, and the specific implementation steps are as follows:

[0084] 1. Dataset collection:

[0085] 1) In actual road scenarios, use cameras or sensor devices to record the road conditions of the AGV in real time during its driving process;

[0086] 2) Using computer simulation software, such as CARLA and Gazebo, to generate different road scenes and obstacles in a virtual environment to obtain a large amount of labeled image data;

[0087] 3) Use existing public datasets, such as Cityscapes and KITTI, which have labeled various semantic information such as roads, vehicles, and pedestrians;

[0088] 2. Dataset annotation: Annotate the boundaries and shapes of road areas in the image to distinguish roads from other objects;

[0089] 3. Model construction: Use deep learning frameworks (such as PyTorch, TensorFlow, etc.) to build a road segmentation model based on PSPNet; Based on the network structure of PSPNet, including the integration of pyramid pooling module and CBAM module; Define the size and number of channels of the input image, and configure the parameters and hyperparameters of the network layer;

[0090] 4. Model training: Use the labeled data set to train the model to learn the semantic segmentation ability of road scenes;

[0091] 1) Data loading: Load the prepared dataset into the training environment;

[0092] 2) Loss function definition: define a loss function suitable for semantic segmentation tasks, such as the cross entropy loss function;

[0093] 3) Optimizer selection: Select a suitable optimization algorithm, such as the Adam optimizer, for optimizing and updating model parameters;

[0094] 5. Model validation: During the training process, the model needs to be validated to evaluate its performance on the validation set;

[0095] 1) Model reasoning: Use the data in the validation set to reason about the model and generate prediction results;

[0096] 2) Evaluation indicators: Calculate the accuracy, recall, IoU (Intersection over Union) and other evaluation indicators between the model prediction results and the standard answers to evaluate the performance of the model;

[0097] 6. Model deployment: After model training and verification, the trained model needs to be deployed to the actual AGV system;

[0098] 1) Model conversion: converting the trained model into a format suitable for the target hardware and software environment;

[0099] 2) Integration into AGV system: Integrate the model into the AGV software platform to ensure that the model can acquire image data and make predictions in real time;

[0100] 3) Interface definition: define the interface and communication protocol between the model and other system components (such as sensors and controllers);

[0101] 7. System testing: After the model deployment is completed, system-level testing and verification are carried out to ensure the stability and reliability of the AGV real-time anti-collision warning system;

[0102] 1) Functional testing: verify whether the system can capture image data, perform road segmentation and obstacle recognition in real time;

[0103] 2) Performance testing: evaluate the system’s response speed, accuracy, and robustness in different scenarios;

[0104] 3) Safety testing: Testing the system’s emergency response and safety control capabilities in the face of emergencies.

[0105] In this embodiment, Figure 2As shown in the figure, the hardware system of AGV real-time collision avoidance is mainly based on the AGV hardware system, which uses sensors to perceive environmental changes, deep learning models to make decisions, and control systems to execute; the AGV hardware system mainly includes a frame 6, a power supply system, a walking system, and a lifting device 5, wherein the power supply system includes a battery 7, and the walking system includes a walking wheel 3, a drive motor 1, and a shock absorber 2; the pressure sensor 8 measures the pressure of the load, and the edge computing platform 9 is used to calculate complex image conversion and model calculations; the camera 4 has significant advantages in the automation system, and through high-resolution image capture and rich visual information, it can accurately identify target objects, colors, textures, and shapes in complex environments, and provide comprehensive environmental perception capabilities; compared with other sensors such as lidar and ultrasonic sensors, cameras are low-cost and flexible to use, and are particularly suitable for real-time image processing and target tracking in dynamic scenes, providing efficient and reliable solutions for automated navigation, safety monitoring, and intelligent identification.

[0106] In this embodiment, the anti-collision warning system of AGV is optimized by improving PSPNet to improve its safety performance in complex environments; CBAM is introduced into PSPNet to improve the model's segmentation performance of the road, and the segmented area is calculated through the segmented image. When the area is smaller than the set warning area, the AGV starts to decelerate, and when the area is smaller than the set safety area, the AGV starts to brake; in addition, as the load of the AGV is different, the warning area and the safety area will also be different; the greater the load, the larger the warning area and the safety area; the specific implementation process is as follows Figure 3 shown.

[0107] In this embodiment, the network structure based on PSPNet is mainly composed of a backbone feature extraction network and a pyramid pooling module; the CBAM attention mechanism is introduced in the backbone feature extraction network, such as Figure 4 As shown, while suppressing the extraction of useless features, it also increases the attention to important features; after segmenting the road, the area of ​​the segmented area is calculated, and it is determined whether the AGV starts to slow down or brake.

[0108] The embodiment of the present invention proposes a real-time anti-collision warning method for AGV based on deep learning, which utilizes a deep learning framework to build a road segmentation model based on an improved PSPNet. Through deep learning technology, high-precision image semantic segmentation can be achieved, and roads can be accurately identified and classified, thereby providing more reliable warning information.

[0109] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A real-time anti-collision warning method for AGV based on deep learning, characterized in that: include: Collect and annotate AGV datasets; Improve the PSPNet road segmentation model to improve the performance of the model; The pixel-level road area calculation method is used to calculate the safe area of ​​the AGV driving road in real time; A safety judgment model based on pressure changes is used to judge the road safety of AGV in real time and make decisions.

2. The method according to claim 1, characterized in that Collecting and annotating AGV datasets includes: Using a data collection device to collect the AGV data set; Annotate the collected AGV data set using an annotation tool; The annotated AGV dataset includes road signs, lane lines, and sidewalks; Annotation of the AGV dataset includes: The boundaries and shapes of the road areas in the AGV dataset images are annotated to distinguish the roads from other objects.

3. The method according to claim 1, characterized in that Improvements to the PSPNet road segmentation model include: Using a deep convolutional neural network to train the labeled AGV dataset to achieve accurate segmentation of roads and obstacles; Introducing CBAM or SE attention mechanism to enhance the model's ability to recognize small targets in complex road scenes; Combined with multi-scale feature extraction technology, it is used to improve the detection performance of road elements of different scales; Using data augmentation techniques to expand the training data set to enhance the robustness and generalization ability of the model; Post-processing the model output, wherein the post-processing includes morphological operations and conditional random fields to optimize the boundaries and details of the segmentation results; Through an online learning mechanism, the model is continuously updated to adapt to new scenarios and environmental changes.

4. The method according to claim 1, characterized in that: The real-time calculation of the safe area of ​​the AGV driving road using the pixel-level road area calculation method includes: Calculate the pixel area of ​​the target road based on the segmented target area; The real area of ​​the target road is inverted through the mapping model between the pixel area and the real road surface.

5. The method according to claim 1, characterized in that The safety judgment model based on pressure change is used to judge the road safety of AGV in real time and make decisions including: The braking area of ​​the AGV corresponding to the current load is calculated by the pressure of the load on the AGV. The braking area under different loads is obtained through a mapping model between the load and the braking area, wherein the braking area includes the warning area and the safety area; After the road is segmented, the segmented area is calculated using the segmented images; When the segmented area is smaller than the set warning area, the AGV starts to slow down, otherwise it drives normally; When the segmented area is smaller than the set safety area, the AGV starts braking, otherwise it drives normally.

6. An AGV real-time anti-collision warning device based on deep learning adopted by the method described in any one of claims 1 to 5, characterized in that: It includes a drive motor, a shock absorber, a running wheel, a camera, a lifting device, a frame, a battery and a pressure sensor.

Images