AGV (Automatic Guided Vehicle) cargo clamping and transporting system and method based on multi-modal perception and intelligent decision

Abstract

The AGV cargo clamping and transporting system comprises a vehicle body, a rotating mechanism is installed on the vehicle body, a rotating shaft mechanism is installed on the rotating mechanism, a clamp mechanism is installed on the rotating shaft mechanism, a sensor and a camera are installed on the vehicle body, and universal wheels and a driving wheel set are installed at the bottom of the vehicle body. According to the robot, the problems that a traditional AGV device is limited in motion freedom degree, high in clamping failure rate, low in efficiency and the like are solved, multi-freedom-degree operation and intelligent fixed clamping can be achieved, work tasks can be completed more conveniently and efficiently, and the robot is high in automation degree, convenient to operate and high in practicability. The working efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of AGV technology, and more specifically, to an AGV cargo picking and transportation system and method based on multimodal perception and intelligent decision-making. Background Technology

[0002] During construction, it is often necessary to transport some heavy materials, which is quite troublesome.

[0003] Traditional methods involve manually securing materials, transporting them to a designated location using a truck-mounted crane, and then manually unloading them. This approach carries risks such as falling objects from heights, high labor costs, and significant time consumption. Furthermore, the gripping mechanisms of traditional AGVs rely heavily on pre-programmed controls, resulting in poor flexibility and difficulty adapting to dynamic obstacles and diverse cargo handling needs in complex environments. Current technologies also suffer from limited gripper freedom and a lack of real-time feedback and intelligent decision-making capabilities, leading to high failure rates and low efficiency. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention aims to provide an AGV cargo gripping and transportation system and method based on multimodal perception and intelligent decision-making. This system solves the problems of limited motion freedom, high gripping failure rate, and low efficiency of traditional AGV devices. It enables multi-degree-of-freedom operation and intelligent fixed gripping, making it more convenient and efficient to complete work tasks and improve work efficiency.

[0005] The present invention adopts the following technical solution:

[0006] The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making includes a vehicle body, a rotating mechanism mounted on the vehicle body, a rotating shaft mechanism mounted on the rotating mechanism, a gripping mechanism mounted on the rotating shaft mechanism, sensors and cameras mounted on the vehicle body, casters and drive wheel sets mounted on the bottom of the vehicle body, a torque limiter and anti-collision sensors mounted on the gripping mechanism, and an electrical control box mounted on the vehicle body, which contains a communication module.

[0007] Furthermore, a touchscreen is located at the front of the vehicle body, with a power button and a camera installed on one side of the touchscreen, and sensors are installed at the top of the vehicle body.

[0008] Furthermore, a power module is installed at the rear of the vehicle body, and a counterweight is located on the lower front side of the vehicle body.

[0009] Furthermore, the clamping mechanism includes two clamping arms, with photoelectric centering detectors and anti-collision sensors installed at the ends of both clamping arms. The other end of the clamping arms is installed inside the clamping arm mounting block. An electrical limit switch is installed on the side of the clamping arm facing the clamping arm mounting block, and a torque limiter is installed on the clamping arm mounting block.

[0010] Furthermore, polyurethane pads are installed at the ends of the two clamping arms.

[0011] Furthermore, the rotating mechanism includes a rotary motor, the output end of which is connected to a gear set, the end of which is connected to a support plate, the bottom of the rotating shaft mechanism is mounted on the support plate, and a clamping mechanism is mounted on the output end of the rotating shaft mechanism.

[0012] Furthermore, the motor output of the rotating shaft mechanism drives the rotating shaft to rotate via a belt. Both ends of the rotating shaft are respectively installed at the rotating shaft mounting block, and the other end of the rotating shaft is connected to the clamping mechanism.

[0013] This invention also discloses an AGV cargo picking and transportation method based on multimodal perception and intelligent decision-making, comprising the following steps:

[0014] Step 1: Start the camera to collect environmental data, identify the initial obstacles and shelf positions through a convolutional neural network (CNN), scan the scene with LiDAR, and combine extended Kalman filter (EKF) to fuse data from multiple sensors to generate a 3D map with centimeter-level accuracy;

[0015] Step 2: The CNN model analyzes the camera footage to identify the outline, label, and stacking posture of the target goods and outputs their three-dimensional coordinates; based on the greedy algorithm and the weight of the goods, it dynamically plans the handling sequence and prioritizes fragile or urgent goods.

[0016] Step 3: Use the improved A algorithm, combined with real-time map data, to plan the shortest path and avoid dynamic obstacles;

[0017] Step 3: If the collision avoidance sensor detects a sudden obstacle, it triggers the deep reinforcement learning (DRL) model to generate a lateral avoidance or deceleration command within 10ms.

[0018] Step 4: Adjust the rotation speed of the rotating mechanism using a PID controller and an AI trajectory prediction model to ensure smooth steering;

[0019] Step 5: The photoelectric centering detector calibrates the center deviation between the clamp arm and the cargo using a binocular vision matching algorithm. When the error is < ±1mm, the clamp is pre-positioned.

[0020] Step 6: The LSTM neural network analyzes historical clamping data, predicts the ideal clamping force range, and dynamically adjusts the closing speed and pressure of the clamping arm. The torque limiter provides real-time feedback of load data. If an overload is detected, the fuzzy control algorithm is immediately triggered to fine-tune the clamping arm angle to ensure stable gripping.

[0021] Step 7: After the task is completed, the clamping parameters and path data are uploaded to the federated learning platform. Multiple AGVs share the data, optimize the global AI model, test the new clamping strategy in a virtual simulation environment, and synchronize it to the AGVs after verification.

[0022] Furthermore, the motor's current and temperature data are transmitted to the cloud, and the Prophet timing prediction model detects anomalies and notifies maintenance in advance. If the collision avoidance sensor identifies a collision risk, the AI ​​module cuts off the power to the drive wheel assembly within 50ms and sounds an alarm via the horn.

[0023] Beneficial effects

[0024] This invention integrates multi-sensor data and AI algorithms to achieve intelligent collaborative control of the multi-dimensional movement of clamps, significantly improving handling accuracy and scene adaptability. This invention can replace high-risk manual labor, offering high safety; it features autonomous navigation, multi-degree-of-freedom operation, and intelligent fixed clamping, making it more convenient and efficient for completing work tasks and improving work efficiency. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the vehicle body portion according to an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of a clamping mechanism according to an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of a rotating mechanism according to an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram of a rotating shaft mechanism according to an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0031] As shown in the figure, this invention discloses an AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making. It includes a vehicle body 1, a rotating mechanism 2 mounted on the vehicle body 1, a rotating shaft mechanism 3 mounted on the rotating mechanism 2, a clamping mechanism 4 mounted on the rotating shaft mechanism 3, sensors 5 and cameras 6 mounted on the vehicle body 1, casters 7 and drive wheel sets 8 mounted on the bottom of the vehicle body 1, a torque limiter 9 and an anti-collision sensor 10 mounted on the clamping mechanism 4, and an electrical control box 11 mounted on the vehicle body 1. The electrical control box 11 contains a communication module. The communication module is used for data transmission. Sensors 5 and cameras 6 are both used for information collection.

[0032] The vehicle body 1 is used for the movement of the AGV, and the rotating mechanism 2 is used to drive the rotating shaft mechanism 3 and the clamping mechanism 4 to rotate. The rotating shaft mechanism 3 drives the clamping mechanism 4 to rotate. The rotating mechanism 2 and the rotating shaft mechanism 3 respectively drive the clamping mechanism 4 to rotate on the Z-axis and Y-axis.

[0033] The collision avoidance sensor 10 can detect sudden obstacles and is used to identify collision risks.

[0034] Of course, in some body designs, a push rod 26 and a horn 27 are also installed. The push rod 26 drives the actuator (i.e., the piston rod of the hydraulic cylinder reciprocates) through hydraulic power, indirectly controlling the pushing, pulling, lifting, and clamping actions of the load components connected to it. The horn 27 is used to emit harsh sounds.

[0035] Anti-collision strips 28 are installed at the front and rear of the vehicle body 1. The anti-collision strips 28 serve a protective function to avoid rigid collisions.

[0036] In one embodiment of the present invention, a touch screen 12 is provided at the front end of the vehicle body 1, a power button 13 and a camera 6 are installed on one side of the touch screen 12, and a sensor 5 is installed at the upper end of the vehicle body 1. The touch screen 12 is easy to operate.

[0037] In one embodiment of the present invention, a power module 14 is installed at the rear end of the vehicle body 1, and a counterweight 15 is provided on the lower front side of the vehicle body 1. The counterweight 15 enables the vehicle body 1 to move smoothly.

[0038] In one embodiment of the present invention, the clamping mechanism 4 includes two clamping arms 16. Photoelectric centering detectors 17 and anti-collision sensors 10 are installed at the ends of the two clamping arms 16. The other end of the clamping arm 16 is installed in the clamping arm mounting block 18. An electrical limit switch 19 is installed on the side of the clamping arm 16 facing the clamping arm mounting block 18. A torque limiter 9 is installed on the clamping arm mounting block 18.

[0039] In one embodiment of the invention, polyurethane pads 20 are also installed at the ends of the two clamping arms 16. The polyurethane pads 20 prevent the clamping arms from making rigid contact with the goods, thereby providing cushioning and protecting the goods.

[0040] In one embodiment of the present invention, the rotating mechanism 2 includes a rotating motor 21, the output end of the rotating motor 21 is connected to a gear set, the end of the gear set is connected to a support plate 22, the bottom of the rotating shaft mechanism 3 is mounted on the support plate 22, and the clamping mechanism 4 is mounted on the output end of the rotating shaft mechanism 3.

[0041] In one embodiment of the present invention, the motor output end 25 of the rotating shaft mechanism 3 drives the rotating shaft 23 to rotate via a belt. Both ends of the rotating shaft 23 are respectively mounted on the rotating shaft mounting block 24, and the other end of the rotating shaft 23 is connected to the clamping mechanism 4. The motor of the rotating shaft structure 3 drives the rotating shaft 23 to rotate, thereby driving the clamping mechanism 4 to rotate.

[0042] This invention also discloses an AGV cargo picking and transportation method based on multimodal perception and intelligent decision-making, comprising the following steps:

[0043] Step 1: Start the camera to collect environmental data, identify the initial obstacles and shelf positions through a convolutional neural network (CNN), scan the scene with LiDAR, and combine extended Kalman filter (EKF) to fuse data from multiple sensors to generate a 3D map with centimeter-level accuracy;

[0044] Step 2: The CNN model analyzes the camera footage to identify the outline, label, and stacking posture of the target goods (such as the tilt angle of the box), and outputs its three-dimensional coordinates; based on the greedy algorithm and the weight of the goods (learned from historical data of the torque limiter), the handling sequence is dynamically planned, and fragile or urgent goods are handled first.

[0045] Step 3: Use the improved A algorithm, combined with real-time map data, to plan the shortest path and avoid dynamic obstacles (such as personnel and other AGVs).

[0046] Step 3: If the collision avoidance sensor detects a sudden obstacle, it triggers the deep reinforcement learning (DRL) model to generate a lateral avoidance or deceleration command within 10ms.

[0047] Step 4: Adjust the rotation speed of the rotating mechanism using a PID controller and an AI trajectory prediction model to ensure smooth steering;

[0048] Step 5: The photoelectric centering detector calibrates the center deviation between the clamp arm and the cargo using a binocular vision matching algorithm. When the error is < ±1mm, the clamp is pre-positioned.

[0049] Step 6: The LSTM neural network analyzes historical clamping data (such as cargo material and surface friction coefficient) to predict the ideal clamping force range and dynamically adjusts the closing speed and pressure of the clamping arm. The torque limiter provides real-time feedback of load data. If an overload is detected (such as cargo slippage), the fuzzy control algorithm is immediately triggered to fine-tune the clamping arm angle to ensure stable gripping.

[0050] Step 7: After the task is completed, the clamping parameters and path data are uploaded to the federated learning platform. Multiple AGVs share the data, optimize the global AI model, test the new clamping strategy in a virtual simulation environment, and synchronize it to the AGVs after verification.

[0051] In one embodiment of the present invention, the current and temperature data of the motor are transmitted to the cloud, and the Prophet timing prediction model detects anomalies, such as signs of wear of transmission components (gear set), and notifies maintenance in advance; if the anti-collision sensor identifies a collision risk, the AI ​​module cuts off the power to the drive wheel set within 50ms and alarms through the horn.

[0052] The above description is merely a preferred embodiment of the present invention; however, the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and its improved concepts, should be covered within the scope of protection of the present invention.

Claims

1. An AGV cargo handling and transportation system based on multimodal perception and intelligent decision-making, characterized in that: The vehicle includes a body, a rotating mechanism mounted on the body, a pivot mechanism mounted on the rotating mechanism, a clamping mechanism mounted on the pivot mechanism, sensors and cameras mounted on the body, casters and drive wheel sets mounted on the bottom of the body, a torque limiter and anti-collision sensors mounted on the clamping mechanism, and an electrical control box mounted on the body, which contains a communication module.

2. The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making according to claim 1, characterized in that: The front of the vehicle is equipped with a touch screen, with a power button and a camera installed on one side of the touch screen, and sensors installed on the top of the vehicle.

3. The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making according to claim 2, characterized in that: A power module is installed at the rear of the vehicle body, and a counterweight is located on the lower front side of the vehicle body.

4. The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making according to claim 1, characterized in that: The clamping mechanism includes two clamping arms. Photoelectric centering detectors and anti-collision sensors are installed at the ends of the two clamping arms. The other end of the clamping arms is installed inside the clamping arm mounting block. An electrical limit switch is installed on the side of the clamping arm facing the clamping arm mounting block. A torque limiter is installed on the clamping arm mounting block.

5. The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making according to claim 4, characterized in that: Polyurethane pads are also installed at the ends of the two clamping arms.

6. The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making according to claim 5, characterized in that: The rotating mechanism includes a rotary motor, the output end of which is connected to a gear set, the end of which is connected to a support plate, the bottom of the rotating shaft mechanism is mounted on the support plate, and the clamping mechanism is mounted on the output end of the rotating shaft mechanism.

7. The AGV cargo gripping and transportation system based on multimodal perception and intelligent decision-making according to claim 6, characterized in that: The motor output of the rotating shaft mechanism drives the rotating shaft to rotate via a belt. Both ends of the rotating shaft are installed at the rotating shaft mounting block, and the other end of the rotating shaft is connected to the clamping mechanism.

8. An AGV cargo gripping and transportation method based on multimodal perception and intelligent decision-making, characterized in that: Includes the following steps: Step 1: Start the camera to collect environmental data, identify the initial obstacles and shelf positions through a convolutional neural network (CNN), scan the scene with LiDAR, and combine extended Kalman filter (EKF) to fuse data from multiple sensors to generate a 3D map with centimeter-level accuracy; Step 2: The CNN model analyzes the camera footage to identify the outline, label, and stacking posture of the target goods and outputs their three-dimensional coordinates; based on the greedy algorithm and the weight of the goods, it dynamically plans the handling sequence and prioritizes fragile or urgent goods. Step 3: Use the improved A algorithm, combined with real-time map data, to plan the shortest path and avoid dynamic obstacles; Step 3: If the collision avoidance sensor detects a sudden obstacle, it triggers the deep reinforcement learning (DRL) model to generate a lateral avoidance or deceleration command within 10ms. Step 4: Adjust the rotation speed of the rotating mechanism using a PID controller and an AI trajectory prediction model to ensure smooth steering; Step 5: The photoelectric centering detector calibrates the center deviation between the clamp arm and the cargo using a binocular vision matching algorithm. When the error is < ±1mm, the clamp is pre-positioned. Step 6: The LSTM neural network analyzes historical clamping data, predicts the ideal clamping force range, and dynamically adjusts the closing speed and pressure of the clamping arm. The torque limiter provides real-time feedback of load data. If an overload is detected, the fuzzy control algorithm is immediately triggered to fine-tune the clamping arm angle to ensure stable gripping. Step 7: After the task is completed, the clamping parameters and path data are uploaded to the federated learning platform. Multiple AGVs share the data, optimize the global AI model, test the new clamping strategy in a virtual simulation environment, and synchronize it to the AGVs after verification.

9. The AGV cargo handling and transportation method based on multimodal perception and intelligent decision-making according to claim 8, characterized in that: The motor's current and temperature data are transmitted to the cloud, and the Prophet timing prediction model detects anomalies and notifies maintenance in advance. If the collision avoidance sensor identifies a collision risk, the AI ​​module cuts off the power to the drive wheel assembly within 50ms and sounds an alarm via the horn.

Images