Method and system for docking of a multi-modal perception unmanned transport vehicle with a loading and unloading device
By using multimodal environmental perception technology, combined with data fusion from vision, lidar, and ultrasonic sensors, and dynamically planning paths and utilizing adaptive PID control, the real-time perception and adaptation issues of docking between unmanned transport vehicles and loading/unloading equipment were solved, achieving an efficient and safe docking process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AISINO CORPORATION
- Filing Date
- 2024-12-30
- Publication Date
- 2026-07-21
AI Technical Summary
The existing docking methods between unmanned transport vehicles and loading and unloading equipment lack real-time dynamic environmental perception and adaptive capabilities, which can easily lead to docking failures, production line shutdowns, or equipment damage, especially in complex environments.
It employs multimodal environmental perception technology, combining visual, lidar, and ultrasonic sensors to acquire environmental data, generates an environmental model through data fusion, dynamically plans the path, and uses adaptive PID control to achieve precise docking.
It has achieved precise docking between unmanned transport vehicles and loading and unloading equipment, improving transportation efficiency, ensuring the safety and stability of the production line, and reducing equipment damage and grain loss.
Smart Images
Figure CN119984264B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information technology application technology, and more specifically, to a method and system for docking unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception. Background Technology
[0002] With the increasing automation of agricultural production and grain transportation, automated guided vehicles (AGVs) are being used more and more widely in the grain transportation process. AGVs transport grain materials between warehouses, farms, and processing plants through automatic navigation systems. During the grain loading and unloading process, the docking of AGVs with loading and unloading equipment (such as unloaders and conveyor belts) is crucial.
[0003] Reference document 1 (CN202310242231.2) provides a shared control obstacle avoidance method based on brain-computer perception fusion, including an environment perception module, a perception fusion module, and a path planning module. The method is characterized in that: the environment perception module uses a brain-computer interface and a lidar to perceive the environment and obtain the environmental information of the robot; the perception fusion module fuses the brain-computer interface perception and lidar perception according to weights to obtain the environmental information required for the robot's path planning; and the path planning control module completes the robot's intelligent obstacle avoidance. The environmental perception module includes a brain-computer interface (BCI) perception module and a lidar perception module. The BCI perception module is used to perceive high-level information about the robot's environment, including a road extraction module, a sub-region segmentation module, and a BCI selection module. The road extraction module extracts road information from the robot's environment. The sub-region segmentation module divides the road area into multiple sub-regions and maps the centroids of the sub-regions onto the BCI paradigm. The BCI selection module selects the centroid points required for biological perception. The lidar perception module perceives basic information about the robot's environment. The perception fusion module includes a weight analysis module and a perception fusion module. The weight analysis module assigns fusion weights to the BCI perception layer and the lidar perception layer. The perception fusion module fuses the two perception layers and generates a grid map. The path planning and control module includes local path planning control and global path planning control.
[0004] Reference document 2 (CN202311195505.3) discloses a method for action decision-making for a virtual character. The method comprises: acquiring state information, which characterizes the game state of the target virtual character; inputting the state information into an action decision model to obtain n target sub-actions serially output by n action output heads in the action decision model, wherein different action output heads correspond to different action types, and the n action output heads are serially connected based on the dependency relationships between the action types, which characterize the dependency constraints between sub-actions under different action types; and controlling the target virtual character to execute a target action composed of the n target sub-actions.
[0005] However, the existing technologies for docking unmanned vehicles with loading and unloading equipment rely on relatively simple sensors or human intervention, lacking real-time dynamic environmental perception and adaptive capabilities. Especially in complex working environments, there is a risk of docking failure, which can lead to production line stagnation or even equipment damage or food loss.
[0006] Therefore, a technology is needed to enable the docking of unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception. Summary of the Invention
[0007] The present invention provides a method and system for docking unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception, in order to solve the problem of how to dock unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception.
[0008] To address the aforementioned problems, this invention provides a method for docking an unmanned transport vehicle with loading and unloading equipment based on multimodal environmental perception, the method comprising:
[0009] Multimodal environmental perception is performed on the loading and unloading equipment and its surrounding environment within a preset range to acquire multimodal environmental perception data; the multimodal environmental perception data is synthesized to generate a multimodal environmental model;
[0010] Based on the multimodal environment model, and taking into account factors such as obstacles, loading and unloading equipment, and travel speed, a path planning algorithm is used to generate a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment; the unmanned transport vehicle travels to the loading and unloading equipment based on the selected docking path.
[0011] When the unmanned transport vehicle is traveling along the selected docking path, it dynamically adjusts the selected docking path based on the real-time acquired multimodal environmental perception data; the unmanned transport vehicle then travels towards the loading and unloading equipment based on the adjusted selected docking path.
[0012] When the unmanned transport vehicle is detected to be within a preset distance from the loading and unloading equipment, the speed and position of the unmanned transport vehicle are adjusted by adaptive PID control to ensure precise docking between the unmanned transport vehicle and the loading and unloading equipment.
[0013] Preferably, the step of performing multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, and acquiring multimodal environmental perception data, includes:
[0014] Visual perception data is acquired by using a camera device to obtain images of the loading and unloading equipment and its surrounding environment within a preset range, and by using a deep learning algorithm to identify the type, location, and status of the loading and unloading equipment in the images.
[0015] Acquire LiDAR perception data, scan the loading and unloading equipment and its surrounding environment within a preset range using 360-degree LiDAR, and acquire point cloud data of the surrounding environment; based on the point cloud data, perceive the distance and relative position between the unmanned transport vehicle and the loading and unloading equipment in real time.
[0016] Acquire ultrasonic wave sensing data. Using multiple ultrasonic sensors, short-distance measurements are taken between the unmanned transport vehicle and the loading and unloading equipment when the distance between them is less than a preset distance. The multiple ultrasonic sensors are respectively installed at the front end, rear end, and side of the unmanned transport vehicle.
[0017] Preferably, it further includes: controlling the docking action between the unmanned transport vehicle and the loading / unloading equipment:
[0018] When the unmanned transport vehicle docks with the loading and unloading equipment, the multimodal environmental perception data is monitored in real time. Based on the real-time acquired multimodal environmental perception data, the status of the loading and unloading equipment and whether the surrounding environment is abnormal are judged.
[0019] When an anomaly is detected in the status of the loading / unloading equipment or the surrounding environment, the docking action is controlled based on a preset anomaly handling strategy.
[0020] Preferably, the anomaly handling strategy includes: adjusting the path and speed of the unmanned transport vehicle, or stopping the unmanned transport vehicle.
[0021] Preferably, the step of synthesizing the multimodal environment perception data to generate a multimodal environment model includes:
[0022] The multimodal environmental perception data is processed using Kalman filtering data fusion technology to eliminate noise.
[0023] According to another aspect of the present invention, the present invention provides a docking system for unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception, the system comprising:
[0024] The acquisition unit is used to perform multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, acquire multimodal environmental perception data, and synthesize the multimodal environmental perception data to generate a multimodal environmental model.
[0025] The generation unit is used to generate a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment based on the multimodal environment model, taking into account factors such as obstacles, loading and unloading equipment, and driving speed, using a path planning algorithm; the unmanned transport vehicle then travels to the loading and unloading equipment based on the selected docking path.
[0026] The first adjustment unit is used to dynamically adjust the selected docking path based on the real-time acquired multimodal environmental perception data when the unmanned transport vehicle is traveling along the selected docking path; the unmanned transport vehicle travels towards the loading and unloading equipment based on the adjusted selected docking path.
[0027] The second adjustment unit is used to adjust the speed and position of the unmanned transport vehicle through adaptive PID control when it senses that the distance between the unmanned transport vehicle and the loading and unloading equipment is within a preset distance, so that the unmanned transport vehicle can accurately dock with the loading and unloading equipment.
[0028] Preferably, the acquisition unit is used to perform multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, acquire multimodal environmental perception data, and is also used to:
[0029] Visual perception data is acquired by using a camera device to obtain images of the loading and unloading equipment and its surrounding environment within a preset range, and by using a deep learning algorithm to identify the type, location, and status of the loading and unloading equipment in the images.
[0030] Acquire LiDAR perception data, scan the loading and unloading equipment and its surrounding environment within a preset range using 360-degree LiDAR, and acquire point cloud data of the surrounding environment; based on the point cloud data, perceive the distance and relative position between the unmanned transport vehicle and the loading and unloading equipment in real time.
[0031] Acquire ultrasonic wave sensing data. Using multiple ultrasonic sensors, short-distance measurements are taken between the unmanned transport vehicle and the loading and unloading equipment when the distance between them is less than a preset distance. The multiple ultrasonic sensors are respectively installed at the front end, rear end, and side of the unmanned transport vehicle.
[0032] Preferably, the second adjustment unit is further configured to: control the docking action between the unmanned transport vehicle and the loading / unloading equipment.
[0033] When the unmanned transport vehicle docks with the loading and unloading equipment, the multimodal environmental perception data is monitored in real time. Based on the real-time acquired multimodal environmental perception data, the status of the loading and unloading equipment and whether the surrounding environment is abnormal are judged.
[0034] When an anomaly is detected in the status of the loading / unloading equipment or the surrounding environment, the docking action is controlled based on a preset anomaly handling strategy.
[0035] Preferably, the anomaly handling strategy includes: adjusting the path and speed of the unmanned transport vehicle, or stopping the unmanned transport vehicle.
[0036] Preferably, the generation unit is used to synthesize the multimodal environment perception data to generate a multimodal environment model, and is also used to:
[0037] The multimodal environmental perception data is processed using Kalman filtering data fusion technology to eliminate noise.
[0038] This invention provides a method and system for docking an unmanned transport vehicle with loading and unloading equipment based on multimodal environmental perception. The method includes: performing multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range to acquire multimodal environmental perception data; synthesizing the multimodal environmental perception data to generate a multimodal environmental model; based on the multimodal environmental model, and considering factors such as obstacles, loading and unloading equipment, and travel speed, generating a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment through a path planning algorithm; the unmanned transport vehicle traveling towards the loading and unloading equipment based on the selected docking path; while the unmanned transport vehicle is traveling along the selected docking path, dynamically adjusting the selected docking path based on the real-time acquired multimodal environmental perception data; the unmanned transport vehicle traveling towards the loading and unloading equipment based on the adjusted selected docking path; and when the distance between the unmanned transport vehicle and the loading and unloading equipment is sensed to be within a preset distance, adjusting the speed and position of the unmanned transport vehicle through adaptive PID control to achieve precise docking between the unmanned transport vehicle and the loading and unloading equipment. This invention proposes a method and system for docking unmanned grain transport vehicles with grain loading and unloading equipment based on multimodal environmental perception. The technical solution of this invention achieves comprehensive perception and analysis of the environment by combining data from multiple sensors such as vision, lidar, and ultrasound, and combines intelligent path planning and control algorithms to ensure that unmanned grain transport vehicles can dock with grain loading and unloading equipment accurately and safely. Attached Figure Description
[0039] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:
[0040] Figure 1 This is a flowchart of a docking method between an unmanned transport vehicle and loading / unloading equipment based on multimodal environmental perception, according to a preferred embodiment of the present invention.
[0041] Figure 2 A flowchart illustrating a preferred embodiment of the present invention for a docking method between an unmanned transport vehicle and loading / unloading equipment based on multimodal environmental perception; and
[0042] Figure 3 This is a structural diagram of a docking system between an unmanned transport vehicle and loading / unloading equipment based on multimodal environmental perception, according to a preferred embodiment of the present invention. Detailed Implementation
[0043] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.
[0044] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.
[0045] Figure 1 This is a flowchart illustrating a preferred embodiment of the present invention regarding a docking method between an unmanned transport vehicle and loading / unloading equipment based on multimodal environmental perception.
[0046] like Figure 1 As shown, this invention provides a method for docking an unmanned transport vehicle with loading and unloading equipment based on multimodal environmental perception. The method includes:
[0047] Step 101: Perform multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, and acquire multimodal environmental perception data; synthesize the multimodal environmental perception data to generate a multimodal environmental model;
[0048] Preferably, multimodal environmental perception is performed on the loading and unloading equipment and its surrounding environment within a preset range to acquire multimodal environmental perception data, including:
[0049] Acquire visual perception data by using camera equipment to capture images of the loading and unloading equipment and its surrounding environment within a preset range, and use deep learning algorithms to identify the type, location, and status of the loading and unloading equipment in the images;
[0050] Acquire LiDAR perception data by scanning the loading and unloading equipment and its surrounding environment within a preset range using 360-degree LiDAR to obtain point cloud data in the surrounding environment; based on the point cloud data, perceive the distance and relative position between the unmanned transport vehicle and the loading and unloading equipment in real time.
[0051] Acquire ultrasonic wave sensing data. Using multiple ultrasonic sensors, short-distance measurements are taken between the unmanned transport vehicle and the loading and unloading equipment when the distance between them is less than a preset distance. The multiple ultrasonic sensors are respectively installed at the front, rear, and sides of the unmanned transport vehicle.
[0052] Preferably, the multimodal environment perception data is synthesized to generate a multimodal environment model, including:
[0053] Kalman filtering data fusion technology is used to process multimodal environmental perception data and eliminate noise.
[0054] Multimodal environment perception
[0055] By using multiple sensors such as visual perception, lidar and ultrasonic sensors to collect environmental information in real time, and by using data fusion algorithms to analyze key information such as the spatial position, attitude and obstacles between the unmanned grain transport vehicle and the loading and unloading equipment.
[0056] The multimodal environment perception of the present invention includes:
[0057] Visual perception: Acquire image information of the surrounding environment through high-resolution cameras, and use deep learning algorithms to identify loading and unloading equipment and its surrounding environment, including the type, location, and status of the equipment.
[0058] LiDAR perception: Using 360-degree LiDAR to scan the surrounding environment, acquiring point cloud data, and sensing the distance and relative position between the unmanned grain transport vehicle and loading / unloading equipment in real time. LiDAR can provide high-precision spatial geometric information, and is particularly suitable for obstacle detection in complex environments.
[0059] Ultrasonic sensors: Multiple ultrasonic sensors are installed for short-range measurements, especially when approaching loading and unloading equipment, to monitor the presence of obstacles in real time, provide accurate distance information, and prevent collisions.
[0060] Data fusion: Data from vision, LiDAR, and ultrasonic sensors are synthesized using data fusion algorithms to generate an accurate environmental model. This model provides the basis for subsequent path planning, docking control, and anomaly handling.
[0061] Step 102: Based on the multimodal environment model, and considering factors such as obstacles, loading and unloading equipment, and driving speed, a path planning algorithm is used to generate a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment; the unmanned transport vehicle travels to the loading and unloading equipment based on the selected docking path.
[0062] This invention performs dynamic path planning:
[0063] Based on environmental data collected by sensors, the unmanned grain transport vehicle plans its path to dock with the grain loading and unloading equipment in real time, and dynamically adjusts the path according to information such as obstacle positions and environmental changes to ensure that the unmanned vehicle can successfully approach the equipment.
[0064] Path generation: Based on environmental data, the optimal path from the current location of the unmanned grain transport vehicle to the grain loading and unloading equipment is automatically generated through a path planning algorithm. This path takes into account various factors such as obstacles, equipment location, and travel speed.
[0065] Step 103: When the unmanned transport vehicle is traveling along the selected docking path, the selected docking path is dynamically adjusted based on the real-time acquired multimodal environmental perception data; the unmanned transport vehicle then travels towards the loading and unloading equipment based on the adjusted selected docking path.
[0066] This invention makes real-time adjustments to the optimized path: During the operation of the unmanned grain transport vehicle, the system monitors changes in the environment in real time (such as changes in the position of loading and unloading equipment, the appearance of new obstacles, etc.) and dynamically adjusts the path to ensure that the unmanned vehicle can avoid obstacles and successfully dock with the equipment.
[0067] Step 104: When the distance between the unmanned transport vehicle and the loading and unloading equipment is detected to be within the preset distance, the speed and position of the unmanned transport vehicle are adjusted by adaptive PID control so that the unmanned transport vehicle can accurately dock with the loading and unloading equipment.
[0068] This invention controls precise docking:
[0069] Based on real-time feedback of environmental data, the relative position and speed between the unmanned grain transport vehicle and the grain loading and unloading equipment are automatically adjusted to ensure high-precision docking and avoid collisions or misalignments.
[0070] Docking precision control: When the unmanned grain transport vehicle approaches the grain loading and unloading equipment, the system adjusts the vehicle's trajectory and speed based on information from sensor feedback to ensure high precision during docking. The system uses adaptive PID control to adjust vehicle speed and displacement to achieve precise docking.
[0071] Docking motion control: By controlling the vehicle's forward, backward, and rotation movements, the unmanned vehicle can precisely adjust its relative position to the equipment, ensuring a smooth connection with the grain loading and unloading equipment.
[0072] Preferably, it also includes: controlling the docking action between the unmanned transport vehicle and the loading and unloading equipment:
[0073] When the unmanned transport vehicle docks with the loading and unloading equipment, it monitors multimodal environmental perception data in real time. Based on the real-time acquisition of multimodal environmental perception data, it judges the status of the loading and unloading equipment and whether the surrounding environment is abnormal.
[0074] When an anomaly is detected in the status of the loading / unloading equipment or the surrounding environment, the docking action is controlled based on a preset anomaly handling strategy.
[0075] Preferably, the anomaly handling strategy includes: adjusting the path and speed of the unmanned transport vehicle, or stopping the unmanned transport vehicle.
[0076] The status monitoring and anomaly handling of this invention include:
[0077] During the docking process, the system continuously monitors the equipment and environmental status to determine in real time whether any abnormalities occur (such as equipment failure, obstacle interference, etc.). Once an abnormality occurs, the system can automatically trigger an early warning and adjust the operating strategy to ensure the safety of the docking process.
[0078] Real-time monitoring: During the docking process, the unmanned grain transport vehicle continuously acquires sensor data to monitor equipment status and environmental changes. Once an anomaly is detected (such as changes in equipment position, obstruction by obstacles, sensor malfunction, etc.), the system will trigger an alarm and adjust according to the preset anomaly handling strategy.
[0079] Anomaly Handling Strategy: When the system detects an anomaly, it can automatically adjust the path, slow down, or stop to avoid docking failure or equipment damage. The system will also send alarms to operators, prompting them with appropriate handling measures.
[0080] The following are examples illustrating embodiments of the present invention:
[0081] 1. Multimodal environment perception
[0082] Vision system: Industrial-grade cameras are installed on unmanned grain transport vehicles to identify the type and location of loading and unloading equipment using machine vision algorithms. The visual data can be further used for equipment identification and target localization.
[0083] LiDAR system: LiDAR is used to scan the surrounding environment in all directions and provide accurate three-dimensional spatial data to help unmanned vehicles identify loading and unloading equipment and other obstacles, ensuring safety during the docking process.
[0084] Ultrasonic sensors: Ultrasonic sensors are installed at the front, rear, and sides of the unmanned grain transport vehicle to measure the distance between it and the loading and unloading equipment, ensuring precise docking in the final stage.
[0085] 2. Data Fusion and Path Planning
[0086] Fusion algorithm: Data from multiple sources such as vision, lidar and ultrasonic sensors are processed through data fusion techniques such as Kalman filtering to eliminate sensor noise and improve the accuracy of environmental perception.
[0087] Path planning: The RRT algorithm generates the shortest path and adjusts the path based on real-time perception of obstacles and environmental changes to ensure that the unmanned grain transport vehicle can reach the target location smoothly.
[0088] 3. Precise docking
[0089] Precise control: When approaching loading and unloading equipment, the system uses sensor feedback to control the speed and position of the unmanned grain transport vehicle, and finely adjusts the docking angle and distance to ensure the success rate and accuracy of docking.
[0090] Speed adjustment: The system automatically slows down the unmanned grain transport vehicle's speed based on the shortening distance and the required docking precision, in order to ensure stability during docking.
[0091] 4. Exception Handling
[0092] Anomaly monitoring: By monitoring the status of loading and unloading equipment (such as changes in position, equipment failures, etc.) and obstacles in the environment in real time, anomalies can be detected in a timely manner and corresponding measures can be taken.
[0093] Emergency Handling: When the system detects that docking cannot continue, it will automatically stop the vehicle and send an abnormal alarm message to prevent equipment damage or grain loss.
[0094] This invention utilizes data fusion from multiple sensors, including vision, lidar, and ultrasonic sensors, to achieve comprehensive perception and precise positioning of the surrounding environment, ensuring that unmanned grain transport vehicles can accurately identify loading and unloading equipment and its relative position.
[0095] This invention combines real-time collected environmental data, enabling unmanned grain transport vehicles to generate selected docking paths through intelligent algorithms and dynamically adjust the paths according to environmental changes, ensuring that obstacles are avoided and that the vehicles can smoothly approach loading and unloading equipment.
[0096] This invention enables unmanned grain transport vehicles to automatically adjust their speed, position, and angle when approaching loading and unloading equipment through precise motion control algorithms, achieving high-precision docking and avoiding collisions or misalignments.
[0097] This invention monitors the equipment status and environmental changes during the docking process in real time. The system can quickly detect anomalies (such as equipment failure or obstacle interference) and automatically adjust operations or issue alarms to ensure the safety and reliability of the docking process.
[0098] This invention utilizes multimodal environmental perception technology to accurately perceive the spatial relationship between unmanned grain transport vehicles and grain loading and unloading equipment, ensuring high-precision automatic docking and reducing human intervention.
[0099] By fusing data from vision, lidar, and ultrasonic sensors, the system can adapt to changes in the factory environment in real time, automatically adjust its path, and ensure reliable docking in complex environments.
[0100] The automated docking process not only improves transportation efficiency, but also effectively ensures the safety and stability of the production line through anomaly detection and emergency handling functions.
[0101] Figure 3 This is a structural diagram of a docking system between an unmanned transport vehicle and loading / unloading equipment based on multimodal environmental perception, according to a preferred embodiment of the present invention.
[0102] like Figure 3 As shown, this invention provides a docking system for unmanned transport vehicles and loading / unloading equipment based on multimodal environmental perception. The system includes:
[0103] The acquisition unit 301 is used to perform multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, acquire multimodal environmental perception data, and synthesize the multimodal environmental perception data to generate a multimodal environmental model.
[0104] Preferably, the acquisition unit 301 is used to perform multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, acquire multimodal environmental perception data, and is also used for:
[0105] Acquire visual perception data by using camera equipment to capture images of the loading and unloading equipment and its surrounding environment within a preset range, and use deep learning algorithms to identify the type, location, and status of the loading and unloading equipment in the images;
[0106] Acquire LiDAR perception data by scanning the loading and unloading equipment and its surrounding environment within a preset range using 360-degree LiDAR to obtain point cloud data in the surrounding environment; based on the point cloud data, perceive the distance and relative position between the unmanned transport vehicle and the loading and unloading equipment in real time.
[0107] Acquire ultrasonic wave sensing data. Using multiple ultrasonic sensors, short-distance measurements are taken between the unmanned transport vehicle and the loading and unloading equipment when the distance between them is less than a preset distance. The multiple ultrasonic sensors are respectively installed at the front, rear, and sides of the unmanned transport vehicle.
[0108] The generation unit 302 is used to generate a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment based on a multimodal environment model, taking into account factors such as obstacles, loading and unloading equipment and driving speed, and through a path planning algorithm; the unmanned transport vehicle then travels to the loading and unloading equipment based on the selected docking path.
[0109] Preferably, the generation unit 302 is used to synthesize multimodal environment perception data to generate a multimodal environment model, and is also used to:
[0110] Kalman filtering data fusion technology is used to process multimodal environmental perception data and eliminate noise.
[0111] The first adjustment unit 303 is used to dynamically adjust the selected docking path based on real-time acquired multimodal environmental perception data when the unmanned transport vehicle is driving along the selected docking path; the unmanned transport vehicle then drives toward the loading and unloading equipment based on the adjusted selected docking path.
[0112] The second adjustment unit 304 is used to adjust the speed and position of the unmanned transport vehicle through adaptive PID control when it senses that the distance between the unmanned transport vehicle and the loading and unloading equipment is within a preset distance, so that the unmanned transport vehicle can accurately dock with the loading and unloading equipment.
[0113] Preferably, the second adjustment unit 304 is further used to: control the docking action between the unmanned transport vehicle and the loading and unloading equipment.
[0114] When the unmanned transport vehicle docks with the loading and unloading equipment, it monitors multimodal environmental perception data in real time. Based on the real-time acquisition of multimodal environmental perception data, it judges the status of the loading and unloading equipment and whether the surrounding environment is abnormal.
[0115] When an anomaly is detected in the status of the loading / unloading equipment or the surrounding environment, the docking action is controlled based on a preset anomaly handling strategy.
[0116] Preferably, the anomaly handling strategy includes: adjusting the path and speed of the unmanned transport vehicle, or stopping the unmanned transport vehicle.
[0117] The preferred embodiment of the present invention provides a docking system for unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception, which corresponds to the preferred embodiment of the present invention providing a docking method for unmanned transport vehicles and loading and unloading equipment based on multimodal environmental perception. These will not be described in detail here.
[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0119] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0122] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.
[0123] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
[0124] The invention has been described with reference to a few embodiments. However, as will be known to those skilled in the art, and as defined in the appended claims, other embodiments besides those disclosed above fall equivalently within the scope of the invention.
[0125] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless otherwise expressly defined herein. All references to “a / / the [device, component, etc.]” are openly interpreted as at least one instance of the device, component, etc., unless otherwise expressly stated. The steps of any method disclosed herein are not necessarily to be performed in the exact order disclosed, unless explicitly stated otherwise.
Claims
1. A method for docking an unmanned transport vehicle with loading and unloading equipment based on multimodal environmental perception, the method comprising: Multimodal environmental perception is performed on the loading and unloading equipment and its surrounding environment within a preset range to obtain multimodal environmental perception data; The multimodal environment perception data is synthesized to generate a multimodal environment model; Based on the multimodal environment model, and taking into account factors such as obstacles, loading and unloading equipment, and driving speed, a path planning algorithm is used to generate a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment. The unmanned transport vehicle travels to the loading and unloading equipment based on the selected docking path; When the unmanned transport vehicle is traveling along the selected docking path, it dynamically adjusts the selected docking path based on the real-time acquired multimodal environmental perception data. The unmanned transport vehicle travels to the loading and unloading equipment based on the adjusted selected docking path; The process of performing multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, and acquiring multimodal environmental perception data, includes: Visual perception data is acquired by using a camera to capture images of the loading / unloading equipment and its surrounding environment within a preset range, and by using a deep learning algorithm to identify the type, location, and status of the loading / unloading equipment in the images; LiDAR perception data is acquired by using a 360-degree LiDAR to scan the loading / unloading equipment and its surrounding environment within a preset range to acquire point cloud data of the surrounding environment; based on the point cloud data, the distance and relative position between the unmanned transport vehicle and the loading / unloading equipment are perceived in real time; ultrasonic perception data is acquired by using multiple ultrasonic sensors to perform short-distance measurements between the unmanned transport vehicle and the loading / unloading equipment when the distance between them is less than a preset distance; the multiple ultrasonic sensors are respectively installed at the front, rear, and sides of the unmanned transport vehicle; When the unmanned transport vehicle is detected to be within a preset distance from the loading and unloading equipment, the speed and position of the unmanned transport vehicle are adjusted through adaptive PID control to ensure precise docking between the unmanned transport vehicle and the loading and unloading equipment. The docking action between the unmanned transport vehicle and the loading and unloading equipment is controlled: when the unmanned transport vehicle docks with the loading and unloading equipment, the multimodal environmental perception data is monitored in real time, and the status of the loading and unloading equipment and whether the surrounding environment is abnormal are judged based on the real-time acquisition of the multimodal environmental perception data. When an anomaly is detected in the status of the loading and unloading equipment or in the surrounding environment, the docking action is controlled based on a preset anomaly handling strategy. The anomaly handling strategy includes adjusting the path and speed of the unmanned transport vehicle, or stopping the unmanned transport vehicle.
2. The method according to claim 1, wherein synthesizing the multimodal environment perception data to generate a multimodal environment model comprises: The multimodal environmental perception data is processed using Kalman filtering data fusion technology to eliminate noise.
3. A docking system for unmanned transport vehicles and loading / unloading equipment based on multimodal environmental perception, the system comprising: The acquisition unit is used to perform multimodal environmental perception on the loading and unloading equipment and its surrounding environment within a preset range, and to acquire multimodal environmental perception data. The system synthesizes the multimodal environmental perception data to generate a multimodal environmental model; it is also used for: acquiring visual perception data by using a camera device to acquire images of the loading and unloading equipment and its surrounding environment within a preset range, and using a deep learning algorithm to identify the type, location, and state of the loading and unloading equipment in the images; acquiring lidar perception data by using a 360-degree lidar to scan the loading and unloading equipment and its surrounding environment within a preset range to acquire point cloud data in the surrounding environment; based on the point cloud data, it perceives the distance and relative position between the unmanned transport vehicle and the loading and unloading equipment in real time; and acquiring ultrasonic perception data by using multiple ultrasonic sensors to perform short-distance measurements between the unmanned transport vehicle and the loading and unloading equipment when the distance between them is less than a preset distance; the multiple ultrasonic sensors are respectively installed at the front end, rear end, and side of the unmanned transport vehicle. The generation unit is used to generate a selected docking path from the current position of the unmanned transport vehicle to the loading and unloading equipment based on the multimodal environment model, taking into account factors such as obstacles, loading and unloading equipment and driving speed, through a path planning algorithm. The unmanned transport vehicle travels to the loading and unloading equipment based on the selected docking path; The first adjustment unit is used to dynamically adjust the selected docking path based on the real-time acquired multimodal environmental perception data when the unmanned transport vehicle is driving based on the selected docking path. The unmanned transport vehicle travels to the loading and unloading equipment based on the adjusted selected docking path; The second adjustment unit is used to adjust the speed and position of the unmanned transport vehicle through adaptive PID control when it is sensed that the distance between the unmanned transport vehicle and the loading and unloading equipment is within a preset distance, so that the unmanned transport vehicle and the loading and unloading equipment can be precisely docked; it is also used to control the docking action between the unmanned transport vehicle and the loading and unloading equipment: when the unmanned transport vehicle docks with the loading and unloading equipment, it monitors the multimodal environmental perception data in real time, and judges the status of the loading and unloading equipment and whether the surrounding environment is abnormal based on the real-time acquired multimodal environmental perception data; When it is determined that there is an abnormality in the status of the loading and unloading equipment or the surrounding environment, the docking action is controlled based on a preset abnormality handling strategy. The anomaly handling strategy includes: adjusting the path and speed of the unmanned transport vehicle, or stopping the unmanned transport vehicle.
4. The system according to claim 3, wherein the generation unit is configured to synthesize the multimodal environment perception data to generate a multimodal environment model, and is further configured to: The multimodal environmental perception data is processed using Kalman filtering data fusion technology to eliminate noise.