Intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision

Abstract

The invention discloses an intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision, and relates to the technical field of intelligent storage and automatic logistics. Comprising a multi-visual perception module, a collaborative decision-making module, a vehicle-mounted execution module and a storage environment mapping module which are connected in sequence. According to the invention, a multi-dimensional machine vision perception system is constructed, perception resources of the vehicle-mounted binocular RGB-D camera, the shelf side vision sensor and the channel side panorama camera are integrated, comprehensive perception of multiple elements such as vehicles, goods locations, goods and channels in a storage environment is realized, and compared with an existing single vision or laser positioning scheme, the system has the advantages that the system is simple in structure and convenient to operate. The sensing range is more comprehensive, the dynamic adaptability is higher, and the dynamic change of the storage environment can be effectively handled.

Description

Technical Field

[0001] This invention relates to the field of intelligent warehousing and automated logistics technology, and more specifically to an intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system and method based on machine vision. Background Technology

[0002] With the rapid development of the e-commerce industry, intelligent warehousing, as a core component of automated logistics systems, places higher demands on the efficiency and accuracy of goods storage and handling. Autonomous warehouse vehicles, as key execution equipment in intelligent warehousing, directly determine the overall operational efficiency of the warehousing system through their multi-vehicle collaborative operation capabilities.

[0003] Existing intelligent warehousing multi-vehicle collaborative technologies mostly rely on pre-laid magnetic strips, QR codes, or laser SLAM positioning and navigation solutions, which have many limitations: First, the initial construction cost of magnetic strips, QR codes, and other pre-laid navigation solutions is high and lacks flexibility, making them unable to adapt to the needs of dynamic adjustments in warehouse locations; second, laser SLAM solutions are easily affected by factors such as shelf obstruction and cargo reflection in complex warehouse environments, leading to a decrease in positioning accuracy, and laser sensors are also expensive; third, existing multi-vehicle collaborative decisions are mostly based on preset path planning, lacking the ability to perceive and respond to real-time dynamic changes in the environment, which easily leads to vehicle congestion and collision risks, especially in scenarios with narrow intersections where collaborative efficiency is low.

[0004] Machine vision technology boasts advantages such as comprehensive information acquisition, relatively low cost, and strong adaptability to dynamic environments, and has been gradually applied in the warehousing field. However, current applications are mostly limited to visual navigation or cargo recognition for single vehicles, failing to form a complete system for collaborative perception across multiple visual nodes and multi-vehicle decision-making linkage. Specifically, existing technologies do not fully integrate multi-dimensional visual resources such as vehicle vision, shelf vision, and aisle vision, making it impossible to construct a comprehensive and real-time model of the dynamic warehousing environment. Furthermore, in the process of multi-vehicle collaborative decision-making, the dynamic features extracted by machine vision (such as real-time vehicle distribution, aisle occupancy status, and cargo dynamic attributes) are not deeply integrated into the decision-making mechanism, resulting in a lack of accuracy and real-time performance in collaborative strategies.

[0005] How to build a perception system based on multi-dimensional machine vision to achieve comprehensive dynamic perception of the warehouse environment, and improve the accuracy and efficiency of multi-vehicle autonomous driving collaborative decision-making based on visual perception information, has become a technical problem that needs to be solved in existing intelligent warehousing technologies.

[0006] Therefore, proposing a machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system and method to solve the difficulties of existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system and method. By integrating multi-dimensional visual perception resources and constructing a dynamic semantic map of the warehouse, it enables accurate decision-making and efficient execution of multi-vehicle collaborative operations, thereby improving the operating efficiency and flexibility of the intelligent warehousing system.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system includes a multi-vision perception module, a collaborative decision-making module, an onboard execution module, and a warehousing environment mapping module connected in sequence; wherein, The multi-vision perception module includes binocular RGB-D cameras deployed on each autonomous warehouse vehicle, shelf-end vision sensors deployed on the shelves, and aisle-side panoramic cameras deployed on the aisle side. The binocular RGB-D cameras on each autonomous warehouse vehicle are used to acquire images of the scene ahead along the vehicle's driving path, 3D contour images of the cargo pallets, and position and posture images of adjacent vehicles. The shelf-end vision sensors are used to acquire images of shelf location occupancy status and cargo identification images. The aisle-side panoramic cameras are used to acquire global scene images of the warehouse aisle and images of the distribution of multiple vehicles. The warehouse environment mapping module is used to receive various image data output by the multi-visual perception module and generate a dynamic semantic map of the warehouse through image stitching, feature matching and 3D reconstruction. The collaborative decision-making module includes an edge computing node and a vehicle-to-vehicle communication interaction unit. The edge computing node is used to receive the dynamic semantic map of the warehouse and the real-time status data of each autonomous warehouse vehicle, and generate multi-vehicle task allocation schemes and path collaboration strategies. The vehicle-to-vehicle communication interaction unit is used to realize the sharing of perception data and the interaction of decision commands among the autonomous warehouse vehicles. The on-board execution module, deployed in each autonomous warehouse vehicle, is used to receive decision instructions output by the collaborative decision module and control the vehicle to complete path tracking, fork lifting, pallet docking, and obstacle avoidance actions.

[0009] Optionally, the multi-vision perception module also includes a visual data preprocessing unit, which performs noise filtering, exposure correction, and distortion correction on the acquired image data, and extracts feature points of shelf uprights, pallet corners, lane lines, and vehicle contours from the image through feature extraction algorithms.

[0010] Optionally, the 3D reconstruction process of the warehouse environment mapping module includes: constructing a 3D model of the storage location based on multi-view images of the storage location collected by the visual sensors at the shelf end; combining the global images collected by the panoramic cameras on the aisle side to achieve the fusion of the storage location model and the aisle model; and updating the 3D coordinates of dynamic obstacles by using the depth images collected by the binocular RGB-D cameras of each autonomous warehouse vehicle.

[0011] A machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making method, executing any of the machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making systems described above, includes the following steps: S1. The multi-vision perception module starts image acquisition. The binocular RGB-D camera of each autonomous warehouse vehicle acquires real-time images of the driving path scene and surrounding vehicles. The visual sensor at the shelf end acquires images of the location status and goods identification. The panoramic camera on the aisle side acquires global images of the aisle. S2, the warehouse environment mapping module receives various types of image data, performs image stitching and 3D reconstruction after preprocessing, and generates a dynamic semantic map of the warehouse containing information on storage locations, aisles, dynamic obstacles and goods, which is synchronized to the collaborative decision-making module in real time. S3, the collaborative decision-making module receives the warehouse dynamic semantic map and the location, speed and load status data of each autonomous warehouse vehicle. Based on the multi-vehicle distribution features and channel occupancy features extracted by machine vision, it generates a multi-vehicle task allocation scheme and clarifies the picking location, delivery location and driving priority of each vehicle. S4. The collaborative decision-making module identifies intersections and narrow passages in the task path of each vehicle by visual feature matching. Based on the real-time position images of multiple vehicles, it calculates the time difference between the arrival of vehicles at the intersection and generates a path collaboration strategy, including the passage order at intersections, the way to avoid oncoming traffic in narrow passages, and the dynamic safety distance threshold. S5. Each autonomous warehouse vehicle shares perception data and task execution status through the vehicle-to-vehicle communication interaction unit. The on-board execution module controls the vehicle's movement according to the collaborative decision-making instructions. It monitors path deviation and obstacle changes in real time through binocular RGB-D cameras, dynamically adjusts driving parameters, and completes the tasks of picking up and placing goods and driving along the path.

[0012] Optionally, the warehouse dynamic semantic map in S2 is updated as follows: when the panoramic camera on the channel side detects a new obstacle in the channel, it simultaneously triggers the binocular RGB-D cameras of the surrounding vehicles to focus and capture images of the obstacle. The three-dimensional size and position of the obstacle are determined by multi-view image fusion and updated to the warehouse dynamic semantic map.

[0013] Optionally, the task allocation scheme in S3 can also be generated based on: images of the occupancy status of the storage locations and images of the goods identified by the visual sensors at the shelf end. The weight and volume characteristics of the goods are determined through image recognition, and the load capacity of each vehicle is matched.

[0014] Optionally, the process for determining the dynamic safety distance threshold in S4 is as follows: Based on the three-dimensional image of the rear of the vehicle acquired by the binocular RGB-D camera, the outline size features of the vehicle are extracted, and combined with the vehicle's driving speed image and the channel illumination intensity image, the dynamic safety distance threshold is calculated through a visual ranging algorithm.

[0015] Optionally, the S5 also includes a visual guidance process for cargo docking: when the vehicle travels to the target cargo location, the binocular RGB-D camera captures a three-dimensional outline image of the pallet, extracts the center coordinates and attitude angle of the pallet, generates fork adjustment commands, and controls the forks to precisely dock with the pallet.

[0016] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system and method, the beneficial effects of which are: 1) This invention constructs a multi-dimensional machine vision perception system, integrating the perception resources of vehicle-mounted binocular RGB-D cameras, shelf-end vision sensors, and aisle-side panoramic cameras, realizing comprehensive perception of multiple elements such as vehicles, locations, goods, and aisles in the warehousing environment. Compared with existing single vision or laser positioning solutions, the perception range is more comprehensive and the dynamic adaptability is stronger, which can effectively cope with dynamic changes in the warehousing environment. 2) Based on multi-visual perception data, a dynamic semantic map of the warehouse is constructed. Through multi-view image fusion and 3D reconstruction, the environmental information is updated in real time, providing an accurate environmental basis for multi-vehicle collaborative decision-making and solving the problem that the preset path in the existing technology cannot adapt to the dynamic changes of the environment. 3) The dynamic features extracted by machine vision (such as multi-vehicle distribution, lane occupancy, cargo attributes, etc.) are deeply integrated into the multi-vehicle collaborative decision-making process, which realizes the precision of task allocation and path coordination. Especially in complex scenarios such as intersections and narrow channels, the passage timing and safe distance are calculated through visual perception, which significantly reduces the risk of vehicle congestion and collision and improves the efficiency of multi-vehicle collaborative operation. 4) No need to lay magnetic strips, QR codes or other auxiliary navigation facilities. It achieves autonomous navigation and collaboration based on machine vision, which reduces the initial construction cost and the later maintenance cost of the warehousing system, improves the system's flexibility and scalability, and can adapt to warehousing scenarios of different sizes and different storage locations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 The present invention provides a structural diagram of a machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system. Figure 2 The flowchart illustrates a collaborative decision-making method for multi-vehicle autonomous driving in intelligent warehousing based on machine vision, as provided by this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1 As shown, this invention discloses an intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system based on machine vision. The system includes a multi-vision perception module, a collaborative decision-making module, an on-board execution module, and a warehousing environment mapping module connected in sequence; wherein, The multi-vision perception module includes binocular RGB-D cameras deployed on each autonomous warehouse vehicle, shelf-end vision sensors deployed on the shelves, and aisle-side panoramic cameras deployed on the aisle side. The binocular RGB-D cameras on each autonomous warehouse vehicle are used to acquire images of the scene ahead along the vehicle's driving path, 3D contour images of the cargo pallets, and position and posture images of adjacent vehicles. The shelf-end vision sensors are used to acquire images of shelf location occupancy status and cargo identification images. The aisle-side panoramic cameras are used to acquire global scene images of the warehouse aisle and images of the distribution of multiple vehicles. Specifically, the multi-visual perception module: Vehicle-mounted binocular RGB-D camera: An industrial-grade binocular camera with a resolution of 1920×1080, a frame rate of 30fps, and a depth measurement range of 0.5-10m is selected. It is deployed on the front of each autonomous warehouse vehicle and above the forks. The front camera is used to collect images of the scene ahead of the driving path and the position and posture images of adjacent vehicles. The camera above the forks is used to collect three-dimensional contour images of the cargo pallet. Shelf-end vision sensors: High-definition industrial cameras are selected and deployed at the beams of each shelf location. One sensor is configured for every two locations to collect images of the location occupancy status and images of QR code / barcode markings on the surface of the goods. Panoramic cameras on the aisle side: 360° panoramic cameras are selected and deployed at the intersections and midpoints of the warehouse aisle. The cameras are 2.5m high and have a coverage radius of 15m. They are used to collect global scene images of the aisle and images of the distribution of multiple vehicles.

[0021] The warehouse environment mapping module is used to receive various image data output by the multi-visual perception module and generate a dynamic semantic map of the warehouse through image stitching, feature matching and 3D reconstruction. Specifically, it is implemented using an industrial-grade server, equipped with the Ubuntu 20.04 operating system and the ROS Noetic framework. After receiving preprocessed image data output from multiple vision perception modules, it performs image stitching through the OpenCV library (using a feature-based image stitching algorithm), and then performs 3D reconstruction through the PCL library (using the Poisson reconstruction algorithm) to generate a dynamic semantic map of the warehouse. The map is updated at a frequency of 10Hz. When the panoramic camera on the channel side detects a new obstacle in the channel using the background subtraction method, it triggers the binocular RGB-D camera of the autonomous warehouse vehicle within 5m of the obstacle to focus and capture images via Ethernet, obtaining obstacle images from 3-5 different perspectives. The three-dimensional size and position of the obstacle are determined by a multi-view image fusion algorithm (point cloud registration based on RANSAC algorithm) and updated to the warehouse dynamic semantic map.

[0022] The collaborative decision-making module includes an edge computing node and a vehicle-to-vehicle communication interaction unit. The edge computing node is used to receive the dynamic semantic map of the warehouse and the real-time status data of each autonomous warehouse vehicle, and generate multi-vehicle task allocation schemes and path collaboration strategies. The vehicle-to-vehicle communication interaction unit is used to realize the sharing of perception data and the interaction of decision commands among the autonomous warehouse vehicles. Specifically, edge computing nodes: edge servers equipped with Intel Core i7-12700 processors are selected and deployed in the central warehouse area. They communicate with each module through 5G industrial routers, run multi-vehicle collaborative decision-making algorithms, receive dynamic semantic maps of the warehouse and real-time status data (location, speed, load, battery level) of each autonomous warehouse vehicle, and generate multi-vehicle task allocation schemes and path collaboration strategies. Vehicle-to-vehicle communication and interaction unit: adopts WiFi 6 wireless communication module, with communication rate ≥1.2Gbps and latency ≤20ms, to realize perception data sharing and decision command interaction between autonomous warehouse vehicles.

[0023] The on-board execution module, deployed in each autonomous warehouse vehicle, is used to receive decision instructions output by the collaborative decision module and control the vehicle to complete path tracking, fork lifting, pallet docking, and obstacle avoidance actions.

[0024] Specifically, it includes an MCU controller, a path tracking actuator, a fork drive mechanism, and an obstacle avoidance actuator. The MCU controller uses an STM32H743 chip. After receiving the decision instructions from the collaborative decision module, it controls the path tracking actuator (servo motor and drive motor) through a PID algorithm to achieve precise vehicle path tracking. The servo motor controls the fork drive mechanism to complete the lifting and lowering of the forks and docking with the pallet. When the binocular RGB-D camera detects an obstacle that exceeds the safe distance, it triggers the obstacle avoidance actuator to achieve emergency braking or path deviation avoidance.

[0025] Furthermore, the multi-vision perception module also includes a visual data preprocessing unit, which performs noise filtering, exposure correction, and distortion correction on the acquired image data, and extracts feature points of shelf uprights, pallet corners, lane lines, and vehicle contours from the image through feature extraction algorithms.

[0026] Specifically, the visual data preprocessing unit uses an FPGA chip to achieve hardware-accelerated preprocessing. The specific processing flow is as follows: first, image noise is filtered by median filtering algorithm, then exposure correction is achieved based on histogram equalization, and finally distortion correction is performed by camera intrinsic parameter matrix. Feature extraction uses SIFT algorithm to extract shelf column feature points, pallet corner points, lane line features and vehicle contour features in the image. Feature point matching uses FLANN matcher.

[0027] Furthermore, the 3D reconstruction process of the warehouse environment mapping module includes: constructing a 3D model of the storage location based on multi-view images of the storage location collected by the visual sensors at the shelf end; merging the storage location model and the aisle model by combining the global images collected by the panoramic cameras on the aisle side; and updating the 3D coordinates of dynamic obstacles by using the depth images collected by the binocular RGB-D cameras of each autonomous warehouse vehicle.

[0028] and Figure 1 Corresponding to the aforementioned system, this invention also provides a machine vision-based collaborative decision-making method for multi-vehicle autonomous driving in intelligent warehousing, used for... Figure 1 For the specific implementation of the system, please refer to the flowchart. Figure 2 As shown, it includes the following steps: S1. The multi-vision perception module starts image acquisition. The binocular RGB-D camera of each autonomous warehouse vehicle acquires real-time images of the driving path scene and surrounding vehicles. The visual sensor at the shelf end acquires images of the location status and goods identification. The panoramic camera on the aisle side acquires global images of the aisle. S2, the warehouse environment mapping module receives various types of image data, performs image stitching and 3D reconstruction after preprocessing, and generates a dynamic semantic map of the warehouse containing information on storage locations, aisles, dynamic obstacles and goods, which is synchronized to the collaborative decision-making module in real time. S3, the collaborative decision-making module receives the warehouse dynamic semantic map and the location, speed and load status data of each autonomous warehouse vehicle. Based on the multi-vehicle distribution features and channel occupancy features extracted by machine vision, it generates a multi-vehicle task allocation scheme and clarifies the picking location, delivery location and driving priority of each vehicle. S4. The collaborative decision-making module identifies intersections and narrow passages in the task path of each vehicle by visual feature matching. Based on the real-time position images of multiple vehicles, it calculates the time difference between the arrival of vehicles at the intersection and generates a path collaboration strategy, including the passage order at intersections, the way to avoid oncoming traffic in narrow passages, and the dynamic safety distance threshold. S5. Each autonomous warehouse vehicle shares perception data and task execution status through the vehicle-to-vehicle communication interaction unit. The on-board execution module controls the vehicle's movement according to the collaborative decision-making instructions. It monitors path deviation and obstacle changes in real time through binocular RGB-D cameras, dynamically adjusts driving parameters, and completes the tasks of picking up and placing goods and driving along the path.

[0029] Furthermore, the update method of the warehouse dynamic semantic map in S2 is as follows: when the panoramic camera on the channel side detects a new obstacle in the channel, it simultaneously triggers the binocular RGB-D cameras of the surrounding vehicles to focus and capture images of the obstacle. The three-dimensional size and position of the obstacle are determined by multi-view image fusion and updated to the warehouse dynamic semantic map.

[0030] Furthermore, the task allocation scheme in S3 is generated based on the following: images of the occupancy status of the storage locations and images of the goods identified by the visual sensors at the shelf end. The weight and volume characteristics of the goods are determined through image recognition, and the load capacity of each vehicle is matched.

[0031] Furthermore, the process of determining the dynamic safe distance threshold in S4 is as follows: Based on the three-dimensional image of the rear of the vehicle acquired by the binocular RGB-D camera, the outline size features of the vehicle are extracted, and combined with the vehicle's driving speed image and the channel illumination intensity image, the dynamic safe distance threshold is calculated through a visual ranging algorithm.

[0032] Furthermore, the S5 also includes a visual guidance process for cargo docking: when the vehicle travels to the target cargo location, the binocular RGB-D camera captures a three-dimensional outline image of the pallet, extracts the center coordinates and attitude angle of the pallet, generates fork adjustment instructions, and controls the forks to precisely dock with the pallet.

[0033] In a specific embodiment: Step 1: The multi-vision perception module starts image acquisition. After the system is powered on, the cameras and sensors of the multi-vision perception module start up synchronously. The vehicle-mounted binocular RGB-D camera collects images of the scene ahead of the driving path in real time (sampling frequency 30fps) and images of surrounding vehicles. The visual sensor at the shelf end collects images of the location status and goods identification every 1 second. The panoramic camera on the aisle side collects a global image of the aisle every 0.5 seconds. All collected image data is transmitted to the visual data preprocessing unit in real time.

[0034] Step 2: Generation and updating of dynamic semantic map of warehouse

[0035] The warehouse environment mapping module receives the preprocessed image data. First, it stitches multiple global images captured by the panoramic camera on the aisle side into a complete panoramic image of the aisle using a feature-based image stitching algorithm. Then, it performs feature matching between the location image captured by the visual sensor at the shelf end and the panoramic image of the aisle to determine the location of the location in the aisle coordinate system. Finally, it combines the depth image captured by the vehicle-mounted binocular RGB-D camera to construct a 3D model of the location and a 3D model of the aisle, and merges them to generate an initial dynamic semantic map of the warehouse. After the map is generated, it is updated every 100ms. When the panoramic camera on the channel side detects a new obstacle, it triggers the binocular RGB-D cameras of the surrounding vehicles to focus and capture images of the obstacle from multiple perspectives. The point cloud is then registered using the RANSAC algorithm to determine the three-dimensional size and position of the obstacle. This information is then updated to the warehouse dynamic semantic map and synchronized to the collaborative decision-making module.

[0036] Step 3: Generation of multi-vehicle task allocation scheme

[0037] The edge computing node of the collaborative decision-making module receives the dynamic semantic map of the warehouse and the real-time status data of each autonomous warehouse vehicle (obtaining position and speed data through onboard GPS and encoders, and load data through pressure sensors). Based on the multi-vehicle distribution features extracted by machine vision (identifying the position of each vehicle through panoramic images of the aisle side) and aisle occupancy features (identifying the proportion of idle areas in the aisle through image segmentation algorithms), combined with the storage location occupancy status images collected by the visual sensors at the shelf end (identifying empty and full storage locations) and cargo identification images (reading cargo weight and volume information through QR code recognition algorithms), the module matches the load capacity of each vehicle (preset maximum load of 500kg and maximum carrying volume of 1m³ for each vehicle), and uses a genetic algorithm to generate a multi-vehicle task allocation scheme, clarifying the picking location, delivery location, and driving priority of each vehicle (vehicles corresponding to urgent orders have the highest driving priority).

[0038] Step 4: Generate Path Coordination Strategy

[0039] The collaborative decision-making module identifies intersections (channel intersections) and narrow channel sections (channels with a width of ≤2m) in the task paths of each vehicle using the SIFT feature matching algorithm. It obtains the real-time position coordinates of each vehicle based on panoramic images of the channel side and images from the vehicle-mounted camera, calculates the time difference between the arrival of vehicles at the intersection, and uses a greedy algorithm to generate the passage order at the intersection (vehicles that arrive at the intersection first and have lower priority give way to vehicles with higher priority). For narrow passages, a passing avoidance method is generated (specifying that vehicles on one side stop in the passing area on the side of the passage, and the other vehicle continues to drive after passing); the dynamic safe distance threshold is calculated using a vision-based ranging method: a three-dimensional image of the rear of the vehicle in front is acquired by an onboard binocular RGB-D camera, the length and width features of the vehicle's outline are extracted, and combined with the vehicle's speed (calculated by the movement speed of the lane lines in the image) and the light intensity of the passage (calculated by the grayscale value of the image), the dynamic safe distance threshold is calculated by a triangulation ranging algorithm. When the distance between the vehicle and the vehicle in front is less than the threshold, a deceleration command is triggered.

[0040] Step 5: Task Execution and Dynamic Adjustment

[0041] Each autonomous warehouse vehicle shares perception data and task execution status through the vehicle-to-vehicle communication interaction unit. The MCU controller of the on-board execution module controls the vehicle's movement according to the collaborative decision instructions. The on-board binocular RGB-D camera monitors the path deviation in real time (by matching the collected path images with the preset path images to calculate the deviation value) and adjusts the servo angle through the PID algorithm to achieve path tracking. When the vehicle reaches the target cargo location, the binocular RGB-D camera above the forks captures a 3D contour image of the pallet, extracts the pallet's center coordinates and attitude angle, detects the pallet's edge through Hough transform, calculates the center coordinates and rotation angle, generates fork adjustment commands, and controls the servo motor to drive the forks to lift, move, and translate, achieving precise docking with the pallet (docking error ≤ 5mm). During the journey, if changes in obstacles are detected (such as the addition of obstacles or the movement of obstacles), the driving parameters are adjusted in real time to complete the cargo loading and unloading and path driving tasks.

[0042] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0043] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A machine vision-based intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system, characterized in that, The system includes a multi-vision perception module, a collaborative decision-making module, a vehicle-mounted execution module, and a warehouse environment mapping module connected in sequence; among them, The multi-vision perception module includes binocular RGB-D cameras deployed on each autonomous warehouse vehicle, shelf-end vision sensors deployed on the shelves, and aisle-side panoramic cameras deployed on the aisle side. The binocular RGB-D cameras on each autonomous warehouse vehicle are used to acquire images of the scene ahead along the vehicle's driving path, 3D contour images of the cargo pallets, and position and posture images of adjacent vehicles. The shelf-end vision sensors are used to acquire images of shelf location occupancy status and cargo identification images. The aisle-side panoramic cameras are used to acquire global scene images of the warehouse aisle and images of the distribution of multiple vehicles. The warehouse environment mapping module is used to receive various image data output by the multi-visual perception module and generate a dynamic semantic map of the warehouse through image stitching, feature matching and 3D reconstruction. The collaborative decision-making module includes an edge computing node and a vehicle-to-vehicle communication interaction unit. The edge computing node is used to receive the dynamic semantic map of the warehouse and the real-time status data of each autonomous warehouse vehicle, and generate multi-vehicle task allocation schemes and path collaboration strategies. The vehicle-to-vehicle communication interaction unit is used to realize the sharing of perception data and the interaction of decision commands among the autonomous warehouse vehicles. The on-board execution module, deployed in each autonomous warehouse vehicle, is used to receive decision instructions output by the collaborative decision module and control the vehicle to complete path tracking, fork lifting, pallet docking, and obstacle avoidance actions.

2. The intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system based on machine vision according to claim 1, characterized in that, The multi-vision perception module also includes a visual data preprocessing unit, which performs noise filtering, exposure correction, and distortion correction on the acquired image data. It also extracts feature points of shelf uprights, pallet corners, lane lines, and vehicle contours from the image using feature extraction algorithms.

3. The intelligent warehousing multi-vehicle autonomous driving collaborative decision-making system based on machine vision according to claim 1, characterized in that, The 3D reconstruction process of the warehouse environment mapping module includes: constructing a 3D model of the storage location based on multi-view images of the storage location collected by the visual sensors at the shelf end; merging the storage location model and the aisle model by combining the global images collected by the panoramic cameras on the aisle side; and updating the 3D coordinates of dynamic obstacles by using the depth images collected by the binocular RGB-D cameras of each autonomous warehouse vehicle.

4. A collaborative decision-making method for multi-vehicle autonomous driving in intelligent warehousing based on machine vision, characterized in that... The intelligent warehouse multi-vehicle autonomous driving collaborative decision-making system based on machine vision, as described in any one of claims 1-3, includes the following steps: S1. The multi-vision perception module starts image acquisition. The binocular RGB-D camera of each autonomous warehouse vehicle acquires real-time images of the driving path scene and surrounding vehicles. The visual sensor at the shelf end acquires images of the location status and goods identification. The panoramic camera on the aisle side acquires global images of the aisle. S2, the warehouse environment mapping module receives various types of image data, performs image stitching and 3D reconstruction after preprocessing, and generates a dynamic semantic map of the warehouse containing information on storage locations, aisles, dynamic obstacles and goods, which is synchronized to the collaborative decision-making module in real time. S3, the collaborative decision-making module receives the warehouse dynamic semantic map and the location, speed and load status data of each autonomous warehouse vehicle. Based on the multi-vehicle distribution features and channel occupancy features extracted by machine vision, it generates a multi-vehicle task allocation scheme and clarifies the picking location, delivery location and driving priority of each vehicle. S4. The collaborative decision-making module identifies intersections and narrow passages in the task path of each vehicle by visual feature matching. Based on the real-time position images of multiple vehicles, it calculates the time difference between the arrival of vehicles at the intersection and generates a path collaboration strategy, including the passage order at intersections, the way to avoid oncoming traffic in narrow passages, and the dynamic safety distance threshold. S5. Each autonomous warehouse vehicle shares perception data and task execution status through the vehicle-to-vehicle communication interaction unit. The on-board execution module controls the vehicle's movement according to the collaborative decision-making instructions. It monitors path deviation and obstacle changes in real time through binocular RGB-D cameras, dynamically adjusts driving parameters, and completes the tasks of picking up and placing goods and driving along the path.

5. The intelligent warehousing multi-vehicle autonomous driving collaborative decision-making method based on machine vision according to claim 4, characterized in that, The update method of the warehouse dynamic semantic map in S2 is as follows: when the panoramic camera on the channel side detects a new obstacle in the channel, it simultaneously triggers the binocular RGB-D cameras of the surrounding vehicles to focus and capture images of the obstacle. The three-dimensional size and position of the obstacle are determined by multi-view image fusion and updated to the warehouse dynamic semantic map.

6. The intelligent warehousing multi-vehicle autonomous driving collaborative decision-making method based on machine vision according to claim 4, characterized in that, The task allocation scheme in S3 is also based on the following: images of the occupancy status of the storage locations and images of the goods identified by the visual sensors at the shelf end. The weight and volume characteristics of the goods are determined through image recognition, and the load capacity of each vehicle is matched.

7. The intelligent warehousing multi-vehicle autonomous driving collaborative decision-making method based on machine vision according to claim 4, characterized in that, The process of determining the dynamic safe distance threshold in S4 is as follows: Based on the three-dimensional image of the rear of the vehicle acquired by the binocular RGB-D camera, the outline size features of the vehicle are extracted, and combined with the vehicle's driving speed image and the channel illumination intensity image, the dynamic safe distance threshold is calculated through a visual ranging algorithm.

8. The intelligent warehousing multi-vehicle autonomous driving collaborative decision-making method based on machine vision according to claim 4, characterized in that, The S5 also includes a visual guidance process for cargo docking: when the vehicle travels to the target cargo location, the binocular RGB-D camera captures a three-dimensional outline image of the pallet, extracts the center coordinates and attitude angle of the pallet, generates fork adjustment commands, and controls the forks to precisely dock with the pallet.

Images