Unmanned delivery method and system for vegetable market

Abstract

The invention discloses an unmanned delivery method and system for a vegetable market, and relates to the technical field of data processing, and the method comprises the steps: carrying out the relative position analysis based on an obtained top marker image, and obtaining the relative position information of each unmanned delivery device; obstacle recognition is carried out on the target images in the four directions, and obstacles corresponding to all the directions are obtained; based on a clustering algorithm, distance and orientation analysis is carried out on the obstacles corresponding to all the directions, and the corresponding obstacle distance and orientation are obtained; calculating instantaneous path information of each unmanned delivery device based on a global path generated by the relative position information, the obstacle distance corresponding to each direction and the orientation; and when the unmanned automatic grabbing mechanism moves to the target position of the vegetable market, different rotating shafts of the unmanned automatic grabbing mechanism are coordinated based on the cargo grabbing position to grab the cargo into the cargo box, and the cargo is returned to the corresponding target stall area. According to the invention, the operation efficiency of the unmanned delivery device is improved, and the unmanned delivery device can be better applied to the vegetable market.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to an unmanned delivery method and system for a wet market. Background Art

[0002] With the increasing maturity of computer technology and artificial intelligence technology, automation technology and intelligent technology have been continuously applied. Driverless and unmanned delivery devices or systems have emerged in large numbers and gradually gained popularity. With the support of unmanned technology, unmanned delivery devices will be first applied in special scenarios. The wet market belongs to a special scenario and is also a major scenario for the application of unmanned systems. Currently, during the movement of unmanned delivery devices, they usually only identify obstacles in the front, while ignoring the impact of obstacles in other directions on the movement of unmanned delivery devices, resulting in the inability of unmanned delivery devices to move effectively in the wet market. At the same time, current unmanned delivery devices also lack the analysis of the distances of obstacles in all directions, resulting in insufficient accuracy of the finally generated path parameters, affecting the operation efficiency of unmanned delivery devices, causing the unmanned delivery devices to fail to reach the designated area of the wet market in time for loading, and thus unable to effectively improve the goods transportation efficiency. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides an unmanned delivery method and system for a wet market, which improves the operation efficiency of unmanned delivery devices and enables unmanned delivery devices to be better applied in the wet market.

[0004] To solve the above technical problems, the present invention provides an unmanned delivery method for a wet market, which is applied to an unmanned delivery device and a general control center. The unmanned delivery device includes an unmanned transport chassis, an unmanned automatic grasping mechanism, and a cargo box. The method includes:

[0005] When each unmanned delivery device moves, the general control center performs relative position analysis based on the top marker images obtained by the unmanned transport chassis to obtain the relative position information of each unmanned delivery device;

[0006] Perform obstacle recognition on the target images in four directions obtained by the unmanned transport chassis to obtain the obstacles corresponding to each direction;

[0007] Based on the clustering algorithm, perform distance and azimuth analysis on the obstacles corresponding to each direction to obtain the obstacle distances and obstacle azimuths corresponding to each direction;

[0008] Calculate the instantaneous path information of each unmanned delivery device based on the global path generated by the relative position information, the obstacle distances corresponding to each direction, and the obstacle azimuths;

[0009] When each unmanned delivery device moves to the target location in the vegetable market according to the instantaneous path information, the different rotating shafts of the unmanned automatic grasping mechanism are coordinated based on the analyzed cargo grasping position to grasp the goods into the cargo box and return to the corresponding target booth area.

[0010] Optionally, the relative position analysis based on the top marker images obtained by the unmanned transportation chassis to obtain the relative position information of each unmanned delivery device includes:

[0011] Perform top marker framing processing on the images obtained by the upward-looking camera of the unmanned transportation chassis to obtain the corresponding top marker images;

[0012] Perform marker serial number analysis on the top marker images to obtain the corresponding marker serial numbers;

[0013] Perform relative position analysis of each unmanned delivery device based on the marker serial numbers to obtain the corresponding relative position information.

[0014] Optionally, the obstacle recognition of the target images in four directions obtained by the unmanned transportation chassis to obtain the obstacles corresponding to each direction includes:

[0015] Perform random spatial projection data enhancement based on the normal distribution on the target images in each direction obtained to obtain the enhanced target images in each direction;

[0016] Extract the first image features at different scales of the enhanced target images, and perform segmentation processing on the enhanced target images based on the first image features at different scales and the segmentation masks corresponding to the first image features to obtain the enhanced target images after segmentation processing;

[0017] Extract the second image features at different scales of the enhanced target images after segmentation processing, and use the second image features to recognize the obstacles corresponding to each direction of the unmanned delivery device based on the obstacle recognition model obtained by parameter convergence using the clustering algorithm.

[0018] Optionally, the distance and azimuth analysis of the obstacles corresponding to each direction based on the clustering algorithm to obtain the obstacle distance and obstacle azimuth corresponding to each direction includes:

[0019] Based on the clustering algorithm and the data points collected by the four-line lidar of the unmanned transportation chassis, calculate the obstacle distance and azimuth to obtain the first obstacle distance and the first obstacle azimuth;

[0020] Based on the enhanced target images in each direction, calculate the obstacle distance and azimuth using feature point matching to obtain the second obstacle distance and the second obstacle azimuth;

[0021] Perform weighted fusion on the first obstacle distance and the second obstacle to obtain the obstacle distance corresponding to each direction, and perform weighted fusion on the first obstacle orientation and the second obstacle orientation to obtain the obstacle orientation corresponding to each direction.

[0022] Optionally, calculating the instantaneous path information of each unmanned delivery device based on the global path generated from the relative position information, the obstacle distance corresponding to each direction, and the obstacle orientation includes:

[0023] Construct an environmental domain based on the relative position information, the obstacle distance corresponding to each direction, and the obstacle orientation, and plan a global path based on the environmental domain;

[0024] Calculate the instantaneous speed, acceleration, and instantaneous rotation angle of each unmanned delivery device based on the global path, and plan the real-time local path sequence of each unmanned delivery device based on the global path.

[0025] Optionally, coordinating the different rotating shafts of the unmanned automatic grasping mechanism based on the analyzed cargo grasping position to grasp the cargo into the cargo box includes:

[0026] Perform loading quantity recognition on the acquired regional image to obtain a number of goods to be grasped;

[0027] Extract the target contours of a number of goods to be grasped from the regional image, and perform grasping position analysis on each cargo using force balance constraints based on the target contours to obtain the grasping positions of each cargo;

[0028] Analyze the telescoping and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism based on the grasping positions of each cargo;

[0029] Control the gripper to grasp each cargo into the cargo box according to the corresponding grasping position based on the telescoping and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism.

[0030] Optionally, controlling the gripper to grasp each cargo into the cargo box according to the corresponding grasping position includes:

[0031] Recognize the real-time motion state of the gripper when grasping each cargo and the real-time distance between each cargo and the cargo box based on the acquired cargo grasping image;

[0032] Cooperatively adjust the telescoping and rotation angles of the first rotating shaft and the second rotating shaft based on the real-time distance and the real-time motion state.

[0033] Optionally, after returning to the corresponding target booth area, it includes:

[0034] Recognize the pose information of each cargo in the cargo box, and analyze the corresponding unloading and grasping positions based on the pose information of each cargo;

[0035] Analyze the types of each cargo based on an image recognition algorithm, and match the corresponding placement positions of each cargo in the target booth area according to the types of each cargo;

[0036] Control the unmanned automatic grasping mechanism to grasp each cargo to the corresponding placement position in the target booth area according to the corresponding unloading and grasping positions.

[0037] In addition, the present invention also provides an unmanned distribution system for a vegetable market. The system includes an unmanned distribution device and a general control center. The unmanned distribution device includes an unmanned transport chassis, an unmanned automatic grasping mechanism, and a cargo box. The system is configured to execute the above-mentioned unmanned distribution method for a vegetable market.

[0038] Optionally, the unmanned transport chassis includes tires, a front anti-collision beam, a front motor, a front-view camera, a rear-view camera, a four-line lidar, an upward-view camera, a left-side camera, a rear anti-collision beam, a battery system, and a control unit;

[0039] The unmanned automatic grasping mechanism includes a first rotating shaft and its controller, a first cargo box camera, a second rotating shaft and its rotating shaft controller, a second cargo box camera, and a gripper.

[0040] In an embodiment of the present invention, relative position analysis is performed based on the top marker images obtained by the unmanned transport chassis, and the relative position information of each unmanned distribution device can be obtained more quickly and accurately. Obstacle recognition is performed on the target images in four directions obtained by the unmanned transport chassis, distance and azimuth analysis are performed on the obstacles corresponding to each direction based on a clustering algorithm, and the instantaneous path information of each unmanned distribution device is calculated based on the global path generated by the relative position information, the obstacle distances and obstacle azimuths corresponding to each direction. The influence of obstacles in each direction and their distances and azimuths on the movement process of the unmanned distribution device is comprehensively considered. Moving according to the calculated instantaneous path information can more reasonably avoid obstacles, so as to coordinate the movement of each unmanned distribution device, improve the operation efficiency of the unmanned distribution device, and enable it to reach the designated area of the vegetable market in time for loading. When each unmanned distribution device moves to the target position of the vegetable market, the different rotating shafts of the unmanned automatic grasping mechanism are coordinated based on the analyzed cargo grasping positions to grasp the cargo into the cargo box, improving the accuracy of cargo grasping and avoiding damage to the cargo. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 is a schematic flow chart of the unmanned delivery method for a wet market in an embodiment of the present invention;

[0043] Figure 2 is a schematic flow chart of the unmanned delivery method for a wet market in another embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of the structural composition of the unmanned delivery system for a wet market in an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of the structural composition of the unmanned delivery device in an embodiment of the present invention. Detailed implementation manners

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1

[0048] Please refer to Figure 1 , Figure 1 is a schematic flow chart of the unmanned delivery method for a wet market in an embodiment of the present invention. The method is applied to an unmanned delivery device and a general control center. The unmanned delivery device includes an unmanned transportation chassis, an unmanned automatic grasping mechanism, and a cargo box. The method includes:

[0049] S11: When each unmanned delivery device moves, the general control center performs relative position analysis based on the top marker image obtained by the unmanned transportation chassis to obtain the relative position information of each unmanned delivery device;

[0050] In the specific implementation process of the present invention, the performing relative position analysis based on the top marker image obtained by the unmanned transportation chassis to obtain the relative position information of each unmanned delivery device includes: performing top marker framing processing on the image obtained by the upward-looking camera of the unmanned transportation chassis to obtain the corresponding top marker image; performing marker serial number analysis on the top marker image to obtain the corresponding marker serial number; performing relative position analysis of each unmanned delivery device based on the marker serial number to obtain the corresponding relative position information.

[0051] Specifically, when each unmanned delivery device moves, the upward-looking camera of the unmanned transport chassis collects corresponding images and transmits the collected images to the general control center. A marker is set at the top between every two booths. The marker is made of a good reflective material and is easy to be recognized by the camera. The top marker in the obtained images is framed. Edge detection of the marker is performed on the obtained images according to edge detection to obtain the corresponding marker edge. The top marker is framed in the collected images based on the preset calibration frame and the corresponding marker edge to obtain the corresponding top marker image. Serial number analysis of the marker is performed on the top marker image. Each top marker contains a different serial number. There are marker points in the top marker image. Marker point recognition is performed on the top marker image. The corresponding serial number is determined according to the recognized marker points. For example, N21 represents the marker in the first column of the second row, and the corresponding serial number is obtained. Relative position analysis of each unmanned delivery device is performed based on the serial number. The current position information of the unmanned delivery device can be mapped from the serial number of the top marker to obtain the corresponding relative position information. Thus, the general control center can perform multi-robot cooperative control according to the obtained relative position information of each unmanned delivery device to ensure the safe and stable operation of multiple unmanned delivery devices.

[0052] S12: Perform obstacle recognition on the target images in four directions obtained by the unmanned transport chassis to obtain the corresponding obstacles in each direction;

[0053] In the specific implementation process of the present invention, the performing obstacle recognition on the target images in four directions obtained by the unmanned transport chassis to obtain the corresponding obstacles in each direction includes: performing random spatial projection data enhancement based on normal distribution on the obtained target images in each direction to obtain the enhanced target images in each direction; extracting the first image features at different scales of the enhanced target images, and performing segmentation processing on the enhanced target images based on the first image features at different scales and the segmentation masks corresponding to the first image features to obtain the enhanced target images after segmentation processing; extracting the second image features at different scales of the enhanced target images after segmentation processing, and using the second image features to recognize the corresponding obstacles in each direction of the unmanned delivery device by using the obstacle recognition model obtained by parameter convergence by the clustering algorithm.

[0054] Specifically, the front view, rear view, left view, and right view cameras in the unmanned transportation chassis are used to collect target images in the front, rear, left, and right directions respectively. The obtained target images in each direction are subjected to random spatial projection data augmentation based on the normal distribution. A random spatial projection matrix is constructed by the rotation matrix, translation matrix, scaling matrix, and shearing matrix. A normal factor is constructed from the preset image normal distribution parameters. The normal factor and the random spatial projection matrix are subjected to the Hadamard product to obtain the normal random spatial projection matrix, which can limit the transformation parameters of the random spatial projection matrix within the normal distribution range. The target images in each direction are transformed according to the normal random spatial projection matrix, that is, the target images in each direction are subjected to spatial projection of the normal distribution to achieve image enhancement. While restricting the distribution range of the output images, the valid data is retained, and the redundant data is deleted, enabling the subsequent model to learn more invariant features in the transformed images, improving its generalization ability, and obtaining the enhanced target images in each direction. The first image features at different scales of the enhanced target images are extracted, and the enhanced target images are input into the feature pyramid. The feature pyramid identifies the first image features at different scales of the enhanced target images from different scales, and each scale corresponds to a different resolution. Based on the first image features at different scales and the segmentation masks corresponding to the first image features, the enhanced target images are segmented. The first image features at each scale are subjected to convolution and fully connected processing, and the resolution probabilities of each preset mask are calculated for the first image features after convolution and fully connected processing to obtain the mask resolution probabilities. The corresponding preset masks are determined according to the mask resolution probabilities, and different-scale segmentation masks are determined according to the preset masks and the mask resolution probabilities in combination with the objective function composed of the mask resolution loss and the boundary loss. The enhanced target images are segmented according to the first image features at different scales and the segmentation masks to obtain the enhanced target images after segmentation processing. The second image features at different scales of the enhanced target images after segmentation processing are extracted, and the enhanced target images after segmentation processing are subjected to several dilated grouped convolutions to obtain the second image features at different scales. Based on the obstacle recognition model obtained by parameter convergence using the clustering algorithm, the second image features are used to identify the obstacles corresponding to each direction of the unmanned delivery device. The second image features at different scales are fused to obtain the target fusion features. The target fusion features are effectively enhanced according to the channel attention mechanism and the spatial attention mechanism to obtain the enhanced target fusion features. Based on the K-means clustering algorithm, the feature matrix generated by feature extraction from the training image sample set is preprocessed to obtain the mean, covariance, and mixture weight parameters. The mean, covariance, and mixture weight parameters are iteratively calculated according to the expectation maximization algorithm until the preset number of iterations is reached to obtain the convergence parameters. The obstacle recognition model is constructed using the Gaussian mixture model according to the convergence parameters, and the enhanced target fusion features are input into the obstacle recognition model to identify the obstacles in each direction of the unmanned delivery device.

[0055] S13: Analyze the distance and azimuth of the obstacles corresponding to each direction based on the clustering algorithm to obtain the obstacle distance and obstacle azimuth corresponding to each direction;

[0056] In the specific implementation process of the present invention, the analyzing the distance and azimuth of the obstacles corresponding to each direction based on the clustering algorithm to obtain the obstacle distance and obstacle azimuth corresponding to each direction includes: calculating the obstacle distance and azimuth based on the clustering algorithm combined with the data points collected by the four-line lidar of the unmanned transportation chassis to obtain the first obstacle distance and the first obstacle azimuth; calculating the obstacle distance and azimuth by using feature point matching based on the enhanced target images in each direction to obtain the second obstacle distance and the second obstacle azimuth; performing weighted fusion on the first obstacle distance and the second obstacle distance to obtain the obstacle distance corresponding to each direction, and performing weighted fusion on the first obstacle azimuth and the second obstacle azimuth to obtain the obstacle azimuth corresponding to each direction.

[0057] Specifically, calculating the obstacle distance and azimuth based on the clustering algorithm combined with the data points collected by the four-line lidar of the unmanned transportation chassis, clustering the data points of the four-line lidar to obtain multiple clusters, and calculating the distance and azimuth angle of the cluster center of each cluster relative to the lidar coordinate system, that is, obtaining the first obstacle distance and the first obstacle azimuth. Calculating the obstacle distance and azimuth by using feature point matching based on the enhanced target images in each direction, extracting the SURF feature points in the enhanced target images in each direction according to the accelerated Speeded Up Robust Features (SURF) algorithm, performing matching operations according to the SURF feature points, calculating the main direction angle difference between the SURF feature points, classifying the SURF feature point pairs with the main direction angle difference less than the preset threshold into one category, classifying the SURF feature point pairs greater than or equal to the preset threshold into one category, taking the category with a larger number as the correct matching feature point pairs, deleting the feature point pairs of the remaining categories, that is, deleting the feature point pairs with large fluctuations and large errors in the horizontal value, forming the three-dimensional data of the obstacle according to the feature point pairs, taking the optical center of the camera in the corresponding direction as the coordinate system origin, and calculating the distance value and azimuth angle according to the three-dimensional data and the coordinate system origin, that is, obtaining the second obstacle distance and the second obstacle azimuth. Performing weighted fusion on the first obstacle distance and the second obstacle distance, that is, performing weighted operations according to the first obstacle distance, the second obstacle distance and their corresponding confidence levels to obtain the obstacle distance corresponding to each direction, and performing weighted fusion on the first obstacle azimuth and the second obstacle azimuth, that is, performing weighted operations according to the first obstacle azimuth, the second obstacle azimuth and their corresponding confidence levels to obtain the obstacle azimuth corresponding to each direction.

[0058] S14: Calculate the instantaneous path information of each unmanned delivery device based on the global path generated from the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientations.

[0059] In the specific implementation process of the present invention, the calculating the instantaneous path information of each unmanned delivery device based on the global path generated from the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientations includes: constructing an environmental domain based on the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientations, and planning a global path based on the environmental domain; calculating the instantaneous speed, acceleration, and instantaneous rotation angle of each unmanned delivery device based on the global path, and planning a real-time local path sequence of each unmanned delivery device based on the global path.

[0060] Specifically, construct an environmental domain based on the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientations. The environmental domain is a grid map, and plan a global path based on the environmental domain. Each unmanned delivery device takes its own relative position as the center, and combines the obstacle distances and obstacle orientations corresponding to each of its directions to draw a global path including obstacles. Calculate the instantaneous speed, acceleration, and instantaneous rotation angle of each unmanned delivery device based on the global path, and plan a real-time local path sequence of each unmanned delivery device based on the global path, so as to accurately control the walking, braking, and turning actions of the unmanned delivery device.

[0061] S15: When each unmanned delivery device moves to the target position in the vegetable market according to the instantaneous path information, coordinate the different rotating shafts of the unmanned automatic grasping mechanism to grasp the goods into the cargo box based on the analyzed goods grasping positions, and return to the corresponding target booth area.

[0062] In the specific implementation process of the present invention, the coordinating the different rotating shafts of the unmanned automatic grasping mechanism to grasp the goods into the cargo box based on the analyzed goods grasping positions includes: identifying the loading quantity based on the acquired regional image to obtain a number of goods to be grasped; extracting the target contours of the number of goods to be grasped from the regional image, and analyzing the grasping positions of each good based on the target contours using force balance constraints to obtain the grasping positions of each good; analyzing the expansion and contraction and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism based on the grasping positions of each good; controlling the gripper to grasp each good into the cargo box according to the corresponding grasping positions based on the expansion and contraction and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism.

[0063] Further, the controlling the gripper to grasp each good into the cargo box according to the corresponding grasping positions includes: identifying the real-time motion state of the gripper when grasping each good and the real-time distance between each good and the cargo box based on the acquired goods grasping image; coordinately adjusting the expansion and contraction and rotation angles of the first rotating shaft and the second rotating shaft based on the real-time distance and real-time motion state.

[0064] Further, after returning to the corresponding target booth area, it includes: identifying the pose information of each cargo in the cargo box, and analyzing the corresponding unloading and grasping positions based on the pose information of each cargo; analyzing the types of each cargo based on an image recognition algorithm, and matching the corresponding placement positions of each cargo in the target booth area based on the types of each cargo; controlling the unmanned automatic grasping mechanism to grasp each cargo to the corresponding placement positions in the target booth area according to the corresponding unloading and grasping positions.

[0065] Specifically, based on the acquired regional image, the loading quantity is recognized. The acquired regional image is marked with detection frames based on a deep learning algorithm, and the number of marked detection frames is counted, which is the number of goods to be grabbed, thereby marking several goods to be grabbed. Based on the regional image, the target contours of several goods to be grabbed are extracted. Based on clustering segmentation, straight-line points and curve points corresponding to each good are generated using the edges, and piecewise fitting is continued based on the straight-line points and curve points to obtain the target contours. Based on the target contours, the grasping positions of each good are analyzed using force balance constraints, a grasping environment constraint domain is established, and several groups of alternative grasping points are determined based on the grasping environment constraint domain using the target contours. Quality evaluation is performed on each group of alternative grasping points, and based on the quality evaluation results, the target grasping points for each good are determined among several groups of alternative grasping points. Based on the target grasping points, the X-axis component, Y-axis component, and Z-axis component of the gripper for each good in the reference coordinate system are determined in combination with three-dimensional friction constraints and internal force balance constraints. Based on the X-axis component, Y-axis component, and Z-axis component, the grasping positions of each good are determined. Based on the grasping positions of each good, the extension and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism are analyzed, that is, the extension of the rotating shaft is determined according to the Z-axis component, and the rotation angle of the rotating shaft is determined according to the X-axis component and the Y-axis component. Based on the extension and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism, the gripper is controlled to grab each good to the cargo box according to the corresponding grasping position. When the gripper grabs each good to the cargo box according to the corresponding grasping position, the real-time motion state of the gripper when grabbing each good and the real-time distance between each good and the cargo box are recognized based on the acquired cargo grasping image. When the gripper grabs each good using the cargo grasping image through a deep learning algorithm, the real-time motion states of the first rotating shaft, the second rotating shaft, and the gripper are recognized. The positions of the goods are marked based on the cargo grasping image, and the real-time distance between the goods and the cargo box is calculated based on the marked positions of the goods. Based on the real-time distance and the real-time motion state, the extension and rotation angles of the first rotating shaft and the second rotating shaft are adjusted collaboratively. In order to ensure that the goods are accurately placed in the cargo box, the states of the corresponding devices need to be adjusted according to the real-time distance between the goods and the cargo box. After returning to the corresponding target booth area, the pose information of each good in the cargo box is recognized, images of each good in the cargo box are collected, downsampling processing is performed on the images of each good, a covariance matrix corresponding to the downsampled image is constructed, the pose information of each good is analyzed based on the covariance matrix using the iterative closest point algorithm, and the unloading grasping positions of the corresponding goods are analyzed based on the pose information of each good. Similarly, the unloading grasping positions of each good are analyzed in combination with force balance constraints.Analyze the types of each cargo based on the image recognition algorithm, extract the features of the images of each cargo, obtain the image features of each cargo in the cargo box, and match the image features of each cargo with the type features in the type feature library. The Euclidean distance between the features can be calculated, and the type label corresponding to the type feature with the largest Euclidean distance is used as the type corresponding to the cargo. Cargoes of different types need to be placed in different positions. Match the corresponding placement positions of each cargo in the target booth area based on the type of each cargo. Control the unmanned automatic grasping mechanism to grasp each cargo to the corresponding placement position in the target booth area according to the corresponding unloading and grasping position.

[0066] In the embodiment of the present invention, relative position analysis is performed based on the top marker images obtained by the unmanned transportation chassis, and the relative position information of each unmanned delivery device can be obtained more quickly and accurately. Obstacle recognition is performed on the target images in four directions obtained by the unmanned transportation chassis. Based on the clustering algorithm, distance and azimuth analysis are performed on the obstacles corresponding to each direction. Based on the global path generated by the relative position information, the obstacle distances and obstacle azimuths corresponding to each direction, the instantaneous path information of each unmanned delivery device is calculated. The influence of obstacles in each direction and their distances and azimuths on the movement process of the unmanned delivery device is comprehensively considered. Moving according to the calculated instantaneous path information can more reasonably avoid obstacles, thereby coordinating the movement of each unmanned delivery device and improving the operation efficiency of the unmanned delivery device, enabling it to reach the designated area of the vegetable market in time for loading. When each unmanned delivery device moves to the target position of the vegetable market, the unmanned automatic grasping mechanism is coordinated based on the analyzed cargo grasping position to grasp the cargo into the cargo box, improving the accuracy of cargo grasping and avoiding damage to the cargo.

[0067] Embodiment 2

[0068] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an unmanned delivery method for a vegetable market in another embodiment of the present invention. The method is applied to an unmanned delivery device and a general control center. The unmanned delivery device includes an unmanned transportation chassis, an unmanned automatic grasping mechanism, and a cargo box. The method includes:

[0069] S201: When each unmanned delivery device moves, the general control center performs relative position analysis based on the top marker images obtained by the unmanned transportation chassis to obtain the relative position information of each unmanned delivery device;

[0070] In the specific implementation process of the present invention, when each unmanned delivery device moves, the upward-looking camera of the unmanned transportation chassis collects corresponding images and transmits the collected images to the general control center. A marker is set at the top between every two booths. The marker is made of a good reflective material and is easy to be recognized by the camera. Perform top marker framing processing on the obtained images, perform edge detection of the marker on the obtained images according to edge detection, obtain the corresponding marker edge, and frame the top marker in the collected image according to the preset calibration frame and the corresponding marker edge to obtain the corresponding top marker image. Perform marker serial number analysis on the top marker image. Each top marker contains a different marker serial number. There are marker points in the top marker image. Perform marker point recognition on the top marker image, determine the corresponding marker serial number according to the recognized marker points, such as N21, indicating the marker in the first column of the second row, and obtain the corresponding marker serial number. Based on the marker serial number, perform relative position analysis of each unmanned delivery device. The current position information of the unmanned delivery device can be mapped from the marker serial number of the top marker to obtain the corresponding relative position information. Thus, the general control center can perform multi-machine cooperative control according to the obtained relative position information of each unmanned delivery device to ensure the safe and stable operation of multiple unmanned delivery devices.

[0071] S202: Perform random spatial projection data augmentation based on the normal distribution on the obtained target images in each direction to obtain the enhanced target images in each direction;

[0072] In the specific implementation process of the present invention, the forward-looking, rear-looking, left-looking, and right-looking cameras in the unmanned transportation chassis respectively collect target images in the front, rear, left, and right directions. Perform random spatial projection data augmentation based on the normal distribution on the obtained target images in each direction. Construct a random spatial projection matrix from the rotation matrix, translation matrix, scaling matrix, and shear matrix. Construct a normal factor from the preset image normal distribution parameters. Perform a Hadamard product on the normal factor and the random spatial projection matrix to obtain a normal random spatial projection matrix, which can limit the transformation parameters of the random spatial projection matrix within the normal distribution range. Perform transformation processing on the target images in each direction according to the normal random spatial projection matrix, that is, perform spatial projection of the normal distribution on the target images in each direction to achieve image enhancement. While restricting the distribution range of the output images, retain the valid data and delete the redundant data, enabling the subsequent model to learn more invariant features in the transformed images and improving its generalization ability to obtain the enhanced target images in each direction.

[0073] S203: Extract the first image features at different scales of the enhanced target images, and perform segmentation processing on the enhanced target images based on the first image features at different scales and the segmentation masks corresponding to the first image features to obtain the enhanced target images after segmentation processing;

[0074] In the specific implementation process of the present invention, first image features of different scales of the enhanced target image are extracted, and the enhanced target image is input into a feature pyramid. The feature pyramid identifies first image features of different scales of the enhanced target image at different scales, and each scale corresponds to a different resolution. Based on the first image features of different scales and the segmentation masks corresponding to the first image features, the enhanced target image is segmented. The first image features of each scale are subjected to convolution and fully connected processing, and the resolution probabilities of each preset mask are calculated for the first image features after convolution and fully connected processing to obtain mask resolution probabilities. The corresponding preset masks are determined according to the mask resolution probabilities, and different-scale segmentation masks are determined according to the preset masks and the mask resolution probabilities in combination with an objective function composed of a mask resolution loss and a boundary loss. The enhanced target image is segmented according to the first image features of different scales and the segmentation masks to obtain the enhanced target image after segmentation processing.

[0075] S204: Extract second image features of different scales of the enhanced target image after segmentation processing, and use the second image features to identify obstacles corresponding to each direction of the unmanned delivery device based on an obstacle recognition model obtained by parameter convergence using a clustering algorithm;

[0076] In the specific implementation process of the present invention, second image features of different scales of the enhanced target image after segmentation processing are extracted, and the enhanced target image after segmentation processing is subjected to several dilated grouped convolutions to obtain second image features of different scales. Based on an obstacle recognition model obtained by parameter convergence using a clustering algorithm, the second image features are used to identify obstacles corresponding to each direction of the unmanned delivery device. The second image features of different scales are fused to obtain a target fusion feature. The target fusion feature is effectively enhanced according to a channel attention mechanism and a spatial attention mechanism to obtain an enhanced target fusion feature. Based on the K-means clustering algorithm, preprocessing is performed on a feature matrix generated by feature extraction from a training image sample set to obtain mean, covariance, and mixture weight parameters. The mean, covariance, and mixture weight parameters are iteratively calculated according to the expectation maximization algorithm until a preset number of iterations is reached to obtain convergence parameters. An obstacle recognition model is constructed using a Gaussian mixture model according to the convergence parameters, and the enhanced target fusion feature is input into the obstacle recognition model to identify obstacles corresponding to each direction of the unmanned delivery device.

[0077] S205: Calculate the distance and azimuth of obstacles based on a clustering algorithm in combination with data points collected by a four-line lidar of an unmanned transportation chassis to obtain a first obstacle distance and a first obstacle azimuth;

[0078] In the specific implementation process of the present invention, based on the clustering algorithm and combining the data points collected by the four-line lidar of the unmanned transportation chassis, the distance and azimuth of the obstacles are calculated. The data points of the four-line lidar are clustered to obtain multiple clusters, and the distance and azimuth angle of the cluster center of each cluster relative to the lidar coordinate system are calculated, that is, the first obstacle distance and the first obstacle azimuth are obtained.

[0079] S206: Based on the enhanced target images in each direction, use feature point matching to calculate the distance and azimuth of the obstacles, and obtain the second obstacle distance and the second obstacle azimuth;

[0080] In the specific implementation process of the present invention, based on the enhanced target images in each direction, use feature point matching to calculate the distance and azimuth of the obstacles. Extract the SURF feature points in the enhanced target images in each direction according to the Speeded Up Robust Features (SURF) algorithm. Perform a matching operation according to the SURF feature points, calculate the main direction angle difference between the SURF feature points, classify the SURF feature point pairs with the main direction angle difference less than the preset threshold into one category, classify the SURF feature point pairs with the main direction angle difference greater than or equal to the preset threshold into one category, take the category with a larger number as the correct matching feature point pair, and delete the feature point pairs of the remaining categories, that is, delete the feature point pairs with large fluctuations and large errors in the horizontal value. Form the three-dimensional data of the obstacles according to the feature point pairs, take the optical center of the camera in the corresponding direction as the origin of the coordinate system, and calculate the distance value and azimuth angle according to the three-dimensional data and the origin of the coordinate system, that is, obtain the second obstacle distance and the second obstacle azimuth.

[0081] S207: Perform weighted fusion on the first obstacle distance and the second obstacle to obtain the obstacle distance corresponding to each direction, and perform weighted fusion on the first obstacle azimuth and the second obstacle azimuth to obtain the obstacle azimuth corresponding to each direction;

[0082] In the specific implementation process of the present invention, perform weighted fusion on the first obstacle distance and the second obstacle, that is, perform a weighted operation according to the first obstacle distance, the second obstacle distance and their corresponding confidence levels to obtain the obstacle distance corresponding to each direction. Perform weighted fusion on the first obstacle azimuth and the second obstacle azimuth, that is, perform a weighted operation according to the first obstacle azimuth, the second obstacle azimuth and their corresponding confidence levels to obtain the obstacle azimuth corresponding to each direction.

[0083] S208: Calculate the instantaneous path information of each unmanned delivery device based on the global path generated by the relative position information, the obstacle distance corresponding to each direction and the obstacle azimuth;

[0084] In the specific implementation process of the present invention, an environmental domain is constructed based on the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientations. The environmental domain is a grid map, and a global path is planned based on the environmental domain. Each unmanned delivery device takes its own relative position as the center and combines the obstacle distances and obstacle orientations corresponding to its respective directions to draw a global path including obstacles. Based on the global path, the instantaneous speed, acceleration, and instantaneous rotation angle of each unmanned delivery device are calculated, and a real-time local path sequence of each unmanned delivery device is planned based on the global path, so as to accurately control the walking, braking, and steering actions of the unmanned delivery device.

[0085] S209: When each unmanned delivery device moves to the target position in the vegetable market according to the instantaneous path information, the different rotating shafts of the unmanned automatic grasping mechanism are coordinated based on the analyzed cargo grasping position to grasp the cargo into the cargo box and return to the corresponding target booth area.

[0086] In the specific implementation process of the present invention, the loading quantity is recognized based on the acquired regional image. The acquired regional image is marked with detection frames based on a deep learning algorithm, and the number of marked detection frames is counted, which is the number of goods to be grabbed, so as to mark several goods to be grabbed. The target contours of several goods to be grabbed are extracted based on the regional image. Straight-line points and curve points corresponding to each good are generated based on the edge by clustering segmentation, and piecewise fitting is continued according to the straight-line points and curve points to obtain the target contour. Based on the target contour, the grasping positions of each good are analyzed using force balance constraints, a grasping environment constraint domain is established, and several groups of alternative grasping points are determined based on the target contour according to the grasping environment constraint domain. The quality of each group of alternative grasping points is evaluated, and the target grasping points for each good are determined from several groups of alternative grasping points according to the quality evaluation results. According to the target grasping points, the X-axis component, Y-axis component, and Z-axis component of the gripper for each good in the reference coordinate system are determined based on three-dimensional friction constraints and internal force balance constraints, and the grasping positions of each good are determined according to the X-axis component, Y-axis component, and Z-axis component. Based on the grasping positions of each good, the extension and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism are analyzed, that is, the extension of the rotating shaft is determined according to the Z-axis component, and the rotation angle of the rotating shaft is determined according to the X-axis component and the Y-axis component. Based on the extension and rotation angles of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism, the gripper is controlled to grab each good to the cargo box according to the corresponding grasping position. When the gripper grabs each good to the cargo box according to the corresponding grasping position, the real-time motion state of the gripper when grabbing each good and the real-time distance between each good and the cargo box are recognized based on the acquired image of the goods being grabbed. When the deep learning algorithm uses the image of the goods being grabbed to recognize the real-time motion state of the first rotating shaft, the second rotating shaft, and the gripper when the gripper grabs each good, the position of the goods is marked according to the image of the goods being grabbed, and the real-time distance between the goods and the cargo box is calculated according to the marked position of the goods. Based on the real-time distance and the real-time motion state, the extension and rotation angles of the first rotating shaft and the second rotating shaft are adjusted collaboratively. In order to ensure that the goods are accurately placed in the cargo box, the state of the corresponding device needs to be adjusted according to the real-time distance between the goods and the cargo box. After returning to the corresponding target booth area, the pose information of each good in the cargo box is recognized, the image of each good in the cargo box is collected, the image of each good is downsampled, the covariance matrix corresponding to the downsampled image is constructed, the pose information of each good is analyzed using the iterative closest point algorithm according to the covariance matrix, and the unloading grasping position corresponding to each good is analyzed based on the pose information of each good. Similarly, the unloading grasping position of each good is analyzed by combining force balance constraints.Analyze the types of each cargo based on the image recognition algorithm, extract the features of the images of each cargo, obtain the image features of each cargo in the cargo box, match the image features of each cargo with the type features in the type feature library, and calculate the Euclidean distance between the features. The type label corresponding to the type feature with the largest Euclidean distance is used as the type corresponding to the cargo. Different types of cargo need to be placed in different positions. Match the placement positions corresponding to each cargo in the target booth area based on the types of each cargo. Control the unmanned automatic grasping mechanism to grasp each cargo to the corresponding placement position in the target booth area according to the corresponding unloading and grasping position.

[0087] In the embodiment of the present invention, relative position analysis is performed based on the top marker images obtained by the unmanned transportation chassis, and the relative position information of each unmanned delivery device can be obtained more quickly and accurately. Obstacle recognition is performed on the target images in four directions obtained by the unmanned transportation chassis, and distance and azimuth analysis of the obstacles corresponding to each direction is performed based on the clustering algorithm. The instantaneous path information of each unmanned delivery device is calculated based on the global path generated by the relative position information, the obstacle distances and obstacle azimuths corresponding to each direction, comprehensively considering the influence of the obstacles in each direction and their distances and azimuths on the movement process of the unmanned delivery device. Moving according to the calculated instantaneous path information can more reasonably avoid obstacles, thereby coordinating the movement of each unmanned delivery device and improving the operation efficiency of the unmanned delivery device, enabling it to reach the designated area of the vegetable market in time for loading. When each unmanned delivery device moves to the target position of the vegetable market, the different rotating shafts of the unmanned automatic grasping mechanism are coordinated based on the analyzed cargo grasping position to grasp the cargo into the cargo box, improving the accuracy of cargo grasping and avoiding damage to the cargo.

[0088] Embodiment III

[0089] Please refer to Figure 3 , Figure 3 which is a schematic structural composition diagram of the unmanned delivery system for the vegetable market in the embodiment of the present invention. The system includes an unmanned delivery device and a general control center. The unmanned delivery device includes an unmanned transportation chassis, an unmanned automatic grasping mechanism, and a cargo box. The system is configured to execute the unmanned delivery method for the vegetable market in the above embodiment.

[0090] In the specific implementation process of the present invention, the schematic structural composition diagram of the unmanned delivery device is as shown in Figure 4As shown in the figure, the driverless transport chassis includes tires 1, front anti-collision beam 2, front-mounted motor 3, front-view camera 4, rear-view camera 15, four-line lidar 5, upward-looking camera 6, left-side camera 7, right-view camera 16, rear anti-collision beam 8, battery system and control unit 9; the driverless automatic grasping mechanism includes a first rotating shaft and its controller 10, a first cargo box camera 11, a second rotating shaft and its rotating shaft controller 13, a second cargo box camera 12 and a gripper 14, and the cargo box 17 serves as a container.

[0091] Specifically, the unmanned delivery device is connected to the general control center through a local area network. The general control center analyzes the data transmitted by the unmanned delivery device to coordinate the motion states of each unmanned delivery device. The unmanned transport chassis in the unmanned delivery device is mainly responsible for the autonomous driving and autonomous carrying functions of the unmanned delivery device, realizing the autonomous driving of the unmanned delivery device from the starting point to the end point. Its unmanned transport chassis includes tires 1, front anti-collision beam 2, front motor 3, front-view camera 4, rear-view camera 15, four-line lidar 5, upward-looking camera 6, left-side camera 7, right-view camera 16, rear anti-collision beam 8, battery system and control unit 9. The front motor 3 of the unmanned transport chassis is symmetric with the battery system and control unit 9, forming front and rear counterweights to increase the stability of the unmanned transport chassis; the front motor 3 converts the electrical energy of the battery into mechanical energy to provide driving force for the unmanned transport chassis. The front anti-collision beam 2 and rear anti-collision beam 8 of the unmanned transport chassis are mainly the last safety barriers of the unmanned transport chassis. When the unmanned transport chassis collides with surrounding obstacles front and rear, the resistance value on the front and rear anti-collision beams will change; when the controller of the unmanned transport chassis receives the change in the anti-collision beam resistance value, the controller of the unmanned transport chassis will issue an emergency braking command to the braking system, and the unmanned transport chassis will execute the maximum braking deceleration and stop in the shortest time. The unmanned transport chassis adopts a sensing solution of front-view camera 4, rear-view camera 15, four-line lidar 5, left-side camera 7 and right-view camera 16. This solution can save costs and reduce the manufacturing cost of the device while ensuring safety; at the same time, this sensing solution can ensure the safety of the unmanned transport chassis and the unmanned delivery device. The front-view camera 4 and the four-line lidar 5 mainly identify the obstacles in front of the unmanned delivery device through image recognition algorithms and clustering algorithms to ensure the safety of the driving environment in front of the unmanned delivery device; the left-side camera 7 / right-side camera 16 / rear-view camera 15 mainly identify the obstacles on the left, right and rear sides of the unmanned delivery device through image recognition algorithms to ensure the safety of the driving environment on the left, right and rear sides of the unmanned delivery device. The upward-looking camera 6 mainly identifies the top positioning information and determines the location information of the unmanned delivery device by recognizing the top serial number through images. The unmanned automatic grasping mechanism includes the first rotating shaft and its controller 10, the first cargo box camera 11, the second rotating shaft and its rotating shaft controller 13, the second cargo box camera 12 and the gripper 14. The first rotating shaft and its controller 10 mainly control the telescoping and turning of the first rotating shaft, and cooperate with the second rotating shaft to reasonably adjust the grasping position of the gripper; the second rotating shaft and its rotating shaft controller 13 mainly control the telescoping and turning of the second rotating shaft, and cooperate with the first rotating shaft to reasonably adjust the grasping position of the gripper, so that the gripper 14 can accurately grasp the goods.The first cargo box camera 11 monitors the movement status of the gripper of the unmanned delivery device and obtains the position of the gripper. The second cargo box camera 12 mainly collects the situation of the goods in the cargo box. When loading, it identifies the loading quantity of the cargo box, and when unloading, it identifies the unloading quantity. In addition, the second cargo box camera 12 also obtains the goods grasping position through image recognition technology and transmits the grasping position information to the general control center. The general control center first controls the telescoping and rotation angle of the first rotating shaft and the second rotating shaft to adjust the gripper to an appropriate grasping position, and then the control center controls the gripper to tighten and open to grasp or release the goods. The cargo box 17 is used as a container to load goods such as vegetables. At the same time,... Figure 3 For the specific implementation manners of the shown unmanned delivery system, reference may be made to the above embodiments, which will not be elaborated herein.

[0092] In the embodiments of the present invention, based on the top marker images obtained by the unmanned transportation chassis, relative position analysis is performed, and the relative position information of each unmanned delivery device can be obtained more quickly and accurately. Obstacle recognition is performed on the target images in four directions obtained by the unmanned transportation chassis. Based on the clustering algorithm, distance and azimuth analysis are performed on the obstacles corresponding to each direction. Based on the global path generated by the relative position information, the obstacle distances and obstacle azimuths corresponding to each direction, the instantaneous path information of each unmanned delivery device is calculated. The influence of obstacles in each direction and their distances and azimuths on the movement process of the unmanned delivery device is comprehensively considered. Moving according to the calculated instantaneous path information can more reasonably avoid obstacles, so as to coordinate the movement of each unmanned delivery device, improve the operation efficiency of the unmanned delivery device, and enable it to reach the designated area of the vegetable market in time for loading. When each unmanned delivery device moves to the target position of the vegetable market, based on the analyzed goods grasping position, different rotating shafts of the unmanned automatic grasping mechanism are coordinated to grasp the goods into the cargo box, improving the accuracy of goods grasping and avoiding damage to the goods.

[0093] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0094] In addition, the above has introduced in detail an unmanned delivery method and system for a wet market provided by the embodiments of the present invention. Specific examples should have been used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An unmanned delivery method for a vegetable market, characterized in that: Applied to an unmanned distribution device and a general control center, the unmanned distribution device includes an unmanned transport chassis, an unmanned automatic grabbing mechanism and a cargo box; the method includes: When each unmanned delivery device moves, the general control center performs relative position analysis based on the top marker image obtained by the unmanned transport chassis to obtain the relative position information of each unmanned delivery device; Obstacle recognition is performed on the target images in four directions acquired by the unmanned transport chassis to obtain the obstacles corresponding to each direction; Based on the clustering algorithm, the distance and orientation of obstacles corresponding to each direction are analyzed to obtain the obstacle distance and obstacle orientation corresponding to each direction; Calculate the instantaneous path information of each unmanned delivery device based on the global path generated by the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientation; When each unmanned delivery device moves to the target location of the vegetable market according to the instantaneous path information, the different rotating axes of the unmanned automatic grabbing mechanism are coordinated based on the analyzed cargo grabbing position to grab the cargo into the cargo box and return to the corresponding target stall area.

2. The unmanned delivery method for vegetable markets according to claim 1, characterized in that: The relative position analysis based on the top marker image obtained by the unmanned transport chassis to obtain the relative position information of each unmanned delivery device includes: Performing top marker framing processing on the image acquired by the upward-looking camera of the unmanned transport chassis to obtain a corresponding top marker image; Performing a marker serial number analysis on the top marker image to obtain a corresponding marker serial number; The relative position analysis of each unmanned delivery device is performed based on the mark serial number to obtain the corresponding relative position information.

3. The unmanned delivery method for vegetable markets according to claim 1, characterized in that: The obstacle recognition is performed on the target images in four directions acquired by the unmanned transport chassis to obtain obstacles corresponding to each direction, including: Performing random spatial projection data enhancement based on normal distribution on the acquired target images in each direction to obtain enhanced target images in each direction; Extracting first image features of different scales of the enhanced target image, performing segmentation processing on the enhanced target image based on the first image features of different scales and the segmentation mask corresponding to the first image features, and obtaining the enhanced target image after the segmentation processing; The second image features of different scales of the enhanced target image after segmentation processing are extracted, and obstacles corresponding to various directions of the unmanned delivery device are identified using the second image features based on an obstacle recognition model obtained by parameter convergence of a clustering algorithm.

4. The unmanned delivery method for vegetable markets according to claim 1, characterized in that: The method of performing distance and orientation analysis on obstacles corresponding to each direction based on a clustering algorithm to obtain obstacle distances and obstacle orientations corresponding to each direction includes: The obstacle distance and orientation are calculated based on the clustering algorithm combined with the data points collected by the four-line laser radar of the unmanned transport chassis to obtain the first obstacle distance and the first obstacle orientation; Based on the enhanced target images in each direction, the obstacle distance and orientation are calculated by using feature point matching to obtain a second obstacle distance and a second obstacle orientation; The first obstacle distance and the second obstacle are weightedly fused to obtain the obstacle distance corresponding to each direction, and the first obstacle orientation and the second obstacle orientation are weightedly fused to obtain the obstacle orientation corresponding to each direction.

5. The unmanned delivery method for vegetable markets according to claim 1, characterized in that: The instantaneous path information of each unmanned delivery device is calculated based on the global path generated by the relative position information, the obstacle distances corresponding to each direction, and the obstacle orientation, including: Constructing an environment domain based on the relative position information, the obstacle distances and obstacle orientations corresponding to each direction, and planning a global path based on the environment domain; The instantaneous speed, acceleration and instantaneous turning angle of each unmanned delivery device are calculated based on the global path, and the real-time local path sequence of each unmanned delivery device is planned based on the global path.

6. The unmanned delivery method for vegetable markets according to claim 1, characterized in that: The method of coordinating different rotating shafts of the unmanned automatic grabbing mechanism based on the analyzed grabbing position of the cargo to grab the cargo into the cargo box includes: Cargo volume recognition is performed based on the acquired regional image to obtain a number of cargoes to be grabbed; Extracting target contours of several goods to be grasped based on the regional image, and analyzing the grasping position of each of the goods based on the target contours using force balance constraints to obtain the grasping position of each of the goods; Analyzing the extension and rotation angle of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism based on the grasping position of each cargo; Based on the extension and rotation angle of the first rotating shaft and the second rotating shaft of the unmanned automatic grasping mechanism, the grasper grasps each cargo into the cargo box according to the corresponding grasping position.

7. The unmanned delivery method for vegetable markets according to claim 6, characterized in that: The control gripper grabs each cargo into the cargo box according to the corresponding grabbing position, including: Based on the acquired cargo grabbing images, the real-time motion state of the grabber when grabbing each cargo and the real-time distance between each cargo and the cargo box are identified; The extension and rotation angle of the first rotating shaft and the second rotating shaft are cooperatively adjusted based on the real-time distance and the real-time motion state.

8. The unmanned delivery method for vegetable markets according to claim 1, characterized in that: After returning to the corresponding target stall area, the method includes: Identify the position information of each item in the cargo box, and analyze the corresponding unloading grabbing position based on the position information of each item; Analyze the types of goods based on image recognition algorithms, and match the corresponding placement of each product in the target stall area based on the type of each product; Control the unmanned automatic grabbing mechanism to grab each product to the corresponding placement position in the target stall area according to the corresponding unloading grabbing position.

9. An unmanned distribution system for a vegetable market, characterized in that: The system includes an unmanned delivery device and a general control center. The unmanned delivery device includes an unmanned transport chassis, an unmanned automatic grasping mechanism and a cargo box. The system is configured to execute the unmanned delivery method for a vegetable market as described in any one of claims 1 to 8.

10. The unmanned delivery system according to claim 9, characterized in that: The unmanned transport chassis includes tires, a front anti-collision beam, a front motor, a front-view camera, a rear-view camera, a four-line laser radar, an upward-looking camera, a left-side camera, a rear anti-collision beam, a battery system and a control unit; The unmanned automatic grabbing mechanism includes a first rotating shaft and a controller thereof, a first cargo box camera, a second rotating shaft and a rotating shaft controller thereof, a second cargo box camera and a grabber.

Images