Intelligent logistics robot picking method and system based on visual perception

Abstract

This invention relates to the field of logistics robot technology, and in particular to a visual perception-based intelligent logistics robot picking method and system. The method involves mapping and affine transformation to obtain an ideal grasping area space for the target goods, using information gain to explore the perspective of the ideal grasping area space, and obtaining the next optimal perspective for dynamic grasping planning of the intelligent logistics robot. Weights are normalized and relaxed for training the candidate operation set, constructing a neural network topology search architecture space, and using a differentiable structure to search the neural network topology search architecture space to generate a grasping convolutional neural network. This invention enables precise picking control of target goods identification, detection, labeling, and grasping by intelligent logistics robots, thereby improving the picking performance and accuracy of intelligent logistics robots.

Description

Technical Field

[0001] This invention relates to the field of logistics robot technology, and in particular to a picking method and system for intelligent logistics robots based on visual perception. Background Technology

[0002] With the rapid development of e-commerce and intelligent manufacturing, the logistics industry is increasingly adopting intelligent logistics robots for picking goods. Logistics robots typically consist of navigation systems, sensor systems, execution systems, and intelligent decision-making systems, completing tasks such as handling and sorting goods through automated and intelligent operations. The development of visual perception technology has provided intelligent logistics robots with more accurate recognition and judgment capabilities, especially in the item picking process. Robots, through devices such as cameras and depth sensors, can acquire the location information and morphological characteristics of items, achieving efficient and accurate picking operations.

[0003] However, existing intelligent logistics robots still face many technical bottlenecks in picking, detection, and grasping. First, current visual perception technologies lack robustness in complex environments, especially under conditions of changing lighting, object occlusion, and irregular object shapes, making it difficult for robots to accurately identify, classify, label, and locate goods. Second, existing visual perception algorithms often require high precision in object positioning and classification, while existing visual sensors still have limitations in computational performance and grasping accuracy when processing large-scale data, preventing robots from achieving efficient real-time decision-making and execution in dynamic environments. Finally, existing picking methods often rely on static rules with fixed grasping paths, leading to significant deviations in the grasping perspective and actions of intelligent logistics robots, making it difficult to cope with changing working environments and task requirements. Summary of the Invention

[0004] This invention overcomes the shortcomings of the prior art and provides a picking method and system for intelligent logistics robots based on visual perception.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The first aspect of this invention provides a picking method for intelligent logistics robots based on visual perception, comprising the following steps: The region of interest (ROI) image of the target cargo is obtained. Based on the perceptual characteristics of human vision for salient regions of an image, the convex hull of the ROI image is enhanced to obtain the contour of the grasped target. The region of interest image is smoothed by filtering on different scale filter kernels and pixel non-maximum suppression is performed to obtain automatic annotation information of the target cargo. Based on the automatic annotation information, the target contour is segmented and detected by a self-attention mechanism, and the detection and segmentation results of the cargo are output. The ideal grasping area space of the target goods is obtained by mapping and affine transformation. Information gain explores the perspective of the ideal grasping area space to obtain the next best perspective for dynamic grasping planning of intelligent logistics robots. By assigning weights to the candidate operation set for normalized training relaxation, a neural network topology search architecture space is constructed. The differentiable structure is used to search the neural network topology search architecture space to generate a grasping convolutional neural network.

[0006] Furthermore, in a preferred embodiment of the present invention, the step of acquiring the region of interest image of the target goods, and enhancing the description of the convex hull of the region of interest image based on the perceptual characteristics of human vision for salient regions of an image to obtain the target contour, specifically includes the following steps: Acquire the region of interest (ROI) image of the target cargo, count the number of pixels at each gray level in the ROI image, and generate the source grayscale image of the ROI image; Based on the perceptual characteristics of human vision for salient regions of an image, a human visual salient perception attention mechanism is searched in a big data network. Based on the human visual salient perception attention mechanism, the cumulative distribution of the source grayscale image is calculated to obtain a series of cumulative distribution functions of the source grayscale image. The series of cumulative distribution functions are normalized to construct a gray-level mapping table. The gray-level values ​​of the source pixels in the source gray-level image are replaced by the gray-level mapping table to generate a balanced histogram of the region of interest based on human visual perception. Extract the spatially occupied pixel groups of the region of interest image and construct the pixel coordinate system of the balanced histogram of the region of interest. Starting from the minimum coordinate point of the Y-axis in the pixel coordinate system as the reference center point, sort the spatially occupied pixel groups according to the angle between the minimum coordinate point and the positive direction of the X-axis to obtain the sorted point set. Construct a maintenance empty stack for contour extraction. Iterate through the sorted point set and check whether the two pixels at the top of the maintenance empty stack are in a counterclockwise direction. If they are, add the pixel to the maintenance empty stack. If not, pop the pixel from the top of the stack and continue checking until the last two pixels in the maintenance empty stack are in a counterclockwise direction. By checking pixel groups, pixel convex hull boundaries that maintain empty stack records are generated, and the target contour of the region of interest image is determined based on the pixel convex hull boundaries.

[0007] Furthermore, in a preferred embodiment of the present invention, the step of filtering and smoothing the region of interest image on different scale filter kernels and performing pixel non-maximum suppression to obtain automatic annotation information of the target cargo, performing self-attention mechanism segmentation and detection on the captured target contour based on the automatic annotation information, and outputting the detection and segmentation result of the cargo, specifically includes the following steps: Obtain the pixel equalization grayscale value of the region of interest, and preset filter kernels of different scales based on the pixel equalization grayscale value. Then, with the pixel as the center, extract all pixel values ​​within the filter kernel window of a certain scale. Calculate the median value of all pixels within the filter kernel window, and use the pixel median value to replace the center pixel value of the region of interest image until the center pixel of all scale filter kernel windows has been replaced, thus obtaining a smoothed image of the region of interest. The Sobel operator is introduced to calculate the gradient magnitude and gradient direction of the smoothed image. If the gradient value of a certain pixel in the smoothed image does not change along the local maximum of the gradient direction, the pixel is suppressed to obtain a thinned edge image of the region of interest. The minimum bounding rectangle of the thinned edge image is calculated based on the captured target contour, and the aspect ratio of the minimum bounding rectangle is obtained. Automatic labeling information of the target goods is generated according to the aspect ratio. The first segmentation feature and the second segmentation feature are obtained through the automatic annotation information description, and the category feature map of the goods is obtained. Based on the self-attention mechanism, the first segmentation feature, the second segmentation feature and the category feature map are adaptively allocated to the multi-channel feature weights, and the detection and segmentation results of the goods are output.

[0008] Furthermore, in a preferred embodiment of the present invention, the step of obtaining the first segmentation feature and the second segmentation feature through the automatically labeled information, and obtaining the category feature map of the goods, and adaptively allocating multi-channel feature weights to the first segmentation feature, the second segmentation feature, and the category feature map based on a self-attention mechanism, and outputting the detection and segmentation result of the goods, specifically includes the following steps: Based on the automatic annotation information, the contour of the captured target is described by feature description to obtain the multidimensional tensor contour features of the target goods in the goods stacking area, which is defined as the first segmentation feature. At the same time, the multidimensional tensor contour features of the remaining goods in the goods stacking area are obtained and defined as the second segmentation feature. Obtain the category feature map of the goods, construct the feature pyramid neural network of the goods stacking area through the first segmentation feature and the second segmentation feature, introduce the self-attention mechanism, and perform mapping calculation on each feature channel in the feature pyramid neural network based on the category feature map in the self-attention mechanism to obtain the feature mapping query sequence, feature mapping key and category value of each feature channel; Calculate the Hanning distance between the feature mapping query sequence and the feature mapping key, determine the self-attention score of each feature channel based on the Hanning distance, and transform the attention of the self-attention score using a probability distribution function to determine the self-attention feature weights associated with different categories for each feature channel. Based on the summation of the self-attention feature weights and category values, the feature pyramid neural network and the category feature map are weighted and fused to generate segmentation feature-goods category mapping rules. The first and second segmentation features are retrieved and identified using the segmentation feature-cargo category mapping rule, and the detection and segmentation results of the cargo are output.

[0009] Furthermore, in a preferred embodiment of the present invention, the affine transformation of the mapping yields an ideal grasping area space for the target goods, and the information gain explores the perspective of the ideal grasping area space to obtain the next optimal perspective for dynamic grasping planning of the intelligent logistics robot, specifically including the following steps: A grasping space for the graspable region is constructed. A multi-spatial and multi-channel attention mechanism is introduced to map the image pixels of the graspable region to the grasping space. Affine transformation calculation and grasping quality evaluation are performed to obtain the ideal grasping region space for the target cargo. Binary exploration is used to explore pixel occlusion of image pixels in the ideal grasping region space, construct a visibility occupancy grid in the graspable region, evaluate the current information entropy of the visibility occupancy grid, and identify the occupancy exploration front based on the current information entropy; A preset occlusion pixel sequence is used to generate multiple candidate observation points based on the occlusion exploration front. If the occlusion pixel sequence can be found for a candidate observation point, the candidate observation point is marked as an infeasible observation point. Feasible observation points are then purified to obtain multiple next-view candidate observation points. The total historical distance traveled by the intelligent logistics robot from its current position to each next-view candidate observation point is extracted from the operation log. The total historical distance traveled is marked as the collision cost. The region of interest is observed from each next-view candidate observation point. At this time, the number of occlusion pixel grids that can be reduced is counted, and the back voxel information gain of each next-view candidate observation point performing occlusion grasping is obtained. The collision cost is subtracted from the information gain of the rear voxel, and the comprehensive observation score of each candidate observation point of the next view is output. If the comprehensive observation score is higher than the preset comprehensive observation score, the candidate observation point of the next view is retained and marked as the next best view. Dynamic planning of view and action is performed on the intelligent logistics robot when grasping target goods based on one or more next best views.

[0010] Furthermore, in a preferred embodiment of the present invention, the construction of the grasping space of the graspable region, the introduction of a multi-spatial and multi-channel attention mechanism to map the image pixels of the graspable region to the grasping space, and the execution of affine transformation calculation and grasping quality evaluation to obtain the ideal grasping region space of the target goods, specifically includes the following steps: Obtain the graspable area of ​​the intelligent logistics robot for the target goods and the operating degrees of freedom of the intelligent logistics robot, and construct the grasping space of the graspable area based on the operating degrees of freedom; A multi-spatial and multi-channel attention mechanism is introduced to map the image pixels of the graspable region to the grasping space one by one, so as to construct a pixel-level grasping generation convolutional neural network. The standard grasping vector of the intelligent logistics robot is obtained through big data network. Based on the standard grasping vector, a confidence score is preset. A sliding window is used to predict candidate grasping boxes on the image pixels of the graspable area, and a confidence score is generated for each candidate grasping box. If the confidence score is lower than the preset confidence score, the candidate capture box is standardized and rotated and aligned. After the rotation is completed, the sliding window area corresponding to the candidate capture box is taken as the candidate capture area, resulting in several candidate capture areas. The grabbing quality of candidate grabbing regions is predicted by generating a convolutional neural network regression prediction based on pixel-level grabbing. A grabbing quality evaluation value is generated for each candidate grabbing region. Only the candidate grabbing region corresponding to the highest grabbing quality evaluation value is extracted and marked as the ideal grabbing region space for the target goods.

[0011] Furthermore, in a preferred embodiment of the present invention, the step of assigning weights to the candidate operation set for normalized training relaxation, constructing a neural network topology search architecture space, and using a differentiable structure to search the neural network topology search architecture space to generate a grasping convolutional neural network specifically includes the following steps: Obtain the candidate operation set for intelligent logistics robots to perform the next best view to grasp goods in the ideal grasping area space. Use Softmax normalization to normalize and relax the weights of all candidate operations in the candidate operation set to a continuous probability distribution, and generate the logistics grasping candidate operation search space. Extract several dynamic programming data of the perspective and actions of the intelligent logistics robot to execute the next best perspective in the ideal grasping area space. Based on several dynamic programming data, train and update the network weights and architecture parameters of the logistics grasping candidate operation search space, output the selection probability of each candidate operation, and remove the candidate operation corresponding to the selection probability if the selection probability is lower than the preset selection probability, thus obtaining the neural network topology search architecture space. Obtain the joint probability distribution of the neural network topology search architecture space when executing the next optimal perspective discretized grasping decision in the ideal grasping region space. Based on the joint probability distribution, preset differentiable hyperparameters. Hyperparameterize the neural network topology search architecture space through differentiable hyperparameters to obtain the network floating-point computation number. The task of grasping the target goods is obtained. Based on the grasping task, an ideal grasping space is constructed to realize the objective function of grasping from the next best perspective. The floating-point computation of the network is introduced into the objective function to obtain the minimized floating-point objective function. The neural network topology is optimized and trained by minimizing the gradient of the floating-point objective function to search the architecture space. Finally, a grasping convolutional neural network is generated and deployed on the decision control terminal of the intelligent logistics robot.

[0012] A second aspect of the present invention provides a vision-based intelligent logistics robot picking system, the intelligent logistics robot picking system including a memory and a processor, the memory storing a vision-based intelligent logistics robot picking method program, and when the intelligent logistics robot picking method program is executed by the processor, the steps of the intelligent logistics robot picking method described in any one of the present invention are implemented.

[0013] This invention addresses the technical deficiencies in the prior art, and its beneficial technical effects are as follows: This invention acquires an image of the region of interest (ROI) of the target goods. Based on the perceptual characteristics of human vision for salient regions in an image, it enhances the description of the convex hull of the ROI image to obtain the target contour. The ROI image is then filtered and smoothed using different scale filter kernels, and pixel non-maximum suppression is performed to obtain automatic labeling information for the target goods. Based on this automatic labeling information, a self-attention mechanism is used for segmentation and detection of the target contour, outputting the detection and segmentation results of the goods. An ideal grasping region space for the target goods is derived through affine transformation. Information gain is used to explore the perspective of the ideal grasping region space to obtain the next optimal perspective for dynamic grasping planning of the intelligent logistics robot. Weights of the candidate operation set are normalized and trained to relax the system, constructing a neural network topology search architecture space. A differentiable structure is used to search the neural network topology search architecture space, generating a grasping convolutional neural network. This invention enables precise picking control of target goods identification, detection, labeling, and grasping by intelligent logistics robots, improving the picking performance and accuracy of intelligent logistics robots, and ensuring timely and reliable transportation of goods. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained from these drawings without creative effort.

[0015] Figure 1 A flowchart of the first method for a vision-based intelligent logistics robot picking method is shown. Figure 2A flowchart of a second method for picking by an intelligent logistics robot based on vision perception is shown. Figure 3 A system framework diagram of a vision-based intelligent logistics robot picking system is shown. Detailed Implementation

[0016] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0017] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0018] The first aspect of this invention provides a picking method for intelligent logistics robots based on visual perception, such as... Figure 1 As shown, it includes the following steps: S102: Obtain the region of interest image of the target cargo, enhance the description of the convex hull of the region of interest image based on the perceptual characteristics of human vision for salient regions of the image, and obtain the outline of the grasped target. S104: The region of interest image is smoothed by filtering on different scale filter kernels and non-maximum suppression of pixels is performed to obtain automatic annotation information of the target cargo. Based on the automatic annotation information, the target contour is segmented and detected by a self-attention mechanism, and the detection and segmentation results of the cargo are output. S106: Mapping and affine transformation to obtain the ideal grasping area space of the target goods, information gain to explore the perspective of the ideal grasping area space, so as to obtain the next best perspective for dynamic grasping planning of intelligent logistics robots. S108: Assign weights to the candidate operation set for normalized training relaxation, construct the neural network topology search architecture space, search the neural network topology search architecture space using differentiable structures, and generate a grasping convolutional neural network.

[0019] Furthermore, in a preferred embodiment of the present invention, the acquisition of the region of interest image of the target goods, based on the perceptual characteristics of human vision for salient regions of an image, enhances the description of the convex hull of the region of interest image to obtain the target contour, such as... Figure 2 As shown, the specific steps include: S202: Obtain the region of interest image of the target goods, count the number of pixels at each gray level in the region of interest image, and generate the source grayscale image of the region of interest image; S204: Based on the perceptual characteristics of human vision for salient regions of an image, search for and obtain the human visual salient perception attention mechanism in the big data network, and perform cumulative distribution calculation on the source grayscale image based on the human visual salient perception attention mechanism to obtain a series of cumulative distribution functions of the source grayscale image. S206: Normalize the series of cumulative distribution functions to construct a gray-level mapping table, and replace the gray-level values ​​of the source pixels in the source gray-level image with the gray-level mapping table to generate a balanced histogram of the region of interest based on human visual perception. S208: Extract the spatially occupied pixel groups of the region of interest image and construct the pixel coordinate system of the balanced histogram of the region of interest. Starting from the minimum coordinate point of the Y-axis coordinate in the pixel coordinate system as the reference center point, sort the spatially occupied pixel groups according to the angle between the minimum coordinate point and the positive direction of the X-axis to obtain the sorted point set. S210: Construct a maintenance empty stack for contour extraction. Iterate through the sorted point set and check whether the two pixels at the top of the maintenance empty stack are in a counterclockwise direction. If they are, add the pixel to the maintenance empty stack. If not, pop the pixel from the top of the stack and continue checking until the last two pixels in the maintenance empty stack are in a counterclockwise direction. S212: By checking the pixel group, generate the pixel convex hull boundary that maintains the empty stack record, and determine the target contour of the region of interest image based on the pixel convex hull boundary.

[0020] It should be noted that existing technologies often face problems such as uneven brightness, blurriness, and poor contrast when processing low-brightness images in dimly lit scenes, resulting in incomplete information about the target cargo in the image. This situation limits the accurate identification and extraction of the target region, affecting the efficiency and reliability of the image processing system in practical applications. To address this, this method represents the region of interest in the image through grayscale transformation and spatial occupancy arrays. By constructing a pixel coordinate system in the image grayscale, the boundary pixels of the target cargo image contour are accurately located. By selecting the minimum Y-axis coordinate point as the starting point, the selection of the starting point is ensured to avoid misalignment, allowing subsequent points to be sorted according to the relationship between the polar angle and this reference point. This avoids uncertainty and ambiguity in the distribution of boundary pixels during the sorting process, eliminating the extraction deviation of the target cargo contour. Sort the spatial occupancy pixel group according to the angle between the minimum coordinate point and the positive X-axis direction, ensuring that the convex hull is fitted along the edge processing points. This ensures that the contour information described by the convex hull is in pixel boundary order, without missing any boundary pixels, making the extracted contour information more accurate. If the two pixels at the top of the maintenance empty stack in the sorted point set are not in a counter-clockwise direction, it indicates that the pixel's orientation is incorrect and it does not belong to the contour pixels of the target in the region of interest image. Therefore, it is not added to the maintenance empty stack, reducing the contour description of erroneous pixels and improving the accuracy of the target contour. Finally, the boundary description of the pixel convex hull is the clear contour information of the target to be grasped. This method can solve the problem of intelligent logistics robots struggling to extract accurate and clear contour information from low-brightness images when picking and grasping targets in dim scenes, effectively improving the target recognition accuracy in low-brightness images and providing a reliable information foundation for the subsequent accurate picking and grasping of intelligent logistics robots.

[0021] It should be noted that although existing image enhancement methods can improve the visual effect of low-brightness images to some extent, they often fail to fully consider the perceptual characteristics of the human visual system for salient regions of an image. This makes it difficult to effectively highlight the target area in complex backgrounds or low-contrast situations, resulting in unclear and incomplete target information. To address this, this method calculates the source grayscale image of the region of interest based on the cumulative distribution of the perceptual characteristics of the human visual system for salient regions of an image. Then, it uses a grayscale mapping table constructed from a series of cumulative distribution functions to perform global histogram equalization on the image. This allows for the re-optimization and adjustment of the grayscale distribution within the region, significantly improving the visibility of details and enhancing contrast in the target area. This makes the target more prominent and effectively improves the accuracy and operability of target recognition in low-brightness images.

[0022] Furthermore, in a preferred embodiment of the present invention, the step of filtering and smoothing the region of interest image on different scale filter kernels and performing pixel non-maximum suppression to obtain automatic annotation information of the target cargo, performing self-attention mechanism segmentation and detection on the captured target contour based on the automatic annotation information, and outputting the detection and segmentation result of the cargo, specifically includes the following steps: Obtain the pixel equalization grayscale value of the region of interest, and preset filter kernels of different scales based on the pixel equalization grayscale value. Then, with the pixel as the center, extract all pixel values ​​within the filter kernel window of a certain scale. Calculate the median value of all pixels within the filter kernel window, and use the pixel median value to replace the center pixel value of the region of interest image until the center pixel of all scale filter kernel windows has been replaced, thus obtaining a smoothed image of the region of interest. The Sobel operator is introduced to calculate the gradient magnitude and gradient direction of the smoothed image. If the gradient value of a certain pixel in the smoothed image does not change along the local maximum of the gradient direction, the pixel is suppressed to obtain a thinned edge image of the region of interest. The minimum bounding rectangle of the thinned edge image is calculated based on the captured target contour, and the aspect ratio of the minimum bounding rectangle is obtained. Automatic labeling information of the target goods is generated according to the aspect ratio. The first segmentation feature and the second segmentation feature are obtained through the automatic annotation information description, and the category feature map of the goods is obtained. Based on the self-attention mechanism, the first segmentation feature, the second segmentation feature and the category feature map are adaptively allocated to the multi-channel feature weights, and the detection and segmentation results of the goods are output.

[0023] It should be noted that in logistics scenarios, when manually annotating large-scale image data, existing large-scale unlabeled annotation technologies still suffer from low efficiency and poor accuracy. Manual annotation is not only time-consuming but also easily affected by human factors, leading to inconsistencies in the annotation results. Furthermore, traditional image processing methods often struggle to efficiently and accurately extract accurate object information from large numbers of images, resulting in annotation quality that cannot meet the demands of automated large-scale data processing. To address this, this method first pre-defines filter kernels of different scales, including small-scale and large-scale kernels. Small-scale kernels are suitable for removing small noise but have limited effectiveness against larger noise or blurred edges. Large-scale kernels can smooth larger noise areas but may cause loss of detail. Therefore, a fusion filter of both is needed to compensate for each other's noise smoothing deficiencies. Therefore, the center pixel value of the region of interest image is replaced by extracting all pixel values ​​within the large-scale and small-scale filter kernel windows, thereby achieving median filtering of image noise through the fusion of different scale filter kernels. Median filtering can remove salt-and-pepper noise and random noise, protect image edge information, effectively suppress strong noise, and avoid excessive blurring. Finally, a display image with higher smoothness of the region of interest can be output, providing a clear image recognition foundation for accurate labeling and cargo classification and segmentation. Next, after generating the smoothed image of the region of interest, its gradient magnitude and gradient direction are further calculated. If the gradient value of a certain pixel in the smoothed image does not change along the local maximum of the gradient direction, it indicates that the edge line of that pixel is blurred. Therefore, the image edge display is refined by suppressing that pixel, thereby obtaining a clear outline of the target object through edge detection, i.e., refining the edge image. Then, the minimum bounding rectangle of each cargo is calculated using the refined edge image, so that the detection result can match the labeling format. The aspect ratio of the minimum bounding rectangle is the automatic labeling information of the target cargo. This method enables non-maximum suppression along the gradient direction, which refines the edges of the target object, thereby accurately locating and extracting the outline of the target object. It also accurately defines the position of the target object based on the minimum bounding rectangle, achieving high-quality automatic annotation of large-scale unlabeled cargo image data.

[0024] It should be noted that current cargo detection and segmentation technologies often suffer from decreased detection accuracy when faced with haphazardly stacked and occluded cargo scenarios. Traditional vision-guided neural network models, lacking effective extraction and aggregation of important features for cargo categories, negatively impact segmentation accuracy. Furthermore, the mapping process between image features and cargo categories struggles to establish effective cross-level feature associations, resulting in insufficient cargo recognition capabilities in stacked environments. Therefore, this method constructs a nonlinear association model based on a feature pyramid neural network, achieving accurate mapping between cross-level image features and cargo categories. This improves the accuracy of cargo detection, classification, and segmentation in complex stacked environments, ensuring the picking accuracy, efficiency, and precision of intelligent logistics robots in grasping different goods in such environments.

[0025] Furthermore, in a preferred embodiment of the present invention, the step of obtaining the first segmentation feature and the second segmentation feature through the automatically labeled information, and obtaining the category feature map of the goods, and adaptively allocating multi-channel feature weights to the first segmentation feature, the second segmentation feature, and the category feature map based on a self-attention mechanism, and outputting the detection and segmentation result of the goods, specifically includes the following steps: Based on the automatic annotation information, the contour of the captured target is described to obtain the multidimensional tensor contour features of the target goods in the goods stacking area, which is defined as the first segmentation feature. At the same time, the multidimensional tensor contour features of the remaining goods in the goods stacking area are obtained and defined as the first segmentation feature. Obtain the category feature map of the goods, construct the feature pyramid neural network of the goods stacking area through the first segmentation feature and the second segmentation feature, introduce the self-attention mechanism, and perform mapping calculation on each feature channel in the feature pyramid neural network based on the category feature map in the self-attention mechanism to obtain the feature mapping query sequence, feature mapping key and category value of each feature channel; Calculate the Hanning distance between the feature mapping query sequence and the feature mapping key, determine the self-attention score of each feature channel based on the Hanning distance, and transform the attention of the self-attention score using a probability distribution function to determine the self-attention feature weights associated with different categories for each feature channel. Based on the summation of the self-attention feature weights and category values, the feature pyramid neural network and the category feature map are weighted and fused to generate segmentation feature-goods category mapping rules. The first and second segmentation features are retrieved and identified using the segmentation feature-cargo category mapping rule, and the detection and segmentation results of the cargo are output.

[0026] It should be noted that, regarding the specific steps for improving the accuracy of cargo detection, classification, and segmentation in complex stacking environments for intelligent logistics robots, this method first obtains the features of the target cargo and the features of other stacked cargo in the complex stacking environment as the mapping input for the intelligent logistics robot's picking and segmentation, namely, the first segmentation feature and the second segmentation feature. Next, a feature pyramid neural network is constructed for the cargo stacking area using the first and second segmentation features. This feature pyramid neural network can accurately extract cargo features at different stacking scales, and can gradually propagate from deep, high-semantic segmentation features (high-level feature maps) to shallower layers, further combining shallow spatial detail information and deep semantic information to improve detection performance. Compared to the poor performance of traditional single-scale features in detecting small targets, it effectively utilizes feature channel information at different scales to improve detection performance. Then, a self-attention mechanism is introduced to adaptively allocate multi-channel feature weights to each feature channel in the feature pyramid neural network and the cargo category feature map, obtaining the feature mapping query sequence, feature mapping key, and category value for each feature channel. The Hanning distance between the feature mapping query sequence and the feature mapping key clarifies the correlation between channels, measuring the importance of a certain feature channel to other channels, thereby enabling the extraction and aggregation of important correlation features of cargo categories. By transforming the attention scores using a probability distribution function, ensuring that the weights of all channels are between 0 and 1, the generated self-attention feature weights assign different importance to different channels, improving the accuracy of cargo detection and differentiation. Finally, the summation of the self-attention feature weights and category values ​​captures the global information of cargo segmentation features and cargo categories, enabling a one-to-one mapping between cargo segmentation features and cargo categories. This achieves a non-linear association between cross-level image features and cargo categories, improving the accuracy of intelligent logistics robots in picking, detection, classification, and segmentation in complex stacking environments.

[0027] Furthermore, in a preferred embodiment of the present invention, the affine transformation of the mapping yields an ideal grasping area space for the target goods, and the information gain explores the perspective of the ideal grasping area space to obtain the next optimal perspective for dynamic grasping planning of the intelligent logistics robot, specifically including the following steps: A grasping space for the graspable region is constructed. A multi-spatial and multi-channel attention mechanism is introduced to map the image pixels of the graspable region to the grasping space. Affine transformation calculation and grasping quality evaluation are performed to obtain the ideal grasping region space for the target cargo. Binary exploration is used to explore pixel occlusion of image pixels in the ideal grasping region space, construct a visibility occupancy grid in the graspable region, evaluate the current information entropy of the visibility occupancy grid, and identify the occupancy exploration front based on the current information entropy; A preset occlusion pixel sequence is used to generate multiple candidate observation points based on the occlusion exploration front. If the occlusion pixel sequence can be found for a candidate observation point, the candidate observation point is marked as an infeasible observation point. Feasible observation points are then purified to obtain multiple next-view candidate observation points. The total historical distance traveled by the intelligent logistics robot from its current position to each next-view candidate observation point is extracted from the operation log. The total historical distance traveled is marked as the collision cost. The region of interest is observed from each next-view candidate observation point. At this time, the number of occlusion pixel grids that can be reduced is counted, and the back voxel information gain of each next-view candidate observation point performing occlusion grasping is obtained. The collision cost is subtracted from the information gain of the rear voxel, and the comprehensive observation score of each candidate observation point of the next view is output. If the comprehensive observation score is higher than the preset comprehensive observation score, the candidate observation point of the next view is retained and marked as the next best view. Dynamic planning of view and action is performed on the intelligent logistics robot when grasping target goods based on one or more next best views.

[0028] It should be noted that in complex environments, target goods often have irregular shapes and are stacked, which can lead to a low success rate in grasping them and make it difficult to guarantee stable and reliable grasping results. To address this, this method constructs a grasping space for the graspable region, utilizes a multi-spatial and multi-channel attention mechanism to generate a pixel-level grasping convolutional neural network, achieving a one-to-one mapping between image pixels and the grasping space. Then, by performing standardized affine transformations on candidate graspable regions and evaluating and selecting reasonable grasping regions based on quality maximization constraints, an ideal grasping region space is obtained. This achieves stable and reliable grasping detection results for the intelligent logistics robot, making its grasping of target goods more reliable and reasonable, and improving the grasping success rate of target goods with irregular shapes and stacking in complex environments.

[0029] It should be noted that, in addition, visual information acquired from a single viewpoint in occluded and stacked environments has limitations, resulting in low target visibility in some grasping areas, thus affecting the accuracy of the grasping task. Existing logistics picking methods often rely on single-viewpoint information for target detection and grasping decisions, which may lead to information loss due to occlusion, thereby increasing the risk of misgrabbing or missed grasping. Secondly, the lack of effective utilization of historical prediction information may lead to redundant viewpoint adjustments and ineffective grasping attempts, reducing the overall grasping efficiency of intelligent logistics robots. To address this, this method uses image pixels to explore the ideal grasping area space through pixel occlusion to generate a visibility occupancy grid that represents the blurred visibility of the graspable area due to occlusion stacking. Then, the current occlusion stacking state of the visibility occupancy grid, i.e., the current information entropy, is evaluated to determine which areas have known occlusion stacking and which areas have inconspicuous occlusion stacking. Based on the current information entropy, candidate observation points for the occlusion exploration frontier are obtained, helping the dynamic planning of intelligent logistics robots find the most beneficial grasping viewpoint to eliminate occlusion stacking. Next, the feasibility of grasping multiple candidate observation points with occlusion stacking on the occlusion exploration front is assessed. If an occlusion pixel sequence can be found for a candidate observation point, grasping at that point is prone to limitations in single-view information, and these points need to be excluded. Therefore, these candidate observation points are marked as infeasible observation points. The final retained candidate observation points are a series of candidate observation points for the next optimal viewpoint. These next-view candidate observation points are then further evaluated. This method uses the back-voxel information gain minus the collision cost of the historical total path length to the next-view candidate observation point. The back-voxel information gain is a measure of how much the viewpoint can reduce the area of ​​insignificant occlusion stacking, ensuring that the intelligent logistics robot selects the optimal next viewpoint, not just the nearest point. By combining the back-voxel information gain expression of the region of interest with constraints based on historical predictions, the next optimal viewpoint is accurately calculated to dynamically optimize the viewpoint and grasping action of the intelligent logistics robot. This effectively reduces false positive grasping and ensures optimal grasping detection under the best viewpoint, thereby improving the grasping accuracy and efficiency in complex scenarios.

[0030] Furthermore, in a preferred embodiment of the present invention, the construction of the grasping space of the graspable region, the introduction of a multi-spatial and multi-channel attention mechanism to map the image pixels of the graspable region to the grasping space, and the execution of affine transformation calculation and grasping quality evaluation to obtain the ideal grasping region space of the target goods, specifically includes the following steps: Obtain the graspable area of ​​the intelligent logistics robot for the target goods and the operating degrees of freedom of the intelligent logistics robot, and construct the grasping space of the graspable area based on the operating degrees of freedom; A multi-spatial and multi-channel attention mechanism is introduced to map the image pixels of the graspable region to the grasping space one by one, so as to construct a pixel-level grasping generation convolutional neural network. The standard grasping vector of the intelligent logistics robot is obtained through big data network. Based on the standard grasping vector, a confidence score is preset. A sliding window is used to predict candidate grasping boxes on the image pixels of the graspable area, and a confidence score is generated for each candidate grasping box. If the confidence score is lower than the preset confidence score, the candidate capture box is standardized and rotated and aligned. After the rotation is completed, the sliding window area corresponding to the candidate capture box is taken as the candidate capture area, resulting in several candidate capture areas. The grabbing quality of candidate grabbing regions is predicted by generating a convolutional neural network regression prediction based on pixel-level grabbing. A grabbing quality evaluation value is generated for each candidate grabbing region. Only the candidate grabbing region corresponding to the highest grabbing quality evaluation value is extracted and marked as the ideal grabbing region space for the target goods.

[0031] It should be noted that this method introduces a multi-spatial and multi-channel attention mechanism to map the image pixels of the graspable region to the grasping space, enabling the intelligent logistics robot to make grasping space selection decisions based on detailed image pixel analysis, thus making grasping detection within the space more corresponding and accurate. The standard grasping vector represents the grasping direction and pose amplitude of the intelligent logistics robot. By scanning different regions in the image through a sliding window, the probability of these regions being grasping positions is calculated, thereby obtaining a series of candidate grasping boxes as the result of grasping detection, increasing the chance of finding the correct grasping region. Each candidate box contains position coordinates (center point, width, height) and grasping angle (grasping posture), and then the probability of successful grasping of the region is evaluated, i.e., the confidence score. If the confidence score is lower than the preset confidence score, it indicates that the position and grasping angle of the candidate grasping box deviate from the robot's grasping feasibility standard. Therefore, a standardized rotational alignment affine transformation needs to be performed to address the issues of different scales, orientations, and shapes of the grasping regions. Unifying the size and orientation of the grasping regions helps improve grasping accuracy. Finally, regression prediction evaluates the grasping quality of each candidate grasping region, and the candidate grasping region corresponding to the highest grasping quality evaluation value is the ideal grasping region space for the intelligent logistics robot to grasp target goods. This method constructs a pixel-level grasping generation convolutional neural network based on a multi-space, multi-channel attention mechanism to achieve a one-to-one mapping between image pixels and the grasping space, thereby enhancing the ability to grasp and recognize target objects in complex scenes. At the same time, the standardized affine transformation processing of candidate regions and the grasping space selection with quality maximization constraints effectively improve the stability and success rate of grasping.

[0032] Furthermore, in a preferred embodiment of the present invention, the step of assigning weights to the candidate operation set for normalized training relaxation, constructing a neural network topology search architecture space, and using a differentiable structure to search the neural network topology search architecture space to generate a grasping convolutional neural network specifically includes the following steps: Obtain the candidate operation set for intelligent logistics robots to perform the next best view to grasp goods in the ideal grasping area space. Use Softmax normalization to normalize and relax the weights of all candidate operations in the candidate operation set to a continuous probability distribution, and generate the logistics grasping candidate operation search space. Extract several dynamic programming data of the perspective and actions of the intelligent logistics robot to execute the next best perspective in the ideal grasping area space. Based on several dynamic programming data, train and update the network weights and architecture parameters of the logistics grasping candidate operation search space, output the selection probability of each candidate operation, and remove the candidate operation corresponding to the selection probability if the selection probability is lower than the preset selection probability, thus obtaining the neural network topology search architecture space. Obtain the joint probability distribution of the neural network topology search architecture space when executing the next optimal perspective discretized grasping decision in the ideal grasping region space. Based on the joint probability distribution, preset differentiable hyperparameters. Hyperparameterize the neural network topology search architecture space through differentiable hyperparameters to obtain the network floating-point computation number. The task of grasping the target goods is obtained. Based on the grasping task, an ideal grasping space is constructed to realize the objective function of grasping from the next best perspective. The floating-point computation of the network is introduced into the objective function to obtain the minimized floating-point objective function. The neural network topology is optimized and trained by minimizing the gradient of the floating-point objective function to search the architecture space. Finally, a grasping convolutional neural network is generated and deployed on the decision control terminal of the intelligent logistics robot.

[0033] It should be noted that existing robot grasping neural network architecture designs often rely heavily on human experience and repeated trials for optimization, a method that is both time-consuming and potentially inefficient. Specifically, the design of neural network architectures for robot grasping tasks often requires manual debugging and optimization based on specific environments. This not only consumes significant human resources but is also easily limited by the designer's experience, failing to fully explore optimal structures. Therefore, this method uses Softmax normalization to normalize and relax the weights of all candidate operations in the candidate operation set of the intelligent logistics robot into a continuous probability distribution. This transforms the originally discrete search space into a differentiable continuous space. Through probability weighting, all candidate operations can be partially activated in the convolutional neural network architecture, rather than activating a single operation, thus improving the search efficiency of the convolutional neural network. Then, using dynamic programming data of the perspective and actions of the intelligent logistics robot in the ideal grasping area, the candidate operation search space for logistics grasping is trained. This allows the discrete search space with a differentiable structure to improve the grasping decision performance of the intelligent logistics robot. Updating network weights ensures that the network can learn normally during the search process, and updating architecture parameters allows the neural network to dynamically adjust its structure, gradually leaning towards better operations. Finally, training and updating the search spaces of both will generate selection probabilities for different candidate operations. If the selection probability is lower than the preset selection probability, it means that the discrete candidate operation under that selection probability cannot meet the reasonable and accurate grasping of the intelligent logistics robot. Therefore, the candidate operation corresponding to that selection probability is removed, and the discrete operation is represented by a continuous variable, realizing the continuous relaxation of the discrete topology search space. This makes the constructed neural network topology search architecture space more efficient in exploring the possibility of network structure, breaking through the limitations of traditional methods, and making the neural network design for robot grasping tasks more efficient and intelligent.

[0034] It should be noted that existing heuristic framework search optimization processes often suffer from computational redundancy and are prone to getting trapped in local optima, preventing the network structure from fully realizing its potential. This approach faces challenges in balancing accuracy and efficiency, especially in computationally intensive convolutional neural network designs. To address this, our method hyperparameterizes the neural network topology search architecture space using pre-defined differentiable hyperparameters of the joint probability distribution of the discretized grasping decision made by the intelligent logistics robot from the next optimal perspective. This yields the network's floating-point computation count, which is then incorporated into the objective function to minimize the floating-point computation count. This reduces the computational latency of the neural network topology search architecture space, resulting in a more accurate and efficient grasping process for the intelligent logistics robot using a differentiable grasping convolutional neural network. Ultimately, this improves the balance between computational accuracy and efficiency when the intelligent logistics robot is picking goods.

[0035] A second aspect of the present invention provides a vision-based intelligent logistics robot picking system, such as... Figure 3 As shown, the intelligent logistics robot picking system includes a memory 31 and a processor 32. The memory 31 stores a visual perception-based intelligent logistics robot picking method program. When the intelligent logistics robot picking method program is executed by the processor 32, it implements any of the steps of the intelligent logistics robot picking method described above.

[0036] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A picking method for intelligent logistics robots based on visual perception, characterized in that, Includes the following steps: The region of interest (ROI) image of the target cargo is obtained. Based on the perceptual characteristics of human vision for salient regions of an image, the convex hull of the ROI image is enhanced to obtain the contour of the grasped target. The region of interest image is smoothed by filtering on different scale filter kernels and pixel non-maximum suppression is performed to obtain automatic annotation information of the target cargo. Based on the automatic annotation information, the target contour is segmented and detected by a self-attention mechanism, and the detection and segmentation results of the cargo are output. The ideal grasping area space of the target goods is obtained by mapping and affine transformation. Information gain explores the perspective of the ideal grasping area space to obtain the next best perspective for dynamic grasping planning of intelligent logistics robots. By assigning weights to the candidate operation set for normalized training relaxation, a neural network topology search architecture space is constructed. The differentiable structure is used to search the neural network topology search architecture space to generate a grasping convolutional neural network.

2. The picking method for an intelligent logistics robot based on vision perception according to claim 1, characterized in that, The process of acquiring the region of interest (ROI) image of the target cargo, enhancing the convex hull description of the ROI image based on the perceptual characteristics of human vision for salient regions of an image, and obtaining the target contour includes the following steps: Acquire the region of interest (ROI) image of the target cargo, count the number of pixels at each gray level in the ROI image, and generate the source grayscale image of the ROI image; Based on the perceptual characteristics of human vision for salient regions of an image, a human visual salient perception attention mechanism is searched in a big data network. Based on the human visual salient perception attention mechanism, the cumulative distribution of the source grayscale image is calculated to obtain a series of cumulative distribution functions of the source grayscale image. The series of cumulative distribution functions are normalized to construct a gray-level mapping table. The gray-level values ​​of the source pixels in the source gray-level image are replaced by the gray-level mapping table to generate a balanced histogram of the region of interest based on human visual perception. Extract the spatially occupied pixel groups of the region of interest image and construct the pixel coordinate system of the balanced histogram of the region of interest. Starting from the minimum coordinate point of the Y-axis in the pixel coordinate system as the reference center point, sort the spatially occupied pixel groups according to the angle between the minimum coordinate point and the positive direction of the X-axis to obtain the sorted point set. Construct a maintenance empty stack for contour extraction. Iterate through the sorted point set and check whether the two pixels at the top of the maintenance empty stack are in a counterclockwise direction. If they are, add the pixel to the maintenance empty stack. If not, pop the pixel from the top of the stack and continue checking until the last two pixels in the maintenance empty stack are in a counterclockwise direction. By checking pixel groups, pixel convex hull boundaries that maintain empty stack records are generated, and the target contour of the region of interest image is determined based on the pixel convex hull boundaries.

3. The picking method for an intelligent logistics robot based on vision perception according to claim 1, characterized in that, The process involves filtering and smoothing the region of interest image using different scale filter kernels and performing pixel non-maximum suppression to obtain automatic annotation information for the target cargo. Based on this automatic annotation information, a self-attention mechanism is used to segment and detect the captured target contour, and the detection and segmentation results of the cargo are output. Specifically, this includes the following steps: Obtain the pixel equalization grayscale value of the region of interest, and preset filter kernels of different scales based on the pixel equalization grayscale value. Then, with the pixel as the center, extract all pixel values ​​within the filter kernel window of a certain scale. Calculate the median value of all pixels within the filter kernel window, and use the pixel median value to replace the center pixel value of the region of interest image until the center pixel of all scale filter kernel windows has been replaced, thus obtaining a smoothed image of the region of interest. The Sobel operator is introduced to calculate the gradient magnitude and gradient direction of the smoothed image. If the gradient value of a certain pixel in the smoothed image does not change along the local maximum of the gradient direction, the pixel is suppressed to obtain a thinned edge image of the region of interest. The minimum bounding rectangle of the thinned edge image is calculated based on the captured target contour, and the aspect ratio of the minimum bounding rectangle is obtained. Automatic labeling information of the target goods is generated according to the aspect ratio. The first segmentation feature and the second segmentation feature are obtained through the automatic annotation information description, and the category feature map of the goods is obtained. Based on the self-attention mechanism, the first segmentation feature, the second segmentation feature and the category feature map are adaptively allocated to the multi-channel feature weights, and the detection and segmentation results of the goods are output.

4. The picking method for an intelligent logistics robot based on vision perception according to claim 3, characterized in that, The process of obtaining the first and second segmentation features through the automatically labeled information, and obtaining the category feature map of the goods, adaptively allocating multi-channel feature weights to the first segmentation features, the second segmentation features, and the category feature map based on a self-attention mechanism, and outputting the detection and segmentation results of the goods, specifically includes the following steps: Based on the automatic annotation information, the contour of the captured target is described by feature description to obtain the multidimensional tensor contour features of the target goods in the goods stacking area, which is defined as the first segmentation feature. At the same time, the multidimensional tensor contour features of the remaining goods in the goods stacking area are obtained and defined as the second segmentation feature. Obtain the category feature map of the goods, construct the feature pyramid neural network of the goods stacking area through the first segmentation feature and the second segmentation feature, introduce the self-attention mechanism, and perform mapping calculation on each feature channel in the feature pyramid neural network based on the category feature map in the self-attention mechanism to obtain the feature mapping query sequence, feature mapping key and category value of each feature channel; Calculate the Hanning distance between the feature mapping query sequence and the feature mapping key, determine the self-attention score of each feature channel based on the Hanning distance, and transform the attention of the self-attention score using a probability distribution function to determine the self-attention feature weights associated with different categories for each feature channel. Based on the summation of the self-attention feature weights and category values, the feature pyramid neural network and the category feature map are weighted and fused to generate segmentation feature-goods category mapping rules. The first and second segmentation features are retrieved and identified using the segmentation feature-cargo category mapping rule, and the detection and segmentation results of the cargo are output.

5. The picking method for an intelligent logistics robot based on vision perception according to claim 1, characterized in that, The affine transformation mapping yields the ideal grasping area space for the target goods. Information gain explores the perspective of this ideal grasping area space to obtain the next optimal perspective for dynamic grasping planning of the intelligent logistics robot. Includes the following steps: A grasping space for the graspable region is constructed. A multi-spatial and multi-channel attention mechanism is introduced to map the image pixels of the graspable region to the grasping space. Affine transformation calculation and grasping quality evaluation are performed to obtain the ideal grasping region space for the target cargo. Binary exploration is used to explore pixel occlusion of image pixels in the ideal grasping region space, construct a visibility occupancy grid in the graspable region, evaluate the current information entropy of the visibility occupancy grid, and identify the occupancy exploration front based on the current information entropy; A preset occlusion pixel sequence is used to generate multiple candidate observation points based on the occlusion exploration front. If the occlusion pixel sequence can be found for a candidate observation point, the candidate observation point is marked as an infeasible observation point. Feasible observation points are then purified to obtain multiple next-view candidate observation points. The total historical distance traveled by the intelligent logistics robot from its current position to each next-view candidate observation point is extracted from the operation log. The total historical distance traveled is marked as the collision cost. The region of interest is observed from each next-view candidate observation point. At this time, the number of occlusion pixel grids that can be reduced is counted, and the back voxel information gain of each next-view candidate observation point performing occlusion grasping is obtained. The collision cost is subtracted from the information gain of the rear voxel, and the comprehensive observation score of each candidate observation point of the next view is output. If the comprehensive observation score is higher than the preset comprehensive observation score, the candidate observation point of the next view is retained and marked as the next best view. Dynamic planning of view and action is performed on the intelligent logistics robot when grasping target goods based on one or more next best views.

6. The picking method for an intelligent logistics robot based on vision perception according to claim 5, characterized in that, The process of constructing the graspable region's grasping space involves introducing a multi-spatial and multi-channel attention mechanism to map the image pixels of the graspable region to the grasping space, and performing affine transformation calculations and grasping quality evaluation to obtain the ideal grasping region space for the target cargo. Specifically, this includes the following steps: Obtain the graspable area of ​​the intelligent logistics robot for the target goods and the operating degrees of freedom of the intelligent logistics robot, and construct the grasping space of the graspable area based on the operating degrees of freedom; A multi-spatial and multi-channel attention mechanism is introduced to map the image pixels of the graspable region to the grasping space one by one, so as to construct a pixel-level grasping generation convolutional neural network. The standard grasping vector of the intelligent logistics robot is obtained through big data network. Based on the standard grasping vector, a confidence score is preset. A sliding window is used to predict candidate grasping boxes on the image pixels of the graspable area, and a confidence score is generated for each candidate grasping box. If the confidence score is lower than the preset confidence score, the candidate capture box is standardized and rotated and aligned. After the rotation is completed, the sliding window area corresponding to the candidate capture box is taken as the candidate capture area, resulting in several candidate capture areas. The grabbing quality of candidate grabbing regions is predicted by generating a convolutional neural network regression prediction based on pixel-level grabbing. A grabbing quality evaluation value is generated for each candidate grabbing region. Only the candidate grabbing region corresponding to the highest grabbing quality evaluation value is extracted and marked as the ideal grabbing region space for the target goods.

7. The picking method for an intelligent logistics robot based on vision perception according to claim 1, characterized in that, The process of normalizing and relaxing the weights assigned to the candidate operation set, constructing a neural network topology search architecture space, and using a differentiable structure to search the neural network topology search architecture space to generate a grasping convolutional neural network specifically includes the following steps: Obtain the candidate operation set for intelligent logistics robots to perform the next best view to grasp goods in the ideal grasping area space. Use Softmax normalization to normalize and relax the weights of all candidate operations in the candidate operation set to a continuous probability distribution, and generate the logistics grasping candidate operation search space. Extract several dynamic programming data of the perspective and actions of the intelligent logistics robot to execute the next best perspective in the ideal grasping area space. Based on several dynamic programming data, train and update the network weights and architecture parameters of the logistics grasping candidate operation search space, output the selection probability of each candidate operation, and remove the candidate operation corresponding to the selection probability if the selection probability is lower than the preset selection probability, thus obtaining the neural network topology search architecture space. Obtain the joint probability distribution of the neural network topology search architecture space when executing the next optimal perspective discretized grasping decision in the ideal grasping region space. Based on the joint probability distribution, preset differentiable hyperparameters. Hyperparameterize the neural network topology search architecture space through differentiable hyperparameters to obtain the network floating-point computation number. The task of grasping the target goods is obtained. Based on the grasping task, an ideal grasping space is constructed to realize the objective function of grasping from the next best perspective. The floating-point computation of the network is introduced into the objective function to obtain the minimized floating-point objective function. The neural network topology is optimized and trained by minimizing the gradient of the floating-point objective function to search the architecture space. Finally, a grasping convolutional neural network is generated and deployed on the decision control terminal of the intelligent logistics robot.

8. A vision-based intelligent logistics robot picking system, characterized in that, The intelligent logistics robot picking system includes a memory and a processor. The memory stores a visual perception-based intelligent logistics robot picking method program. When the intelligent logistics robot picking method program is executed by the processor, the steps of the intelligent logistics robot picking method as described in any one of claims 1-7 are implemented.

Images