Operation Management Method and System for Logistics Picking Robots Based on Feedback Analysis

Abstract

The present invention relates to the field of logistics robots, and discloses a method and system for operating and managing a logistics picking robot based on feedback analysis, including the following steps: constructing a three-dimensional point cloud map of the goods stacking area through image analysis and three-dimensional modeling methods, and combining a multi-channel attention mechanism to perform graphic enhancement on the stacked goods in the three-dimensional point cloud map to determine the coordinate positions of the stacked goods, thereby generating the grasping points of the goods. Using the logistics picking robot to conduct a goods grasping stability test in combination with the grasping points of the goods, and optimizing the goods grasping stability of the logistics picking machine based on the test results. The present invention can optimize the goods grasping stability based on pixel-level grasping of a multi-space and multi-channel attention mechanism, realize stable and reliable goods grasping processing, and avoid the problem of low visibility of target in some grasping areas caused by the limitation of single-view information in an occluded stacking environment.

Description

Technical Field

[0001] The present invention relates to the field of logistics robots, and particularly to an operation management method and system for a logistics picking robot based on feedback analysis. Background Art

[0002] When goods are stacked and a logistics picking robot is used to grab the goods, during the grabbing process, there may be problems such as uneven brightness, blurriness, and poor contrast in the images recognized by the robot, resulting in incomplete target information of the goods. All of the above situations may lead to problems such as poor grabbing effect and damaged goods during the goods grabbing process.

[0003] Aiming at the problem of low grabbing success rate caused by irregular shapes of target goods and mutual stacking of objects in a cluttered environment, a pixel-level grabbing generation convolutional neural network based on multi-space and multi-channel attention mechanisms is constructed to realize the one-to-one mapping between image pixels and the grabbing space; the candidate regions are standardized through affine transformation, a grabbing selection network considering the grabbing depth is constructed, and the grabbing set is evaluated and screened based on the quality maximization constraint to realize stable and reliable grabbing detection. Therefore, an operation management method and system for a logistics picking robot based on feedback analysis are proposed to achieve the accuracy and stability of the logistics picking robot in grabbing goods. Summary of the Invention

[0004] The present invention overcomes the deficiencies of the prior art and provides an operation management method and system for a logistics picking robot based on feedback analysis.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] The first aspect of the present invention provides an operation management method for a logistics picking robot based on feedback analysis, including the following steps:

[0007] Obtain the goods image, depth information, and goods point cloud data in the goods stacking area, generate a point cloud map of the goods stacking area, and perform filtering processing on the point cloud map of the goods stacking area to achieve spatio-temporal alignment of the map parameters, and obtain the target goods stacking map;

[0008] Introduce a multi-channel attention module into the target goods stacking map to enhance the graphics of the occluded areas in the target goods stacking map and identify the coordinate positions of the stacked goods;

[0009] Generate the target grabbing points of the goods according to the coordinate positions of the goods, and evaluate the grabbing stability of the logistics picking robot at the target grabbing points of the goods;

[0010] Optimize the grabbing of the logistics picking robot with unqualified grabbing stability, and control the logistics picking robot after grabbing optimization to grab the goods.

[0011] Further, in a preferred embodiment of the present invention, obtaining a cargo image, depth information, and cargo point cloud data in the cargo stacking area, generating a point cloud map of the cargo stacking area, and performing filtering processing on the point cloud map of the cargo stacking area to achieve spatio-temporal alignment of map parameters, and obtaining a target cargo stacking map, specifically:

[0012] Determine the cargo stacking area, where the cargo stacking area is the area where the cargo that needs to be operated and managed by the logistics picking robot is stacked;

[0013] Obtain an RGB-D camera, and based on the RGB-D camera, obtain a scene color image in the cargo stacking area to obtain a cargo image of the cargo stacking area, which is calibrated as the target cargo image. At the same time, based on the RGB-D camera, obtain the cargo depth information in the cargo stacking area to obtain the cargo depth information of the cargo stacking area;

[0014] Fuse the cargo depth information of the cargo stacking area into the target cargo image, so that each pixel point of the cargo contains the corresponding cargo depth information. Introduce 3D modeling software, and import the target cargo image fused with the cargo depth information into the 3D modeling software to obtain the cargo point cloud data of the target cargo image fused with the cargo depth information;

[0015] Based on the cargo point cloud data of the target cargo image fused with the cargo depth information in the 3D modeling software, construct a point cloud map of the cargo stacking area;

[0016] In the 3D modeling software, introduce the Kalman filtering algorithm, and based on the Kalman filtering algorithm, perform parameter alignment on the cargo point cloud data on the point cloud map of the cargo stacking area;

[0017] Among them, the method of performing parameter alignment on the cargo point cloud data on the point cloud map of the cargo stacking area is to define the three-dimensional coordinate information of the cargo point cloud data, thereby constructing the speed parameter and spatial parameter when obtaining the target cargo image, and based on the speed parameter and spatial parameter when obtaining the target cargo image, establish a state transition model;

[0018] Obtain the generation time of the target cargo image, and combine it with the state transition model to achieve spatio-temporal synchronization of the point cloud map of the cargo stacking area, and obtain a spatio-temporally aligned point cloud map of the cargo stacking area, which is calibrated as the target cargo stacking map.

[0019] Further, in a preferred embodiment of the present invention, introducing a multi-channel attention module into the target cargo stacking map, performing graphic enhancement on the occluded area in the target cargo stacking map, and identifying the coordinate positions of the stacked cargo, specifically:

[0020] Introduce a CNN neural network to extract RGB branch features from the target cargo stacking map, obtaining the texture features and color features of the target cargo stacking map. At the same time, perform depth branch extraction on the target cargo stacking map to obtain the geometric structure features of the target cargo stacking map;

[0021] Introduce a bilinear attention module. In the bilinear attention module, import the texture features, color features, and geometric structure features of the target cargo stacking map, and calculate the weight ratios of different features in the target cargo stacking map through the bilinear attention module. At the same time, use the bilinear attention module to generate a spatial mask in the target cargo stacking map to enhance the feature response speed of the cargo occlusion boundary in the target cargo stacking map;

[0022] In the target cargo stacking map, combine the spatial mask and the weight ratios of different features, and perform dilated convolution on the target cargo stacking map through the CNN neural network to obtain an enhanced target cargo stacking map;

[0023] In the enhanced target cargo stacking map, locate the position of the spatial mask, output the visible cargo area and the occluded cargo area based on the spatial mask position, and determine the coordinate positions of different cargos in the visible cargo area within the 3D modeling software;

[0024] For the occluded cargo area, introduce a geometric completion network to perform depth speculation on the occluded cargo area, construct the cargo contours in the occluded cargo area, and combine with the 3D modeling software to determine the coordinate positions of different cargos in the occluded cargo area.

[0025] Further, in a preferred embodiment of the present invention, generating a cargo target grasping point according to the coordinate positions of the cargos and evaluating the grasping stability of the logistics picking robot at the cargo target grasping point specifically includes:

[0026] Define the coordinate positions of different cargos in the visible cargo area and the occluded cargo area as cargo target coordinate positions, and introduce a big data network. Based on the big data network, retrieve the graspable positions of all types of cargos in the cargo stacking area when using the logistics picking robot, and mark them as cargo target grasping points;

[0027] Obtain the rated grasping force numerical range of the logistics picking robot, and based on the big data network, analyze all the cargo target grasping points of the cargos to determine the minimum force closure numerical index of different cargos at the cargo target grasping points;

[0028] Judge whether there is a situation where the minimum force closure numerical index of the cargo target grasping point does not remain within the rated grasping force numerical range of the logistics picking robot. If so, label the corresponding cargo as type I cargo, and label the remaining cargos as type II qualified cargos;

[0029] If there is a type of goods, adjust the numerical range of the grasping force of the logistics picking robot, and control the numerical range of the grasping force of the logistics picking robot to meet the minimum force closure numerical index of the goods at the target grasping point of the goods, so that all goods are second-class qualified goods;

[0030] Combine the second-class qualified goods and the logistics picking robot to conduct dynamic simulation grasping simulation, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods based on the simulation results.

[0031] Further, in a preferred embodiment of the present invention, the combining the second-class qualified goods and the logistics picking robot to conduct dynamic simulation grasping simulation, and evaluating the grasping stability of the logistics picking robot at the target grasping point of the goods specifically includes:

[0032] Introduce dynamic simulation grasping simulation software, and input the current working parameters of the logistics picking robot into the dynamic simulation grasping simulation software to obtain a logistics picking simulation robot. At the same time, import the target goods stacking map into the dynamic simulation grasping simulation software to obtain second-class qualified simulated goods;

[0033] Among them, set the type and specification data of the second-class qualified simulated goods in the dynamic simulation grasping simulation software, and at the same time obtain the distance that the second-class qualified simulated goods need to be grasped, and calibrate it as the standard grasping distance;

[0034] In the dynamic simulation grasping simulation software, use the logistics picking simulation robot to simulate grasping the second-class qualified simulated goods. During the simulation grasping process, test the slippage of the second-class qualified simulated goods when the logistics picking simulation robot moves within the standard grasping distance, and preset the maximum slippage of the second-class qualified simulated goods when the logistics picking simulation robot moves;

[0035] Analyze the slippage and the maximum slippage of the second-class qualified simulated goods when the logistics picking simulation robot moves to obtain a slippage difference. Preset the maximum slippage difference. If the slippage difference is not greater than the maximum slippage difference, evaluate that the grasping stability of the logistics picking robot at the target grasping point of the goods is qualified, and divide the target grasping point of the goods into a qualified grasping point of the goods;

[0036] If the slippage difference is greater than the maximum slippage difference, evaluate that the grasping stability of the logistics picking robot at the target grasping point of the goods is unqualified.

[0037] Further, in a preferred embodiment of the present invention, the grasping optimization of the logistics picking robot with unqualified grasping stability is carried out, and the logistics picking robot after grasping optimization is controlled to grasp the goods, specifically including:

[0038] In the dynamic simulation grasping simulation software, obtain the logistics picking simulation robot with unqualified stability, calibrate it as a type I simulation robot, and set a replanning algorithm in the type I simulation robot;

[0039] Among them, the replanning algorithm is that when the detected slip amount difference is greater than the maximum slip amount difference, a slip risk is generated in the type I simulation robot, and the movement of the type I simulation robot is stopped;

[0040] After stopping the movement of the type I simulation robot, determine the center-of-gravity position of the type II qualified simulation goods, and update the goods target grasping point based on the center-of-gravity position of the type II qualified simulation goods to obtain an optimized goods target grasping point;

[0041] Control the type I simulation robot to simulate grasping the type II qualified simulation goods in combination with the optimized goods target grasping point. If the slip amount difference is still greater than the maximum slip amount difference, in the dynamic simulation grasping simulation software, determine all suitable goods grasping points of the type II qualified simulation goods, and introduce the Pareto front analysis algorithm to generate a consumption curve combining the slip amount difference and time corresponding to different goods grasping points, marked as the target analysis curve;

[0042] Analyze the target analysis curve, select the goods grasping point corresponding to the smallest slip amount difference within the same time, and classify it as the goods qualified grasping point;

[0043] Import the parameters of the type I simulation robot set in the dynamic simulation grasping simulation software into the logistics picking robot for parameter update to obtain a type I logistics picking robot, and control the type I logistics picking robot to grasp the goods in the goods stacking area in combination with the goods qualified grasping point.

[0044] The second aspect of the present invention also provides a logistics picking robot operation management system based on feedback analysis. The logistics picking robot operation management system includes a memory and a processor. The memory stores a logistics picking robot operation management method. When the logistics picking robot operation management method is executed by the processor, the following steps are implemented:

[0045] Obtain the goods image, depth information, and goods point cloud data in the goods stacking area, generate a point cloud map of the goods stacking area, and perform filtering processing on the point cloud map of the goods stacking area to achieve spatio-temporal alignment of map parameters and obtain a target goods stacking map;

[0046] Introduce a multi-channel attention module into the target goods stacking map to enhance the graphics of the occluded area in the target goods stacking map and identify the coordinate positions of the stacked goods;

[0047] Generate a target grasping point for the goods according to the coordinate position of the goods, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods;

[0048] Perform grasping optimization on the logistics picking robot with unqualified grasping stability, and control the logistics picking robot after grasping optimization to perform goods grasping.

[0049] The technical defects existing in the background art solved by the present invention, the present invention has the following beneficial effects: constructing a three-dimensional point cloud map of the goods stacking area through image analysis and three-dimensional modeling methods, and combining a multi-channel attention mechanism to perform graphic enhancement on the stacked goods in the three-dimensional point cloud map to determine the coordinate position of the stacked goods, thereby generating the grasping point of the goods. Use the logistics picking robot to combine the grasping point of the goods to perform the goods grasping stability test, and optimize the goods grasping stability of the logistics picking machine based on the test results. The present invention can optimize the goods grasping stability based on the pixel-level grasping of the multi-space and multi-channel attention mechanism, realize stable and reliable goods grasping processing, and avoid the problem of low visibility of some grasping area targets caused by the limitation of single-view information in the occluded stacking environment. Brief Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0051] Figure 1 Shows a flowchart of a method for operating and managing a logistics picking robot based on feedback analysis;

[0052] Figure 2 Shows a flowchart of a method for evaluating the grasping stability of a logistics picking robot at the target grasping point of the goods;

[0053] Figure 3 Shows a program view of a logistics picking robot operation management system based on feedback analysis. Detailed Embodiments

[0054] In order to be able to more clearly understand the above-mentioned objects, features and advantages of the present invention, the following will further describe the present invention in detail with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0055] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0056] Figure 1 The flowchart of the operation management method of the logistics picking robot based on feedback analysis is shown, including the following steps:

[0057] S102: Obtain the cargo image, depth information, and cargo point cloud data in the cargo stacking area, generate a point cloud map of the cargo stacking area, and perform filtering processing on the point cloud map of the cargo stacking area to achieve spatio-temporal alignment of the map parameters, and obtain the target cargo stacking map;

[0058] S104: Introduce a multi-channel attention module into the target cargo stacking map, enhance the graphics of the occluded areas in the target cargo stacking map, and identify the coordinate positions of the stacked cargos;

[0059] S106: Generate the target grasping points of the cargos according to the coordinate positions of the cargos, and evaluate the grasping stability of the logistics picking robot at the target grasping points of the cargos;

[0060] S108: Optimize the grasping of the logistics picking robot with unqualified grasping stability, and control the logistics picking robot after grasping optimization to perform cargo grasping.

[0061] Further, in a preferred embodiment of the present invention, the obtaining the cargo image, depth information, and cargo point cloud data in the cargo stacking area, generating a point cloud map of the cargo stacking area, and performing filtering processing on the point cloud map of the cargo stacking area to achieve spatio-temporal alignment of the map parameters, and obtaining the target cargo stacking map is specifically as follows:

[0062] Determine the cargo stacking area, where the cargo stacking area is the area where the cargos that need to be operated and managed by the logistics picking robot are stacked;

[0063] Obtain an RGB-D camera, and based on the RGB-D camera, obtain the scene color image in the cargo stacking area to obtain the cargo image of the cargo stacking area, which is calibrated as the target cargo image. At the same time, based on the RGB-D camera, obtain the cargo depth information in the cargo stacking area to obtain the cargo depth information of the cargo stacking area;

[0064] Fuse the cargo depth information of the cargo stacking area into the target cargo image so that each pixel point of the cargo contains the corresponding cargo depth information. Introduce 3D modeling software, and import the target cargo image fused with the cargo depth information into the 3D modeling software to obtain the cargo point cloud data of the target cargo image fused with the cargo depth information;

[0065] Based on the point cloud data of the target cargo image that incorporates the depth information of the cargo in the three-dimensional modeling software, construct a point cloud map of the cargo stacking area;

[0066] In the three-dimensional modeling software, introduce the Kalman filtering algorithm. Based on the Kalman filtering algorithm, perform parameter alignment on the cargo point cloud data on the point cloud map of the cargo stacking area;

[0067] Among them, the method of performing parameter alignment on the cargo point cloud data on the point cloud map of the cargo stacking area is to define the three-dimensional coordinate information of the cargo point cloud data, thereby constructing the speed parameter and spatial parameter when obtaining the target cargo image. Based on the speed parameter and spatial parameter when obtaining the target cargo image, establish a state transition model;

[0068] Obtain the generation time of the target cargo image. Combine with the state transition model to achieve the spatio-temporal synchronization of the point cloud map of the cargo stacking area, and obtain a spatio-temporally aligned point cloud map of the cargo stacking area, which is calibrated as the target cargo stacking map.

[0069] It should be noted that the RGB-D camera can obtain the color image, the depth information of the image, and the point cloud data in the scene, which is the premise for constructing the cargo stacking point cloud map. And in the three-dimensional modeling software, the acquired data can be integrated and analyzed to achieve the construction of the cargo stacking point cloud map. The Kalman filtering algorithm is a filtering algorithm for realizing data alignment. Because the time and space dimensions are different during data acquisition, the difference in time and space dimensions will lead to the lack of rigor of the data. Therefore, it is necessary to achieve data alignment and improve the rigor of the data, so as to achieve precise positioning when the subsequent logistics picking robot operates. Align the cargo point cloud data through the Kalman filtering algorithm. First, it is necessary to control the speed parameter and spatial parameter of the robot when obtaining the target cargo image, that is, the three-dimensional coordinates of the robot with speed information, and construct a state transition model. Among them, the state transition model describes how the robot evolves over time. For example, the displacement motion state of the robot under the uniform speed state and the state with acceleration. Finally, combine with the state transition model, and map the data in the software to achieve the time-space state synchronization and obtain the target cargo stacking map.

[0070] Furthermore, in a preferred embodiment of the present invention, a multi-channel attention module is introduced into the target cargo stacking map to enhance the graphics of the occluded area in the target cargo stacking map and identify the coordinate positions of the stacked cargo. Specifically:

[0071] Introduce a CNN neural network to extract RGB branch features from the target cargo stacking map, obtaining the texture features and color features of the target cargo stacking map. At the same time, perform depth branch extraction on the target cargo stacking map to obtain the geometric structure features of the target cargo stacking map;

[0072] Introduce a bilinear attention module. In the bilinear attention module, import the texture features, color features, and geometric structure features of the target cargo stacking map, and calculate the weight ratios of different features in the target cargo stacking map through the bilinear attention module. At the same time, use the bilinear attention module to generate a spatial mask in the target cargo stacking map to enhance the feature response speed of the cargo occlusion boundary in the target cargo stacking map;

[0073] In the target cargo stacking map, combine the spatial mask and the weight ratios of different features, and perform dilated convolution on the target cargo stacking map through the CNN neural network to obtain an enhanced target cargo stacking map;

[0074] In the enhanced target cargo stacking map, locate the position of the spatial mask, output the visible cargo area and the occluded cargo area based on the position of the spatial mask, and determine the coordinate positions of different cargos in the visible cargo area within the 3D modeling software;

[0075] For the occluded cargo area, introduce a geometric completion network to perform depth speculation on the occluded cargo area, construct the cargo contours in the occluded cargo area, and combine with the 3D modeling software to determine the coordinate positions of different cargos in the occluded cargo area.

[0076] It should be noted that after obtaining the target cargo stacking map, it is necessary to enhance the cargo on the target cargo stacking map to achieve the purpose of locating the cargo position. Since the cargo is not completely loose, some cargo will be in an occluded situation. If there is cargo occlusion, it is easy to touch the cargo during the picking process, resulting in the collapse of the cargo and causing losses. There will also be a problem that the cargo cannot be picked. First, a CNN neural network is introduced, which is a convolutional neural network. The purpose is to filter background noise by combining depth information, identify the cargo position, and separate overlapping cargo. The CNN neural network is a deep learning model specifically used to process grid-like data (such as images, videos, audio). Its core automatically extracts spatial local features through convolutional operations. The CNN neural network includes a convolutional layer, a pooling layer, and a fully connected layer. By performing a sliding convolutional scan in the target cargo stacking map in the convolutional layer, feature data can be extracted, and the feature data is downsampled in the pooling layer, that is, max pooling is performed to reduce the computational amount. Finally, the extracted feature map is output through the fully connected layer, and the texture features and color features of the target cargo stacking map can be obtained. During the working process of the CNN neural network, the loss function is a function for training included in the convolutional kernel during the sliding convolutional scan in the convolutional layer. The working process is that the convolutional kernel slides for convolution, and the loss function calculates the cross-entropy loss of the data in the target cargo stacking map to achieve data feature extraction.

[0077] There is a multi-channel attention mechanism in the multi-channel attention module, which can enhance the detection ability of occluded areas. The attention mechanism is a technology that mimics human visual attention. By dynamically allocating weights, the model focuses on the key parts of the input data. The bilinear attention module contains an attention mechanism. By analyzing the texture features, color features, and geometric structure features of the target cargo stacking map, the importance of different features can be evaluated, and the key areas can be highlighted, that is, by allocating weights to different data to focus on the key feature data.

[0078] The following gives the text description of the CNN neural network framework diagram:

[0079] Input the target cargo stacking map - the convolutional layer performs sliding convolutional processing to extract feature data - the bilinear attention module calculates the weight ratio of different features in the target cargo stacking map - the pooling layer downsamples to reduce the computational amount - repeat convolution + attention + pooling - the fully connected layer outputs features, and through the cross-entropy loss of the data in the target cargo stacking map by the loss function, data feature extraction is achieved.

[0080] When performing sliding convolutional processing in the convolutional layer to extract feature data, the corresponding convolutional formula is:

[0081] Among them, X is the input target goods stacking map, i, j, k, v, u, c are constants, W represents the convolution kernel, b is the bias term, and Y is the output feature map.

[0082] The pooling layer performs downsampling to reduce the computational amount. That is, the corresponding formula during pooling processing is:

[0083] Among them, R is the pooling window, S is the stride, and other letters are the same as the corresponding letters in the convolution formula.

[0084] During the process of calculating the weight proportion of different features in the target goods stacking map by the bilinear attention module, the formula for weight calculation is:

[0085]

[0086] Among them, GAP represents global pooling, FC is the fully connected layer, represents three different feature data. When the feature data obtained in the CNN neural network needs to perform element-wise multiplication after being output by the fully connected layer, the cross-entropy loss of the data needs to be calculated. The calculation formula is: Among them, is the true label, is the predicted probability. Through this formula, the purpose of data feature extraction can be achieved. Element-wise multiplication can also enhance the response of key regions. Finally, different feature data are spliced to achieve the purpose of highlighting key regions. The key regions are, for example, more dependent on depth geometric information in the occluded region, and more dependent on texture features and color features in other regions. The purpose of generating the spatial mask is to enhance the feature response at the occlusion boundary to separate the goods, facilitating logistics picking. Dilated convolution is used to expand the receptive field to obtain the context information of the occluded object, thereby strengthening the features in the occluded region and realizing the positioning of the coordinate positions of the goods in the occluded region. Because in 3D software, the coordinate positions at different locations can all be located. By combining the occluded region and the visible region, the coordinate positions of all goods can be determined.

[0087] Furthermore, in a preferred embodiment of the present invention, the logistics picking robot with unqualified grasping stability is optimized for grasping, and the optimized logistics picking robot is controlled to perform goods grasping, specifically:

[0088] In the dynamic simulation grasping simulation software, obtain the logistics picking simulation robot with unqualified stability, calibrate it as a type of simulation robot, and set a replanning algorithm in the type of simulation robot;

[0089] Among them, the replanning algorithm is that when the monitored slip amount difference is greater than the maximum slip amount difference, a slip risk is generated in the type of simulation robot, and the movement of the type of simulation robot is stopped;

[0090] After stopping the movement of a first type of simulated robot, determine the center-of-gravity position of qualified simulated goods of a second type, and update the target grasping point of the goods based on the center-of-gravity position of the qualified simulated goods of the second type to obtain an optimized target grasping point of the goods;

[0091] Control the first type of simulated robot to perform simulated grasping on the qualified simulated goods of the second type in combination with the optimized target grasping point of the goods. If the difference in slip amount is still greater than the maximum difference in slip amount, then in the dynamic simulation grasping simulation software, determine all suitable goods grasping points of the qualified simulated goods of the second type, and introduce the Pareto front analysis algorithm to generate a consumption curve combining the difference in slip amount and time corresponding to different goods grasping points, marked as the target analysis curve;

[0092] Analyze the target analysis curve, select the corresponding goods grasping point with the smallest difference in slip amount within the same time, and classify it as a qualified goods grasping point;

[0093] Import the parameters of the first type of simulated robot set in the dynamic simulation grasping simulation software into the logistics picking robot for parameter update to obtain a first type of logistics picking robot, and control the first type of logistics picking robot to perform grasping processing on the goods in the goods stacking area in combination with the qualified goods grasping point.

[0094] It should be noted that the replanning algorithm is that when the sensor detects that the grasping force is abnormal or the pose of the goods deviates beyond the threshold, the replanning process is triggered. For example, if the grasping pressure of the robot drops suddenly or the slip is too large, the movement is immediately stopped and the grasping point is recalculated to improve stability. After stopping the movement of the robot, it is necessary to replan the process of the robot grasping the goods, such as replanning the grasping point, because different grasping points result in different displacement states during grasping. Update the pose of the goods according to the real-time point cloud, and preferentially select the grasping point closer to the center of gravity to improve stability. Therefore, calculate the center-of-gravity position of the qualified simulated goods of the second type to obtain an optimized target grasping point of the goods. If the difference in slip amount is still greater than the maximum difference in slip amount when using the optimized target grasping point of the goods, it proves that the grasping point is still incorrect, and it is necessary to introduce the Pareto front analysis algorithm to construct the consumption curve of the difference in slip amount and time corresponding to different goods grasping points. The optimal solution of the curve is the corresponding goods grasping point with the smallest difference in slip amount within the same time. Different goods correspond to different goods grasping points, so specific parameter settings need to be made for the robot. Finally, import the parameters of the first type of simulated robot set in the dynamic simulation grasping simulation software into the logistics picking robot for parameter update.

[0095] Figure 2 The flowchart of the method for evaluating the grasping stability of a logistics picking robot at the target grasping point of the goods is shown, including the following steps:

[0096] S202: Generate a target grasping point for the goods according to the coordinate position of the goods, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods;

[0097] S204: Combine the second-class qualified goods and the logistics picking robot to perform dynamic simulation grasping, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods based on the simulation results.

[0098] Further, in a preferred embodiment of the present invention, the generating a target grasping point for the goods according to the coordinate position of the goods and evaluating the grasping stability of the logistics picking robot at the target grasping point of the goods is specifically as follows:

[0099] Define the coordinate positions of different goods in the visible goods area and the occluded goods area as the target coordinate positions of the goods, and introduce a big data network. Based on the big data network, retrieve the graspable positions of all types of goods in the goods stacking area when using the logistics picking robot, and mark them as the target grasping points of the goods;

[0100] Obtain the rated grasping force value range of the logistics picking robot, and analyze the target grasping points of all goods based on the big data network to determine the minimum force closure value index of different goods at the target grasping points of the goods;

[0101] Judge whether there is a situation where the minimum force closure value index of the target grasping point of the goods does not remain within the rated grasping force value range of the logistics picking robot. If so, label the corresponding goods as first-class goods, and label the remaining goods as second-class qualified goods;

[0102] If there are first-class goods, adjust the grasping force value range of the logistics picking robot to control the grasping force value range of the logistics picking robot to meet the minimum force closure value index of the goods at the target grasping point of the goods, so that all goods are second-class qualified goods;

[0103] Combine the second-class qualified goods and the logistics picking robot to perform dynamic simulation grasping, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods based on the simulation results.

[0104] It should be noted that goods are usually packed in containers such as boxes or bags. For containers of different sizes, the grasping points are different, and there are points on the containers that can be grasped by the robot, that is, the target grasping points of the goods. In the big data network, the target grasping points of the goods can be obtained according to the specifications and types of the containers. The logistics picking robot grasps the goods through devices such as hooks. The device has an upper limit of grasping force, that is, the force closure value. If the minimum force closure value index of the goods at the target grasping point of the goods does not remain within the rated grasping force value range of the logistics picking robot, it proves that the goods cannot be firmly grasped. At this time, it is necessary to adjust the grasping force value of the robot to ensure that the goods can be firmly grasped, obtain all the second-class qualified goods, and start dynamic simulation grasping simulation to evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods.

[0105] Furthermore, in a preferred embodiment of the present invention, the combination of the second-class qualified goods and the logistics picking robot is used for dynamic simulation grasping simulation, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results. Specifically:

[0106] Introduce dynamic simulation grasping simulation software, and input the current working parameters of the logistics picking robot into the dynamic simulation grasping simulation software to obtain a logistics picking simulation robot. At the same time, import the target goods stacking map into the dynamic simulation grasping simulation software to obtain second-class qualified simulated goods;

[0107] Among them, set the type and specification data of the second-class qualified simulated goods in the dynamic simulation grasping simulation software, and at the same time obtain the distance that the second-class qualified simulated goods need to be grasped, which is calibrated as the standard grasping distance;

[0108] In the dynamic simulation grasping simulation software, simulate the grasping of the second-class qualified simulated goods by the logistics picking simulation robot. During the simulation grasping process, test the slip amount of the second-class qualified simulated goods when the logistics picking simulation robot moves within the standard grasping distance, and preset the maximum slip amount of the second-class qualified simulated goods when the logistics picking simulation robot moves;

[0109] Analyze the combination of the slip amount and the maximum slip amount of the second-class qualified simulated goods when the logistics picking simulation robot moves to obtain a slip amount difference, and preset a maximum slip amount difference. If the slip amount difference is not greater than the maximum slip amount difference, it is evaluated that the grasping stability of the logistics picking robot at the target grasping point of the goods is qualified, and the target grasping point of the goods is divided into a qualified grasping point of the goods;

[0110] If the slip amount difference is greater than the maximum slip amount difference, it is evaluated that the grasping stability of the logistics picking robot at the target grasping point of the goods is unqualified.

[0111] It should be noted that since there are some obstacles in the actual situation of the grasping test, the grasping test is carried out in a simulated manner. By introducing a dynamic simulation grasping simulation software, the grasping simulation can be carried out. The distance that the second-class qualified simulated goods need to be grasped, that is, the distance from the storage point to the destination where the goods are grasped, is calibrated as the standard grasping distance. In the standard grasping distance, if the robot grasps statically, the situation of not grasping firmly will not occur, but in the case of movement, the situation of goods slipping may occur. According to the difference in the slipping amount, it can be judged whether there is danger during the movement of the goods. If the difference in the slipping amount is greater than the maximum difference in the slipping amount, it is evaluated that the grasping stability of the logistics picking robot at the target grasping point of the goods is unqualified. At this time, the grasping cannot continue, and the robot also needs to be optimized and updated to prevent the difference in the slipping amount from being greater than the maximum difference in the slipping amount.

[0112] As Figure 3 shown, the second aspect of the present invention also provides a logistics picking robot operation management system based on feedback analysis. The logistics picking robot operation management system includes a memory 31 and a processor 32. The memory 31 stores a logistics picking robot operation management method. When the logistics picking robot operation management method is executed by the processor 32, the following steps are realized:

[0113] Obtain the goods image, depth information and goods point cloud data in the goods stacking area, generate a point cloud map of the goods stacking area, and perform filtering processing on the point cloud map of the goods stacking area to achieve spatio-temporal alignment of the map parameters, and obtain a target goods stacking map;

[0114] Introduce a multi-channel attention module into the target goods stacking map, perform graphic enhancement on the occluded areas in the target goods stacking map, and identify the coordinate positions of the stacked goods;

[0115] Generate a target grasping point for the goods according to the coordinate positions of the goods, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods;

[0116] Perform grasping optimization on the logistics picking robot with unqualified grasping stability, and control the logistics picking robot after grasping optimization to perform goods grasping.

[0117] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A logistics picking robot operation management method based on feedback analysis, characterized in that: The following steps are involved: Acquire cargo images, depth information, and cargo point cloud data in the cargo stacking area, generate a point cloud map of the cargo stacking area, and perform filtering processing on the point cloud map of the cargo stacking area to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map; A multi-channel attention module is introduced into the target cargo stacking map to enhance the graphics of the occluded areas in the target cargo stacking map and identify the coordinate positions of the stacked cargo; Generate the cargo target grabbing point based on the cargo coordinate position, and evaluate the grabbing stability of the logistics picking robot at the cargo target grabbing point; Perform grasping optimization on the logistics picking robots with unqualified grasping stability, and control the logistics picking robots after grasping optimization to grasp goods; The method of performing grasping optimization on the logistics picking robot with unqualified grasping stability and controlling the logistics picking robot after grasping optimization to grasp goods is specifically as follows: In the dynamic simulation grabbing simulation software, a logistics picking simulation robot with unqualified stability is obtained, calibrated as a type of simulation robot, and a replanning algorithm is set in the type of simulation robot; Wherein, the replanning algorithm generates a slip risk in a type of simulated robot and stops the movement of the type of simulated robot when it is detected that the slip difference is greater than the maximum slip difference; After stopping the movement of the first type of simulated robot, the center of gravity position of the second type of qualified simulated goods is determined, and based on the center of gravity position of the second type of qualified simulated goods, the target grasping point of the goods is updated to obtain the optimized target grasping point of the goods; Control the first type of simulated robot to simulate the grasping of the second type of qualified simulated goods in combination with the optimized target grasping point of the goods. If the slip difference is still greater than the maximum slip difference, then determine all suitable grasping points of the second type of qualified simulated goods in the dynamic simulation grasping simulation software, and introduce the Pareto frontier analysis algorithm to generate a consumption curve that combines the slip difference and time corresponding to different grasping points of the goods, which is marked as the target analysis curve; Analyze the target analysis curve, select the corresponding cargo grabbing point with the smallest slip difference within the same time, and classify it as a qualified cargo grabbing point; The parameters of a type of simulation robot set in the dynamic simulation grasping simulation software are imported into the logistics picking robot to update the parameters, so as to obtain a type of logistics picking robot, and the type of logistics picking robot is controlled to grasp and process the goods in the goods stacking area in combination with the qualified grasping points of the goods.

2. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The cargo image, depth information and cargo point cloud data are obtained in the cargo stacking area, a point cloud map of the cargo stacking area is generated, and the point cloud map of the cargo stacking area is filtered to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map, specifically: Determine a cargo stacking area, wherein the cargo stacking area is an area where cargo that needs to be operated and managed by a logistics picking robot is stacked; Acquire an RGB-D camera, and acquire a scene color image in the cargo stacking area based on the RGB-D camera to obtain a cargo image in the cargo stacking area, calibrate it as a target cargo image, and acquire cargo depth information in the cargo stacking area based on the RGB-D camera to obtain cargo depth information in the cargo stacking area; The cargo depth information of the cargo stacking area is integrated into the target cargo image so that each pixel of the cargo contains the corresponding cargo depth information, and a three-dimensional modeling software is introduced, and the target cargo image integrated with the cargo depth information is imported into the three-dimensional modeling software to obtain cargo point cloud data of the target cargo image integrated with the cargo depth information; In the three-dimensional modeling software, based on the cargo point cloud data of the target cargo image integrated with the cargo depth information, a point cloud map of the cargo stacking area is constructed; In the three-dimensional modeling software, a Kalman filter algorithm is introduced, and based on the Kalman filter algorithm, parameter alignment of cargo point cloud data is performed on a cargo stacking area point cloud map; The method for aligning the parameters of the cargo point cloud data on the cargo stacking area point cloud map is to define the three-dimensional coordinate information of the cargo point cloud data, thereby constructing the speed parameters and space parameters when acquiring the target cargo image, and establishing the state transfer model based on the speed parameters and space parameters when acquiring the target cargo image; The generation time of the target cargo image is obtained, and combined with the state transfer model, the spatiotemporal synchronization of the cargo stacking area point cloud map is realized to obtain the spatiotemporal aligned cargo stacking area point cloud map, which is calibrated as the target cargo stacking map.

3. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The multi-channel attention module is introduced into the target cargo stacking map to perform graphic enhancement on the blocked areas in the target cargo stacking map and identify the coordinate positions of the stacked cargoes, specifically: The CNN neural network is introduced to extract RGB branch features of the target cargo stacking map to obtain the texture features and color features of the target cargo stacking map. At the same time, the depth branch extraction is performed on the target cargo stacking map to obtain the geometric structure features of the target cargo stacking map. A bilinear attention module is introduced, in which texture features, color features and geometric structure features of the target cargo stacking map are imported, and the weight proportions of different features in the target cargo stacking map are calculated by the bilinear attention module. At the same time, the bilinear attention module is used to generate a spatial mask in the target cargo stacking map to enhance the feature response speed of the cargo occlusion boundary in the target cargo stacking map; In the target cargo stacking map, the spatial mask and the weight proportions of different features are combined, and the target cargo stacking map is subjected to dilated convolution through the CNN neural network to obtain an enhanced target cargo stacking map; In the enhanced target cargo stacking map, locate the spatial mask position, output the visible cargo area and the obscured cargo area based on the spatial mask position, and determine the coordinate positions of different cargoes in the visible cargo area in the three-dimensional modeling software; For the obscured cargo area, a geometric completion network is introduced to perform depth inference on the obscured cargo area, construct the cargo outline in the obscured cargo area, and combine it with 3D modeling software to determine the coordinate positions of different cargoes in the obscured cargo area.

4. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The method generates a target grabbing point for the goods according to the coordinate position of the goods, and evaluates the grabbing stability of the logistics picking robot at the target grabbing point for the goods, specifically: The coordinate positions of different goods in the visible goods area and the obscured goods area are defined as the target coordinate positions of the goods, and a big data network is introduced. Based on the big data network, the graspable positions of all types of goods in the goods stacking area when using the logistics picking robot are retrieved and marked as the target grasping points of the goods. Obtain the rated grasping force value range of the logistics picking robot, and analyze the cargo target grasping points of all goods based on the big data network to determine the minimum force closing value index of different goods at the cargo target grasping points; Determine whether there are goods whose minimum force closing value index at the target grasping point of the goods is not maintained within the rated grasping force value range of the logistics picking robot. If so, mark the corresponding goods as Class I goods, and mark the remaining goods as Class II qualified goods; If there are Class I goods, the grabbing force value range of the logistics picking robot is adjusted to control the grabbing force value range of the logistics picking robot to meet the minimum force closure value index of the goods at the target grabbing point of the goods, so that all goods are qualified Class II goods; Combining the second type of qualified goods and the logistics picking robot, dynamic simulation grasping simulation is carried out, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results.

5. The logistics picking robot operation management method based on feedback analysis according to claim 4 is characterized in that: The two types of qualified goods and the logistics picking robot are combined to perform dynamic simulation grasping simulation, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results, specifically: Introducing dynamic simulation grabbing simulation software, and inputting the current working parameters of the logistics picking robot into the dynamic simulation grabbing simulation software to obtain the logistics picking simulation robot, and at the same time importing the target cargo stacking map into the dynamic simulation grabbing simulation software to obtain the second category of qualified simulated cargo; Among them, the type and specification data of the second-class qualified simulated goods are set in the dynamic simulation grabbing simulation software, and the distance required for grabbing the second-class qualified simulated goods is obtained and calibrated as the standard grabbing distance; In the dynamic simulation grabbing simulation software, the logistics picking simulation robot simulates the grabbing of the second category of qualified simulated goods, and during the simulated grabbing process, the slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves within the standard grabbing distance is tested, and the maximum slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves is preset; The slip amount and maximum slip amount of the two types of qualified simulated goods when the logistics picking simulation robot moves are analyzed to obtain the slip amount difference, and the maximum slip amount difference is preset. If the slip amount difference is not greater than the maximum slip amount difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated to be qualified, and the target grasping point of the goods is classified as a qualified grasping point of the goods; If the slip difference is greater than the maximum slip difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated as unqualified.

6. Logistics picking robot operation management system based on feedback analysis, characterized in that: The logistics picking robot operation management system includes a memory and a processor, wherein the memory stores a logistics picking robot operation management method program. When the logistics picking robot operation management method program is executed by the processor, the logistics picking robot operation management method steps as described in any one of claims 1-5 are implemented.

Images