Logistics picking robot operation management method and system based on feedback analysis

Abstract

The present invention relates to the field of logistics robots, and discloses a logistics picking robot operation management method and system based on feedback analysis, including the following steps: constructing a three-dimensional point cloud map of the cargo stacking area through image analysis and three-dimensional modeling methods, and performing graphic enhancement on the stacked cargo in the three-dimensional point cloud map in combination with a multi-channel attention mechanism, determining the coordinate position of the stacked cargo, and thereby generating the cargo grabbing points. A logistics picking robot is used to perform cargo grabbing stability testing in combination with the cargo grabbing points, and the cargo grabbing stability of the logistics picking machine is optimized based on the test results. The present invention can optimize cargo grabbing stability based on pixel-level grabbing of a multi-space, multi-channel attention mechanism, achieve stable and reliable cargo grabbing processing, and avoid the problem of low visibility of targets in some grabbing areas due to the limitation of single-view information in an occluded stacking environment.

Description

Technical Field

[0001] The present invention relates to the field of logistics robots, and in particular to a logistics picking robot operation management method and system based on feedback analysis. Background Art

[0002] When goods are stacked and picked up by logistics picking robots, the images recognized by the robots may be uneven in brightness, blurred, or have poor contrast, resulting in incomplete information about the target goods. The above situations may lead to problems such as poor picking results and damaged goods during the picking process.

[0003] In order to solve the problem of low grasping success rate caused by irregular shapes of target goods and stacking of objects in cluttered environments, a pixel-level grasping generation convolutional neural network based on multi-space and multi-channel attention mechanism is constructed to achieve one-to-one mapping between image pixels and grasping space; the candidate area is standardized through affine transformation, and a grasping selection network considering grasping depth is constructed. The grasping set is evaluated and screened based on the quality maximization constraint to achieve stable and reliable grasping detection. Therefore, a logistics picking robot operation management method and system based on feedback analysis is proposed to achieve the accuracy and stability of logistics picking robots in grasping goods. Summary of the invention

[0004] The present invention overcomes the deficiencies of the prior art and provides a logistics picking robot operation management method and system based on feedback analysis.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: The first aspect of the present invention provides a logistics picking robot operation management method based on feedback analysis, comprising the following steps: Acquire cargo images, depth information, and cargo point cloud data in the cargo stacking area, generate a point cloud map of the cargo stacking area, and perform filtering processing on the point cloud map of the cargo stacking area to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map; A multi-channel attention module is introduced into the target cargo stacking map to enhance the graphics of the occluded areas in the target cargo stacking map and identify the coordinate positions of the stacked cargo; Generate the cargo target grabbing point based on the cargo coordinate position, and evaluate the grabbing stability of the logistics picking robot at the cargo target grabbing point; The logistics picking robots with unqualified grasping stability are optimized for grasping, and the logistics picking robots after grasping optimization are controlled to grasp goods.

[0006] Furthermore, in a preferred embodiment of the present invention, the cargo image, depth information and cargo point cloud data are obtained in the cargo stacking area to generate a point cloud map of the cargo stacking area, and the point cloud map of the cargo stacking area is filtered to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map, specifically: Determine a cargo stacking area, wherein the cargo stacking area is an area where cargo that needs to be operated and managed by a logistics picking robot is stacked; Acquire an RGB-D camera, and acquire a scene color image in the cargo stacking area based on the RGB-D camera to obtain a cargo image in the cargo stacking area, calibrate it as a target cargo image, and acquire cargo depth information in the cargo stacking area based on the RGB-D camera to obtain cargo depth information in the cargo stacking area; The cargo depth information of the cargo stacking area is integrated into the target cargo image so that each pixel of the cargo contains the corresponding cargo depth information, and a three-dimensional modeling software is introduced, and the target cargo image integrated with the cargo depth information is imported into the three-dimensional modeling software to obtain cargo point cloud data of the target cargo image integrated with the cargo depth information; In the three-dimensional modeling software, based on the cargo point cloud data of the target cargo image integrated with the cargo depth information, a point cloud map of the cargo stacking area is constructed; In the three-dimensional modeling software, a Kalman filter algorithm is introduced, and based on the Kalman filter algorithm, parameter alignment of cargo point cloud data is performed on a cargo stacking area point cloud map; The method for aligning the parameters of the cargo point cloud data on the cargo stacking area point cloud map is to define the three-dimensional coordinate information of the cargo point cloud data, thereby constructing the speed parameters and space parameters when acquiring the target cargo image, and establishing the state transfer model based on the speed parameters and space parameters when acquiring the target cargo image; The generation time of the target cargo image is obtained, and combined with the state transfer model, the spatiotemporal synchronization of the cargo stacking area point cloud map is realized to obtain the spatiotemporal aligned cargo stacking area point cloud map, which is calibrated as the target cargo stacking map.

[0007] Furthermore, in a preferred embodiment of the present invention, a multi-channel attention module is introduced into the target cargo stacking map to perform graphic enhancement on the occluded areas in the target cargo stacking map to identify the coordinate positions of the stacked cargo, specifically: The CNN neural network is introduced to extract RGB branch features of the target cargo stacking map to obtain the texture features and color features of the target cargo stacking map. At the same time, the depth branch extraction is performed on the target cargo stacking map to obtain the geometric structure features of the target cargo stacking map. A bilinear attention module is introduced, in which texture features, color features and geometric structure features of the target cargo stacking map are imported, and the weight proportions of different features in the target cargo stacking map are calculated by the bilinear attention module. At the same time, the bilinear attention module is used to generate a spatial mask in the target cargo stacking map to enhance the feature response speed of the cargo occlusion boundary in the target cargo stacking map; In the target cargo stacking map, the spatial mask and the weight proportions of different features are combined, and the target cargo stacking map is subjected to dilated convolution through the CNN neural network to obtain an enhanced target cargo stacking map; In the enhanced target cargo stacking map, locate the spatial mask position, output the visible cargo area and the obscured cargo area based on the spatial mask position, and determine the coordinate positions of different cargoes in the visible cargo area in the three-dimensional modeling software; For the obscured cargo area, a geometric completion network is introduced to perform depth inference on the obscured cargo area, construct the cargo outline in the obscured cargo area, and combine it with 3D modeling software to determine the coordinate positions of different cargoes in the obscured cargo area.

[0008] Furthermore, in a preferred embodiment of the present invention, the target grabbing point of the goods is generated according to the coordinate position of the goods, and the grabbing stability of the logistics picking robot at the target grabbing point of the goods is evaluated, specifically: The coordinate positions of different goods in the visible goods area and the obscured goods area are defined as the target coordinate positions of the goods, and a big data network is introduced. Based on the big data network, the graspable positions of all types of goods in the goods stacking area when using the logistics picking robot are retrieved and marked as the target grasping points of the goods. Obtain the rated grasping force value range of the logistics picking robot, and analyze the cargo target grasping points of all goods based on the big data network to determine the minimum force closing value index of different goods at the cargo target grasping points; Determine whether the minimum force closing value index of the target grasping point of the goods is not maintained within the rated grasping force value range of the logistics picking robot. If so, the corresponding goods are marked as Class I goods, and the remaining goods are marked as Class II qualified goods; If there are Class I goods, the grabbing force value range of the logistics picking robot is adjusted to control the grabbing force value range of the logistics picking robot to meet the minimum force closure value index of the goods at the target grabbing point of the goods, so that all goods are qualified Class II goods; Combining the second type of qualified goods and the logistics picking robot, dynamic simulation grasping simulation is carried out, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results.

[0009] Furthermore, in a preferred embodiment of the present invention, the combination of the two types of qualified goods and the logistics picking robot is used to perform dynamic simulation grasping simulation, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results, specifically: Introducing dynamic simulation grabbing simulation software, and inputting the current working parameters of the logistics picking robot into the dynamic simulation grabbing simulation software to obtain the logistics picking simulation robot, and at the same time importing the target cargo stacking map into the dynamic simulation grabbing simulation software to obtain the second category of qualified simulated cargo; Among them, the type and specification data of the second-class qualified simulated goods are set in the dynamic simulation grabbing simulation software, and the distance required for grabbing the second-class qualified simulated goods is obtained and calibrated as the standard grabbing distance; In the dynamic simulation grabbing simulation software, the logistics picking simulation robot simulates the grabbing of the second category of qualified simulated goods, and during the simulated grabbing process, the slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves within the standard grabbing distance is tested, and the maximum slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves is preset; The slip amount and maximum slip amount of the two types of qualified simulated goods when the logistics picking simulation robot moves are analyzed to obtain the slip amount difference, and the maximum slip amount difference is preset. If the slip amount difference is not greater than the maximum slip amount difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated to be qualified, and the target grasping point of the goods is classified as a qualified grasping point of the goods; If the slip difference is greater than the maximum slip difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated as unqualified.

[0010] Furthermore, in a preferred embodiment of the present invention, the logistics picking robot with unqualified grasping stability is optimized for grasping, and the logistics picking robot after grasping optimization is controlled to grasp goods, specifically: In the dynamic simulation grabbing simulation software, a logistics picking simulation robot with unqualified stability is obtained, calibrated as a type of simulation robot, and a replanning algorithm is set in the type of simulation robot; Wherein, the replanning algorithm generates a slip risk in a type of simulated robot and stops the movement of the type of simulated robot when it is detected that the slip difference is greater than the maximum slip difference; After stopping the movement of the first type of simulated robot, the center of gravity position of the second type of qualified simulated goods is determined, and based on the center of gravity position of the second type of qualified simulated goods, the target grasping point of the goods is updated to obtain the optimized target grasping point of the goods; Control the first type of simulated robot to simulate the grasping of the second type of qualified simulated goods in combination with the optimized target grasping point of the goods. If the slip difference is still greater than the maximum slip difference, then determine all suitable grasping points of the second type of qualified simulated goods in the dynamic simulation grasping simulation software, and introduce the Pareto frontier analysis algorithm to generate a consumption curve that combines the slip difference and time corresponding to different grasping points of the goods, which is marked as the target analysis curve; Analyze the target analysis curve, select the corresponding cargo grabbing point with the smallest slip difference within the same time, and classify it as a qualified cargo grabbing point; The parameters of a type of simulation robot set in the dynamic simulation grasping simulation software are imported into the logistics picking robot to update the parameters, so as to obtain a type of logistics picking robot, and the type of logistics picking robot is controlled to grasp and process the goods in the goods stacking area in combination with the qualified grasping points of the goods.

[0011] The second aspect of the present invention further provides a logistics picking robot operation management system based on feedback analysis, wherein the logistics picking robot operation management system comprises a memory and a processor, wherein the memory stores a logistics picking robot operation management method, and when the logistics picking robot operation management method is executed by the processor, the following steps are implemented: Acquire cargo images, depth information, and cargo point cloud data in the cargo stacking area, generate a point cloud map of the cargo stacking area, and perform filtering processing on the point cloud map of the cargo stacking area to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map; A multi-channel attention module is introduced into the target cargo stacking map to enhance the graphics of the occluded areas in the target cargo stacking map and identify the coordinate positions of the stacked cargo; Generate the cargo target grabbing point based on the cargo coordinate position, and evaluate the grabbing stability of the logistics picking robot at the cargo target grabbing point; The logistics picking robots with unqualified grasping stability are optimized for grasping, and the logistics picking robots after grasping optimization are controlled to grasp goods.

[0012] The present invention solves the technical defects existing in the background technology, and the present invention has the following beneficial effects: constructing a three-dimensional point cloud map of the cargo stacking area through image analysis and three-dimensional modeling methods, and combining the multi-channel attention mechanism to perform graphic enhancement on the stacked cargo in the three-dimensional point cloud map, determine the coordinate position of the stacked cargo, and thus generate the cargo grabbing points. Use a logistics picking robot to test the cargo grabbing stability in combination with the cargo grabbing points, and optimize the cargo grabbing stability of the logistics picking machine based on the test results. The present invention can optimize cargo grabbing stability based on pixel-level grabbing of multi-space and multi-channel attention mechanisms, realize stable and reliable cargo grabbing processing, and avoid the problem of low visibility of targets in some grabbing areas due to the limitation of single-view information in an occluded stacking environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, drawings of other embodiments can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A flow chart of a logistics picking robot operation management method based on feedback analysis is shown; Figure 2 A flow chart of a method for evaluating the grasping stability of a logistics picking robot at a target grasping point of a cargo is shown; Figure 3 The program view of the logistics picking robot operation management system based on feedback analysis is shown. DETAILED DESCRIPTION

[0015] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0016] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0017] Figure 1 A flow chart of a logistics picking robot operation management method based on feedback analysis is shown, comprising the following steps: S102: acquiring cargo images, depth information, and cargo point cloud data in the cargo stacking area, generating a point cloud map of the cargo stacking area, and filtering the point cloud map of the cargo stacking area to achieve spatiotemporal alignment of map parameters, thereby obtaining a target cargo stacking map; S104: introducing a multi-channel attention module into the target cargo stacking map, performing graphic enhancement on the occluded areas in the target cargo stacking map, and identifying the coordinate positions of the stacked cargo; S106: Generate a target grabbing point for the cargo according to the coordinate position of the cargo, and evaluate the grabbing stability of the logistics picking robot at the target grabbing point for the cargo; S108: Performing grasping optimization on the logistics picking robot with unqualified grasping stability, and controlling the logistics picking robot after grasping optimization to grasp goods.

[0018] Furthermore, in a preferred embodiment of the present invention, the cargo image, depth information and cargo point cloud data are obtained in the cargo stacking area to generate a point cloud map of the cargo stacking area, and the point cloud map of the cargo stacking area is filtered to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map, specifically: Determine a cargo stacking area, wherein the cargo stacking area is an area where cargo that needs to be operated and managed by a logistics picking robot is stacked; Acquire an RGB-D camera, and acquire a scene color image in the cargo stacking area based on the RGB-D camera to obtain a cargo image in the cargo stacking area, calibrate it as a target cargo image, and acquire cargo depth information in the cargo stacking area based on the RGB-D camera to obtain cargo depth information in the cargo stacking area; The cargo depth information of the cargo stacking area is integrated into the target cargo image so that each pixel of the cargo contains the corresponding cargo depth information, and a three-dimensional modeling software is introduced, and the target cargo image integrated with the cargo depth information is imported into the three-dimensional modeling software to obtain cargo point cloud data of the target cargo image integrated with the cargo depth information; In the three-dimensional modeling software, based on the cargo point cloud data of the target cargo image integrated with the cargo depth information, a point cloud map of the cargo stacking area is constructed; In the three-dimensional modeling software, a Kalman filter algorithm is introduced, and based on the Kalman filter algorithm, parameter alignment of cargo point cloud data is performed on a cargo stacking area point cloud map; The method for aligning the parameters of the cargo point cloud data on the cargo stacking area point cloud map is to define the three-dimensional coordinate information of the cargo point cloud data, thereby constructing the speed parameters and space parameters when acquiring the target cargo image, and establishing the state transfer model based on the speed parameters and space parameters when acquiring the target cargo image; The generation time of the target cargo image is obtained, and combined with the state transfer model, the spatiotemporal synchronization of the cargo stacking area point cloud map is realized to obtain the spatiotemporal aligned cargo stacking area point cloud map, which is calibrated as the target cargo stacking map.

[0019] It should be noted that the RGB-D camera can obtain the color image, image depth information and point cloud data in the scene, which is the premise for building the cargo stacking point cloud map. In the 3D modeling software, the acquired data can be integrated and analyzed to realize the construction of the cargo stacking point cloud map. The Kalman filter algorithm is a filtering algorithm for data alignment. Because the time and space dimensions are different when the data is collected, the difference in time and space dimensions will lead to the lack of rigor in the data, so it is necessary to achieve data alignment and improve the rigor of the data, so as to achieve accurate positioning when the subsequent logistics picking robot operates. To align the cargo point cloud data through the Kalman filter algorithm, it is first necessary to control the speed parameters and spatial parameters of the robot when acquiring the target cargo image, that is, the three-dimensional coordinates of the robot with speed information, and build a state transition model. Among them, the state transition model describes how the robot evolves over time, such as the state of the robot's displacement motion in the uniform state and the state with acceleration. Finally, combined with the state transition model, the data is mapped in the software to achieve time and space state synchronization and obtain the target cargo stacking map.

[0020] Furthermore, in a preferred embodiment of the present invention, a multi-channel attention module is introduced into the target cargo stacking map to perform graphic enhancement on the occluded areas in the target cargo stacking map to identify the coordinate positions of the stacked cargo, specifically: The CNN neural network is introduced to extract RGB branch features of the target cargo stacking map to obtain the texture features and color features of the target cargo stacking map. At the same time, the depth branch extraction is performed on the target cargo stacking map to obtain the geometric structure features of the target cargo stacking map. A bilinear attention module is introduced, in which texture features, color features and geometric structure features of the target cargo stacking map are imported, and the weight proportions of different features in the target cargo stacking map are calculated by the bilinear attention module. At the same time, the bilinear attention module is used to generate a spatial mask in the target cargo stacking map to enhance the feature response speed of the cargo occlusion boundary in the target cargo stacking map; In the target cargo stacking map, the spatial mask and the weight proportions of different features are combined, and the target cargo stacking map is subjected to dilated convolution through the CNN neural network to obtain an enhanced target cargo stacking map; In the enhanced target cargo stacking map, locate the spatial mask position, output the visible cargo area and the obscured cargo area based on the spatial mask position, and determine the coordinate positions of different cargoes in the visible cargo area in the three-dimensional modeling software; For the obscured cargo area, a geometric completion network is introduced to perform depth inference on the obscured cargo area, construct the cargo outline in the obscured cargo area, and combine it with 3D modeling software to determine the coordinate positions of different cargoes in the obscured cargo area.

[0021] It should be noted that after obtaining the target cargo stacking map, it is necessary to enhance the cargo on the target cargo stacking map to achieve the purpose of locating the cargo position, because the cargo is not completely loosely placed, and some cargo will be in an obstructed situation. If there is cargo obstruction, it is easy to touch the cargo during the picking process, resulting in cargo collapse and loss, and there will also be problems with cargo that cannot be picked. First, the CNN neural network is introduced, which is a convolutional neural network. The purpose is to filter background noise in combination with deep information, identify cargo locations and separate overlapping cargo. The CNN neural network is a deep learning model specifically used to process grid-like data (such as images, videos, and audio). Its core automatically extracts spatial local features through convolution operations. The CNN neural network includes convolutional layers, pooling layers, and fully connected layers. Through the convolutional layer, sliding convolution scanning can be performed in the target cargo stacking map to extract feature data, and the feature data is downsampled in the pooling layer, that is, the maximum pooling process is performed to reduce the amount of calculation. Finally, the extracted feature map is output through the fully connected layer to obtain the texture features and color features of the target cargo stacking map. In the working process of CNN neural network, the loss function is the function used for training contained in the convolution kernel when the convolution layer slides and scans. The working process is convolution kernel sliding convolution. The loss function realizes data feature extraction by calculating the cross entropy loss of the data in the target cargo stacking map.

[0022] The multi-channel attention mechanism exists in the multi-channel attention module, which can enhance the detection ability of occluded areas. The attention mechanism is a technology that imitates human visual attention. By dynamically assigning weights, the model focuses on the key parts of the input data. The attention mechanism is included in the bilinear attention module. By analyzing the texture features, color features, and geometric structure features of the target cargo stacking map, the importance of different features can be evaluated and the key areas can be highlighted, that is, the key feature data can be focused by assigning weights to different data.

[0023] The following is a textual description of the CNN neural network framework diagram: Input the target cargo stacking map - the convolution layer performs sliding convolution to extract feature data - the bilinear attention module calculates the weight ratio of different features in the target cargo stacking map - the pooling layer downsamples to reduce the amount of calculation - repeated convolution + attention + pooling - the fully connected layer outputs features, and uses the cross entropy loss of the data in the target cargo stacking map through the loss function to realize data feature extraction.

[0024] When sliding convolution is performed in the convolution layer to extract feature data, the corresponding convolution formula is:

[0025] Among them, X is the input target cargo stacking map, i, j, k, v, u, c are constants, W represents the convolution kernel, b is the bias term, and Y is the output feature map.

[0026] The pooling layer performs downsampling to reduce the amount of calculation, that is, the corresponding formula for pooling is: Among them, R is the pooling window, S is the step size, and the other letters are the same as the corresponding letters in the convolution formula.

[0027] In the process of calculating the weight proportion of different features in the target cargo stacking map by the bilinear attention module, the weight calculation formula is:

[0028] Among them, GAP stands for global pooling, FC is the fully connected layer, Represents three different feature data. The feature data obtained in the CNN neural network needs to be element-by-element multiplied after the full connection layer outputs. The cross entropy loss of the data needs to be calculated in the process. The calculation formula is: in, is the true label, is the prediction probability. This formula can be used to extract data features. Element-by-element multiplication can also enhance the response of key areas, and finally different feature data are spliced ​​to achieve the purpose of highlighting key areas. Key areas are, for example, more dependent on depth geometry information in occluded areas, and other areas are more dependent on texture features and color features. The purpose of generating a spatial mask is to enhance the feature response to the occluded boundary, achieve the separation of goods, and facilitate logistics picking. The dilated convolution is to expand the contextual information of the occluded object, thereby strengthening the features of the occluded area and locating the coordinate position of the goods in the occluded area, because in 3D software, the coordinate positions of different positions can be located. Combining the occluded area and the visible area, the coordinate positions of all goods can be determined.

[0029] Furthermore, in a preferred embodiment of the present invention, the logistics picking robot with unqualified grasping stability is optimized for grasping, and the logistics picking robot after grasping optimization is controlled to grasp goods, specifically: In the dynamic simulation grabbing simulation software, a logistics picking simulation robot with unqualified stability is obtained, calibrated as a type of simulation robot, and a replanning algorithm is set in the type of simulation robot; Wherein, the replanning algorithm generates a slip risk in a type of simulated robot and stops the movement of the type of simulated robot when it is detected that the slip difference is greater than the maximum slip difference; After stopping the movement of the first type of simulated robot, the center of gravity position of the second type of qualified simulated goods is determined, and based on the center of gravity position of the second type of qualified simulated goods, the target grasping point of the goods is updated to obtain the optimized target grasping point of the goods; Control the first type of simulated robot to simulate the grasping of the second type of qualified simulated goods in combination with the optimized target grasping point of the goods. If the slip difference is still greater than the maximum slip difference, then determine all suitable grasping points of the second type of qualified simulated goods in the dynamic simulation grasping simulation software, and introduce the Pareto frontier analysis algorithm to generate a consumption curve that combines the slip difference and time corresponding to different grasping points of the goods, which is marked as the target analysis curve; Analyze the target analysis curve, select the corresponding cargo grabbing point with the smallest slip difference within the same time, and classify it as a qualified cargo grabbing point; The parameters of a type of simulation robot set in the dynamic simulation grasping simulation software are imported into the logistics picking robot to update the parameters, so as to obtain a type of logistics picking robot, and the type of logistics picking robot is controlled to grasp and process the goods in the goods stacking area in combination with the qualified grasping points of the goods.

[0030] It should be noted that the replanning algorithm is that the sensor detects abnormal grasping force or the cargo posture deviation exceeds the threshold, triggering the replanning process. For example, if the robot grasping pressure drops suddenly or the slip is too large, the movement is stopped immediately and the grasping point is recalculated to improve stability. After stopping the robot movement, it is necessary to replan the robot's grasping process, such as replanning the grasping point, because different grasping points have different displacement states during grasping. According to the real-time point cloud, the cargo posture is updated, and the grasping point closer to the center of gravity is preferred to improve stability. Therefore, the center of gravity position of the second type of qualified simulated cargo is calculated to obtain the optimized cargo target grasping point. If the optimized cargo target grasping point is used, the slip difference is still greater than the maximum slip difference, which proves that the grasping point is still wrong. It is necessary to introduce the Pareto front analysis algorithm to realize the construction of the slip difference and time consumption curve corresponding to different cargo grasping points. The optimal solution of the curve is the corresponding cargo grasping point with the smallest slip difference in the same time. Different cargoes have different corresponding cargo grasping points, so specific parameters of the robot need to be set. Finally, the parameters of a type of simulated robot set in the dynamic simulation grasping simulation software are imported into the logistics picking robot for parameter update.

[0031] Figure 2 A flow chart of a method for evaluating the grasping stability of a logistics picking robot at a target grasping point of a cargo is shown, comprising the following steps: S202: Generate a target grabbing point for the goods according to the coordinate position of the goods, and evaluate the grabbing stability of the logistics picking robot at the target grabbing point for the goods; S204: Perform dynamic simulated grasping simulation in combination with the second-class qualified goods and the logistics picking robot, and evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods based on the simulation results.

[0032] Furthermore, in a preferred embodiment of the present invention, the target grabbing point of the goods is generated according to the coordinate position of the goods, and the grabbing stability of the logistics picking robot at the target grabbing point of the goods is evaluated, specifically: The coordinate positions of different goods in the visible goods area and the obscured goods area are defined as the target coordinate positions of the goods, and a big data network is introduced. Based on the big data network, the graspable positions of all types of goods in the goods stacking area when using the logistics picking robot are retrieved and marked as the target grasping points of the goods. Obtain the rated grasping force value range of the logistics picking robot, and analyze the cargo target grasping points of all goods based on the big data network to determine the minimum force closing value index of different goods at the cargo target grasping points; Determine whether the minimum force closing value index of the target grasping point of the goods is not maintained within the rated grasping force value range of the logistics picking robot. If so, the corresponding goods are marked as Class I goods, and the remaining goods are marked as Class II qualified goods; If there are Class I goods, the grabbing force value range of the logistics picking robot is adjusted to control the grabbing force value range of the logistics picking robot to meet the minimum force closure value index of the goods at the target grabbing point of the goods, so that all goods are qualified Class II goods; Combining the second type of qualified goods and the logistics picking robot, dynamic simulation grasping simulation is carried out, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results.

[0033] It should be noted that goods are usually packed in containers such as boxes or bags. Containers of different sizes have different grasping points, and there are points on the container that can be grasped by the robot, that is, the target grasping point of the goods. In the big data network, the target grasping point of the goods can be obtained according to the specifications and types of the container. The logistics picking robot grabs the goods through equipment such as grabbing hooks. The equipment has an upper limit on the grasping force, that is, the force closure value. If the minimum force closure value indicator of the goods at the target grasping point of the goods is not maintained within the rated grasping force value range of the logistics picking robot, it proves that the goods cannot be grasped firmly. At this time, the robot needs to adjust the grasping force value to ensure that the goods can be grasped firmly and all the second-class qualified goods can be obtained. Then, dynamic simulation grasping simulation is started to evaluate the grasping stability of the logistics picking robot at the target grasping point of the goods.

[0034] Furthermore, in a preferred embodiment of the present invention, the combination of the two types of qualified goods and the logistics picking robot is used to perform dynamic simulation grasping simulation, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results, specifically: Introducing dynamic simulation grabbing simulation software, and inputting the current working parameters of the logistics picking robot into the dynamic simulation grabbing simulation software to obtain the logistics picking simulation robot, and at the same time importing the target cargo stacking map into the dynamic simulation grabbing simulation software to obtain the second category of qualified simulated cargo; Among them, the type and specification data of the second-class qualified simulated goods are set in the dynamic simulation grabbing simulation software, and the distance required for grabbing the second-class qualified simulated goods is obtained and calibrated as the standard grabbing distance; In the dynamic simulation grabbing simulation software, the logistics picking simulation robot simulates the grabbing of the second category of qualified simulated goods, and during the simulated grabbing process, the slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves within the standard grabbing distance is tested, and the maximum slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves is preset; The slip amount and maximum slip amount of the two types of qualified simulated goods when the logistics picking simulation robot moves are analyzed to obtain the slip amount difference, and the maximum slip amount difference is preset. If the slip amount difference is not greater than the maximum slip amount difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated to be qualified, and the target grasping point of the goods is classified as a qualified grasping point of the goods; If the slip difference is greater than the maximum slip difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated as unqualified.

[0035] It should be noted that, since there are some obstacles in the actual grasping test, the grasping test is carried out in a simulated way, and the grasping simulation can be carried out by introducing dynamic simulation grasping simulation software. The distance required for grasping of the second-class qualified simulated goods is obtained, that is, the distance from the storage point to the destination where the goods are grasped, which is calibrated as the standard grasping distance. In the standard grasping distance, if the robot grasps statically, it will not be unable to grasp firmly, but the goods may slip when moving. According to the difference in slip, it can be judged whether the goods are dangerous during the movement. If the difference in slip is greater than the maximum slip difference, the logistics picking robot is evaluated to have unqualified grasping stability at the target grasping point of the goods. At this time, the grasping cannot continue, and the robot needs to be optimized and updated to prevent the difference in slip from being greater than the maximum slip difference.

[0036] like Figure 3 As shown, the second aspect of the present invention further provides a logistics picking robot operation management system based on feedback analysis, the logistics picking robot operation management system includes a memory 31 and a processor 32, the memory 31 stores a logistics picking robot operation management method, and when the logistics picking robot operation management method is executed by the processor 32, the following steps are implemented: Acquire cargo images, depth information, and cargo point cloud data in the cargo stacking area, generate a point cloud map of the cargo stacking area, and perform filtering processing on the point cloud map of the cargo stacking area to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map; A multi-channel attention module is introduced into the target cargo stacking map to enhance the graphics of the occluded areas in the target cargo stacking map and identify the coordinate positions of the stacked cargo; Generate the cargo target grabbing point based on the cargo coordinate position, and evaluate the grabbing stability of the logistics picking robot at the cargo target grabbing point; The logistics picking robots with unqualified grasping stability are optimized for grasping, and the logistics picking robots after grasping optimization are controlled to grasp goods.

[0037] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A logistics picking robot operation management method based on feedback analysis, characterized in that: The following steps are involved: Acquire cargo images, depth information, and cargo point cloud data in the cargo stacking area, generate a point cloud map of the cargo stacking area, and perform filtering processing on the point cloud map of the cargo stacking area to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map; A multi-channel attention module is introduced into the target cargo stacking map to enhance the graphics of the occluded areas in the target cargo stacking map and identify the coordinate positions of the stacked cargo; Generate the cargo target grabbing point based on the cargo coordinate position, and evaluate the grabbing stability of the logistics picking robot at the cargo target grabbing point; The logistics picking robots with unqualified grasping stability are optimized for grasping, and the logistics picking robots after grasping optimization are controlled to grasp goods.

2. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The cargo image, depth information and cargo point cloud data are obtained in the cargo stacking area, a point cloud map of the cargo stacking area is generated, and the point cloud map of the cargo stacking area is filtered to achieve spatiotemporal alignment of map parameters to obtain a target cargo stacking map, specifically: Determine a cargo stacking area, wherein the cargo stacking area is an area where cargo that needs to be operated and managed by a logistics picking robot is stacked; Acquire an RGB-D camera, and acquire a scene color image in the cargo stacking area based on the RGB-D camera to obtain a cargo image in the cargo stacking area, calibrate it as a target cargo image, and acquire cargo depth information in the cargo stacking area based on the RGB-D camera to obtain cargo depth information in the cargo stacking area; The cargo depth information of the cargo stacking area is integrated into the target cargo image so that each pixel of the cargo contains the corresponding cargo depth information, and a three-dimensional modeling software is introduced, and the target cargo image integrated with the cargo depth information is imported into the three-dimensional modeling software to obtain cargo point cloud data of the target cargo image integrated with the cargo depth information; In the three-dimensional modeling software, based on the cargo point cloud data of the target cargo image integrated with the cargo depth information, a point cloud map of the cargo stacking area is constructed; In the three-dimensional modeling software, a Kalman filter algorithm is introduced, and based on the Kalman filter algorithm, parameter alignment of cargo point cloud data is performed on a cargo stacking area point cloud map; The method for aligning the parameters of the cargo point cloud data on the cargo stacking area point cloud map is to define the three-dimensional coordinate information of the cargo point cloud data, thereby constructing the speed parameters and space parameters when acquiring the target cargo image, and establishing the state transfer model based on the speed parameters and space parameters when acquiring the target cargo image; The generation time of the target cargo image is obtained, and combined with the state transfer model, the spatiotemporal synchronization of the cargo stacking area point cloud map is realized to obtain the spatiotemporal aligned cargo stacking area point cloud map, which is calibrated as the target cargo stacking map.

3. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The multi-channel attention module is introduced into the target cargo stacking map to perform graphic enhancement on the blocked areas in the target cargo stacking map and identify the coordinate positions of the stacked cargoes, specifically: The CNN neural network is introduced to extract RGB branch features of the target cargo stacking map to obtain the texture features and color features of the target cargo stacking map. At the same time, the depth branch extraction is performed on the target cargo stacking map to obtain the geometric structure features of the target cargo stacking map. A bilinear attention module is introduced, in which texture features, color features and geometric structure features of the target cargo stacking map are imported, and the weight proportions of different features in the target cargo stacking map are calculated by the bilinear attention module. At the same time, the bilinear attention module is used to generate a spatial mask in the target cargo stacking map to enhance the feature response speed of the cargo occlusion boundary in the target cargo stacking map; In the target cargo stacking map, the spatial mask and the weight proportions of different features are combined, and the target cargo stacking map is subjected to dilated convolution through the CNN neural network to obtain an enhanced target cargo stacking map; In the enhanced target cargo stacking map, locate the spatial mask position, output the visible cargo area and the obscured cargo area based on the spatial mask position, and determine the coordinate positions of different cargoes in the visible cargo area in the three-dimensional modeling software; For the obscured cargo area, a geometric completion network is introduced to perform depth inference on the obscured cargo area, construct the cargo outline in the obscured cargo area, and combine it with 3D modeling software to determine the coordinate positions of different cargoes in the obscured cargo area.

4. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The method generates a target grabbing point for the goods according to the coordinate position of the goods, and evaluates the grabbing stability of the logistics picking robot at the target grabbing point for the goods, specifically: The coordinate positions of different goods in the visible goods area and the obscured goods area are defined as the target coordinate positions of the goods, and a big data network is introduced. Based on the big data network, the graspable positions of all types of goods in the goods stacking area when using the logistics picking robot are retrieved and marked as the target grasping points of the goods. Obtain the rated grasping force value range of the logistics picking robot, and analyze the cargo target grasping points of all goods based on the big data network to determine the minimum force closing value index of different goods at the cargo target grasping points; Determine whether the minimum force closing value index of the target grasping point of the goods is not maintained within the rated grasping force value range of the logistics picking robot. If so, the corresponding goods are marked as Class I goods, and the remaining goods are marked as Class II qualified goods; If there are Class I goods, the grabbing force value range of the logistics picking robot is adjusted to control the grabbing force value range of the logistics picking robot to meet the minimum force closure value index of the goods at the target grabbing point of the goods, so that all goods are qualified Class II goods; Combining the second type of qualified goods and the logistics picking robot, dynamic simulation grasping simulation is carried out, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results.

5. The logistics picking robot operation management method based on feedback analysis according to claim 4 is characterized in that: The two types of qualified goods and the logistics picking robot are combined to perform dynamic simulation grasping simulation, and the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated based on the simulation results, specifically: Introducing dynamic simulation grabbing simulation software, and inputting the current working parameters of the logistics picking robot into the dynamic simulation grabbing simulation software to obtain the logistics picking simulation robot, and at the same time importing the target cargo stacking map into the dynamic simulation grabbing simulation software to obtain the second category of qualified simulated cargo; Among them, the type and specification data of the second-class qualified simulated goods are set in the dynamic simulation grabbing simulation software, and the distance required for grabbing the second-class qualified simulated goods is obtained and calibrated as the standard grabbing distance; In the dynamic simulation grabbing simulation software, the logistics picking simulation robot simulates the grabbing of the second category of qualified simulated goods, and during the simulated grabbing process, the slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves within the standard grabbing distance is tested, and the maximum slippage of the second category of qualified simulated goods when the logistics picking simulation robot moves is preset; The slip amount and maximum slip amount of the two types of qualified simulated goods when the logistics picking simulation robot moves are analyzed to obtain the slip amount difference, and the maximum slip amount difference is preset. If the slip amount difference is not greater than the maximum slip amount difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated to be qualified, and the target grasping point of the goods is classified as a qualified grasping point of the goods; If the slip difference is greater than the maximum slip difference, the grasping stability of the logistics picking robot at the target grasping point of the goods is evaluated as unqualified.

6. The logistics picking robot operation management method based on feedback analysis according to claim 1 is characterized in that: The method of performing grasping optimization on the logistics picking robot with unqualified grasping stability, and controlling the logistics picking robot after grasping optimization to grasp goods, is specifically as follows: In the dynamic simulation grabbing simulation software, a logistics picking simulation robot with unqualified stability is obtained, calibrated as a type of simulation robot, and a replanning algorithm is set in the type of simulation robot; Wherein, the replanning algorithm generates a slip risk in a type of simulated robot and stops the movement of the type of simulated robot when it is detected that the slip difference is greater than the maximum slip difference; After stopping the movement of the first type of simulated robot, the center of gravity position of the second type of qualified simulated goods is determined, and based on the center of gravity position of the second type of qualified simulated goods, the target grasping point of the goods is updated to obtain the optimized target grasping point of the goods; Control the first type of simulated robot to simulate the grasping of the second type of qualified simulated goods in combination with the optimized target grasping point of the goods. If the slip difference is still greater than the maximum slip difference, then determine all suitable grasping points of the second type of qualified simulated goods in the dynamic simulation grasping simulation software, and introduce the Pareto frontier analysis algorithm to generate a consumption curve that combines the slip difference and time corresponding to different grasping points of the goods, which is marked as the target analysis curve; Analyze the target analysis curve, select the corresponding cargo grabbing point with the smallest slip difference within the same time, and classify it as a qualified cargo grabbing point; The parameters of a type of simulation robot set in the dynamic simulation grasping simulation software are imported into the logistics picking robot to update the parameters, so as to obtain a type of logistics picking robot, and the type of logistics picking robot is controlled to grasp and process the goods in the goods stacking area in combination with the qualified grasping points of the goods.

7. Logistics picking robot operation management system based on feedback analysis, characterized in that: The logistics picking robot operation management system includes a memory and a processor, wherein the memory stores a logistics picking robot operation management method program. When the logistics picking robot operation management method program is executed by the processor, the logistics picking robot operation management method steps as described in any one of claims 1-6 are implemented.

Images