Robot grabbing detection method and device for various backgrounds

Abstract

The invention provides a robot grabbing detection method and device for various backgrounds, and relates to the technical field of robot intelligent control. The method comprises the following steps: acquiring a grabbed data set; preprocessing the grabbed data set to obtain a preprocessed grabbed data set; constructing a self-adaptive multi-background suppression grabbing network; according to the preprocessed capture data set, training an adaptive multi-background suppression capture network to obtain a trained adaptive multi-background suppression capture network; rGB-D image data of an object to be grabbed are obtained; rGB-D image data of an object to be grabbed are input into the trained self-adaptive multi-background suppression grabbing network, and a grabbing quality graph, a grabbing angle graph and a grabbing width graph are obtained; according to the grabbing quality graph, the grabbing angle graph and the grabbing width graph, grabbing pose data are estimated; and motion planning is carried out according to the grabbing pose data, so that the robot executes a grabbing task. According to the invention, the real-time performance and the accuracy of the grabbing operation of the robot system can be improved.

Description

Technical Field

[0001] The present invention relates to the field of robot intelligent control technology, and in particular to a robot grasping detection method and device for multiple backgrounds. Background Art

[0002] With the rapid development of robot technology, intelligence has become the core development direction of the new generation of robots. As a key link in the process of intelligence, robot grasping technology is crucial for mechanical processing, assembly and handling operations. As a non-contact means of environmental detection, visual perception stands out with high efficiency, stability and rich information. The visual system can provide key information on the shape, posture and size of the target object. In particular, the application of deep learning technology has greatly improved the robot's recognition accuracy of the target object and the ability to accurately detect the graspable position. For the complex working environment of single objects and multiple objects, the robot grasping detection network constructed by deep learning methods and the research of end-to-end grasping detection algorithms can promote the robot's autonomous learning ability and enhance its adaptability to unknown environments. Therefore, the grasping detection method based on deep learning is of great significance to the robot's detection and recognition capabilities in unstructured grasping environments and to promote the intelligent development of robot grasping technology.

[0003] Although grasping technology based on visual perception has made certain progress in this field, the existing technology still exposes many problems and shortcomings in unstructured environments with changing backgrounds. At present, most robot grasping technologies rely on preset environmental information and object models, which are incapable of doing so in complex and changing working environments, and seriously limit the robot's autonomous decision-making ability. Especially in terms of feature extraction, the existing technology is not enough to accurately extract key grasping features in environments with complex backgrounds. In unstructured scenarios with background interference, robots need to identify and focus on important grasping areas, but existing technologies often cannot effectively identify and utilize regional features. There are also several problems with the existing grasping network architecture. In addition, the existing technology often only stays at the linear operation level when processing features, and fails to make full use of feature information at different levels, resulting in redundancy in information fusion and failure to make substantial contributions to information gain. In background interference scenarios, due to factors such as illumination changes, object stacking and object diversity, the robustness and grasping accuracy of existing methods are low, and the extracted features may show large inconsistencies in scale and semantic levels, which seriously restricts the quality of the fusion value. Existing methods fail to fully utilize the complementary advantages of the rich detail information contained in low-level features and the powerful semantic information displayed by high-level features, resulting in insufficient adaptability of existing methods when dealing with target objects of varying sizes and complex shapes. The above problems not only limit the robot's autonomous grasping ability in complex environments, but also seriously affect the overall accuracy and reliability. They are technical problems that need to be focused on in future research. Summary of the invention

[0004] In order to solve the existing problems of grasping detection of unknown objects including single objects, cluttered stacked objects and transparent objects under different colors of grasping background interference including different lighting environments, and the technical problem of low grasping accuracy of existing grasping network technology, the embodiment of the present invention provides a robot grasping detection method and device for multiple backgrounds. The technical solution is as follows:

[0005] On the one hand, a robot grasping detection method for multiple backgrounds is provided, the method is implemented by a robot grasping detection device for multiple backgrounds, and the method includes:

[0006] S1, obtain the crawled data set;

[0007] S2, preprocessing the captured data set to obtain a preprocessed captured data set;

[0008] S3, build an adaptive multi-background suppression grasping network;

[0009] S4, training the adaptive multi-background suppression grabbing network according to the preprocessed grabbing data set to obtain a trained adaptive multi-background suppression grabbing network;

[0010] S5, obtaining RGB-D image data of the object to be grasped;

[0011] S6, inputting the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map, and a grasping width map;

[0012] S7. Estimate grasping posture data according to the grasping quality map, the grasping angle map, and the grasping width map; perform motion planning according to the grasping posture data to enable the robot to perform the grasping task.

[0013] On the other hand, a robot grasping detection device for multiple backgrounds is provided, and the device is applied to a robot grasping detection method for multiple backgrounds. The device includes:

[0014] A first acquisition unit, used to acquire a captured data set;

[0015] A preprocessing unit, used for preprocessing the captured data set to obtain a preprocessed captured data set;

[0016] Construction unit, used to build an adaptive multi-background suppression grasping network;

[0017] A training unit, used for training the adaptive multi-background suppression grabbing network according to the preprocessed grabbing data set to obtain a trained adaptive multi-background suppression grabbing network;

[0018] A second acquisition unit, used to acquire RGB-D image data of the object to be grasped;

[0019] A third acquisition unit is used to input the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map and a grasping width map;

[0020] The execution unit is used to estimate the grasping posture data according to the grasping quality map, the grasping angle map and the grasping width map; and perform motion planning according to the grasping posture data to enable the robot to perform the grasping task.

[0021] On the other hand, a robot grasping detection device for multiple backgrounds is provided, and the robot grasping detection device for multiple backgrounds comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned robot grasping detection methods for multiple backgrounds is implemented.

[0022] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned robot grasping detection methods for multiple backgrounds.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] The embodiment of the present invention first obtains a grasping data set; preprocesses the grasping data set; obtains the preprocessed grasping data set; secondly, constructs an adaptive multi-background suppression grasping network; trains the adaptive multi-background suppression grasping network according to the preprocessed grasping data set to obtain a trained adaptive multi-background suppression grasping network; obtains RGB-D image data of the object to be grasped; inputs the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map and a grasping width map; finally, estimates grasping posture data according to the grasping quality map, the grasping angle map and the grasping width map; performs motion planning according to the grasping posture data, so that the robot performs a grasping task.

[0025] The adaptive multi-background suppression grasping network proposed in the present invention can accurately identify the importance of each channel in the feature map, and accordingly adjust the contribution of each channel in the feature map by weight; the adaptive multi-background suppression grasping network focuses on the features that are critical to the grasping task, and can ignore irrelevant features, ensuring excellent grasping detection effects under a variety of different backgrounds. The adaptive multi-background suppression grasping network proposed in the present invention solves the gradient vanishing problem in the network training process, simplifies the learning process, accelerates the convergence speed of the network, and improves the generalization ability of the network to objects in new positions; the present invention can dynamically adjust the channel attention and spatial attention through the CBAM module set, which can enhance the adaptability and complementary advantages of feature extraction and effectively deal with complex and changeable target objects; the adaptive multi-background suppression grasping network can be applied to the grasping detection tasks of objects with different sizes, colors, shapes and surface textures; the present invention constructs a ROS-based upper robot system platform, which can monitor the robot's motion state in real time, and integrate grasping detection and motion planning. The integrated design can achieve high-level automation and intelligence, and can detect the robot's state in real time to ensure that the robot is in a safe state. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 It is a flow chart of a robot grasping detection method for multiple backgrounds provided by an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of the structure of an adaptive multi-background suppression capture network architecture provided by an embodiment of the present invention;

[0029] Figure 3 is a general structural diagram of a residual module provided by an embodiment of the present invention;

[0030] Figure 4 is a general structural diagram of a SEGCM module provided in an embodiment of the present invention;

[0031] Figure 5 is a general structural diagram of a CBAM module provided in an embodiment of the present invention;

[0032] Figure 6 It is a robot intelligent grasping system framework provided by an embodiment of the present invention;

[0033] Figure 7 It is the robot grasping process architecture provided by the embodiment of the present invention;

[0034] Figure 8 is a grasping detection result on the Cornell and Jacquard datasets provided by an embodiment of the present invention;

[0035] Fig. 9 is a grasping detection result in a single object scene provided by an embodiment of the present invention;

[0036] Fig.10 is a grasping detection result in a complex multi-object scene provided by an embodiment of the present invention;

[0037] Fig.11 The single object and multi-object grasping detection results in various backgrounds provided by the embodiments of the present invention;

[0038] Fig.12 It is a block diagram of a robot grasping detection device for multiple backgrounds provided by an embodiment of the present invention;

[0039] Fig.13 It is a structural schematic diagram of a robot grasping detection device for multiple backgrounds provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0041] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0042] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0043] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0044] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0045] The embodiment of the present invention provides a robot grasping detection method for various backgrounds, which can be implemented by a robot grasping detection device for various backgrounds, and the robot grasping detection device for various backgrounds can be a terminal or a server. Figure 1 The flowchart of the robot grasping detection method for multiple backgrounds is shown, and the processing flow of the method may include the following steps:

[0046] S1. Get the crawled dataset.

[0047] Among them, the captured data sets used in the embodiments of the present invention are the Cornell data set and the Jacquard data set; wherein, the Cornell data set contains 885 images, which are expanded by cropping, rotating and scaling operations on the Cornell data set; wherein, the Jacquard data set consists of 11,619 different scenes of 54,485 different objects, with a total of 4,967,454 captured annotations. For each scene, a rendered RGB image, a segmentation mask, two depth images and captured annotations are provided.

[0048] S2. Preprocess the captured data set to obtain a preprocessed captured data set.

[0049] Optionally, S2 preprocesses the captured data set, including:

[0050] The captured data set is cleaned, converted, and annotated.

[0051] In a feasible implementation manner, preprocessing the captured data set further includes: decompressing the data set, viewing and understanding the data set file. By preprocessing the captured data set, the quality and availability of the data can be ensured.

[0052] In a feasible implementation, the preprocessed captured data set is divided into a training set and a test set in a ratio of 1:9.

[0053] S3. Construct an adaptive multi-background suppression grasping network.

[0054] Among them, the adaptive multi-background suppression grasping network can automatically adjust its parameters and behaviors according to different environments and conditions to achieve optimal performance. The adaptive multi-background suppression grasping network can handle a variety of different background environments, not just a single or specific background. The adaptive multi-background suppression grasping network has the ability to suppress or reduce background interference, which can improve the accuracy of target recognition and grasping.

[0055] in, Figure 2 It is a structural schematic diagram of the adaptive multi-background suppression grasping network architecture provided by an embodiment of the present invention; the adaptive multi-background suppression grasping network takes a 224×224 RGBD image as input, performs feature extraction on the RGBD image through the adaptive multi-background suppression grasping network, predicts high-quality grasping postures for unknown objects, and outputs the final grasping quality map, grasping angle map, and grasping width map.

[0056] Among them, the value of each point on the grasping quality graph represents the grasping confidence at the grasping point, that is, the success probability of the grasping posture provides a basis for grasping decision.

[0057] Among them, each point value on the grasping angle diagram represents the angle that the robot's end effector needs to rotate at the grasping point; the grasping angle diagram can improve the accuracy of the grasping action.

[0058] Among them, the value of each point on the grasping width diagram represents the size that the robot's end effector needs to open at the grasping point; the grasping width diagram can ensure the adaptability and accuracy of the grasping action.

[0059] Optionally, the adaptive multi-background suppression grabbing network includes: a convolution layer, a deconvolution layer, 5 residual modules, a SEGCM module and a CBAM module.

[0060] Among them, three convolutional layers are used to perform deep feature extraction on RGBD images, which effectively extracts and encodes the preliminary features of the image. Through convolution operations, a feature map containing 128 channels is generated. Among them, adding normalization processing operations after each convolutional layer can optimize the training process and improve the convergence speed of the network.

[0061] In a feasible implementation, three deconvolution layers are used to enlarge the size of the input feature map, and the size of the feature map is gradually restored through upsampling operations, and finally enlarged to 224×224 pixels, the same as the original input image.

[0062] in, Figure 3 It is a general structural diagram of the residual module provided in an embodiment of the present invention.

[0063] Optionally, the residual module includes: two 3×3 convolution kernels and a Mish activation function;

[0064] The residual module is used to perform convolution operations on the input feature map, extract local features, and output a fused feature map.

[0065] In a feasible implementation, the residual module improves the accuracy and efficiency of the network's prediction of the grasping pose of unknown objects by solving the gradient vanishing problem, improving training efficiency, enhancing feature transfer, and building a deep network structure.

[0066] in, Figure 4 It is a general structural diagram of the SEGCM module provided in an embodiment of the present invention.

[0067] Optionally, the SEGCM module includes: 3 1×1 convolution kernels, 3×3 convolution kernels, a Mish activation function, a Sigmoid activation function, and a fully connected layer;

[0068] The SEGCM module is used to extract the relationship between channels, optimize the feature representation through global context information, and output key features.

[0069] In a feasible implementation, the SEGCM module learns the relationship between channels, accurately identifies and strengthens the key information in the feature map, ignores insignificant features, enables the network to more comprehensively extract image features, and adaptively generate high-quality grasping poses under changing grasping backgrounds.

[0070] in, Figure 5 It is a general structural diagram of the CBAM module provided in an embodiment of the present invention.

[0071] Optionally, the CBAM module includes: a channel attention module and a spatial attention module;

[0072] Channel attention module, used to perform pooling operation on input features to obtain attention-enhanced features;

[0073] The spatial attention module is used to perform pooling operations on the attention enhancement features to obtain spatial attention features.

[0074] In a feasible implementation, the CBAM module can improve the accuracy and efficiency of grasp detection and enhance the interpretability of the network. The CBAM module adaptively learns channel and spatial attention weights to improve the feature expression ability of the convolutional neural network. By combining channel attention and spatial attention, the CBAM module can capture the correlation between features in different dimensions, thereby improving the real-time performance and accuracy of grasp detection tasks.

[0075] Among them, the use of deconvolution layer for processing can retain the rich feature information extracted from the convolution layer, ensure the precise correspondence of features in the spatial dimension, and provide a high-resolution feature map for generating high-quality grasping pose prediction.

[0076] S4. According to the preprocessed grasping data set, the adaptive multi-background suppression grasping network is trained to obtain a trained adaptive multi-background suppression grasping network.

[0077] Optionally, the specific implementation process of S4 may include S41-S45:

[0078] S41, input the preprocessed capture data set into the adaptive multi-background suppression capture network, perform feature extraction through the convolution layer, and obtain preliminary features;

[0079] S42, inputting the preliminary features into the residual module, processing them through a 3×3 convolution kernel, and obtaining local features; according to the residual mechanism, processing the local features through normalization, Mish activation function, and a 3×3 convolution kernel, and outputting fused features; the fused features include the preliminary features of the original input and the features processed by the convolution layer;

[0080] Among them, each kernel in the 3×3 convolution kernel can cover a 3×3 pixel area of ​​the input feature map, which can effectively extract local features.

[0081] Among them, the normalization process normalizes the input of the layer so that the mean of the output is close to 0 and the variance is close to 1, thereby stabilizing the training process, accelerating the convergence of the network and improving the generalization ability of the model.

[0082] Among them, the Mish activation function is a non-monotonic activation function that can improve the nonlinear expression ability of the network and can be expressed by the following formula (1):

[0083] (1)

[0084] Among them, f(x) represents the result after being processed by the Mish activation function; x represents the input activation value.

[0085] Among them, after being processed by the Mish activation function, further processing with a 3×3 convolution kernel can further refine the processing features, and the output has the same dimension as the input feature.

[0086] In a feasible implementation, the residual module adopts a residual connection mechanism, which can alleviate the gradient vanishing problem in the network and increase the expressiveness of the features by performing two layers of convolution on the input feature map and directly adding the feature maps processed by the activation function.

[0087] S43, input the fused features into the SEGCM module, process them through two 1×1 convolution kernels, Mish activation function and Sigmoid activation function to obtain a compressed feature vector; according to the compressed feature vector, process it through a 3×3 convolution kernel, Mish activation function, Sigmoid activation function and a fully connected layer to obtain an attention weight map;

[0088] In a feasible implementation manner, the specific implementation process of the SEGCM module may include S431-S433:

[0089] S431, input the fused features into the SEGCM module, perform global average pooling on the input fused features, compress the spatial information of each channel into a single value, obtain a compressed feature vector, use two 1×1 convolution kernels to perform nonlinear transformation on the compressed feature vector, wherein the Mish activation function is used between the two convolution kernels to increase the ability to extract nonlinear features, and generate an attention weight for each channel through the Sigmoid activation function; multiply the generated attention weight for each channel by each channel of the original feature map to obtain a scaled feature map;

[0090] S432, branching the scaled feature map includes: a first branch and a second branch. The processing of the first branch may include:

[0091] (1) Perform global average pooling and global pooling operations on the scaled feature map to obtain feature vectors of global average pooling and global pooling;

[0092] (2) Processing is performed through a 1×1 convolution kernel and a Mish activation function to obtain the processing result;

[0093] (3) A 3×3 convolution kernel and a Sigmoid activation function are used to process the processing result and output the attention weight of the first branch; the Sigmoid activation function is used to ensure that the output value is between 0 and 1.

[0094] The processing of the second branch may include:

[0095] (1) Perform global average pooling on the scaled feature map to obtain the processing result after global average pooling;

[0096] (2) The processing results after global average pooling are processed through two fully connected layers to obtain the attention weight of the second branch; among them, the Mish activation function is used after the first fully connected layer; and the Sigmoid activation function is used after the second fully connected layer.

[0097] S433. Obtain an attention weight map according to the attention weight of the first branch and the attention weight of the second branch; apply the attention weight map to the original feature map before the first branch and the second branch to strengthen important features and suppress unimportant features.

[0098] Among them, through the SEGCM module, the network can not only extract the relationship between channels, but also use the global context information to optimize the hot evidence representation; the SEGCM module enables the network to accurately identify and focus on key features when dealing with grasping tasks in complex backgrounds, thereby improving the performance and generalization ability of the model.

[0099] S44, input the attention weight map into the CBAM module, process it through the channel attention module, and obtain the attention-weighted channel features; input the attention-weighted channel features into the spatial attention module to obtain the spatial attention features; generate the attention enhancement features according to the spatial attention features and the attention-weighted channel features;

[0100] In a feasible implementation manner, the specific implementation process of the CBAM module may include S441-S445:

[0101] S441, input the attention weight map into the channel attention module, perform a maximum pooling operation on each channel of the attention weight map, and generate a global maximum feature vector for each channel; perform an average pooling operation on each channel of the attention weight map, and generate an average feature vector for each channel;

[0102] S442, input the global maximum eigenvector of each channel and the average eigenvector of each channel into a 7×7 convolution kernel, learn the attention weight of each channel, process it through the Sigmoid activation function, and obtain the channel attention weight;

[0103] S443, multiplying the attention weight by each channel of the original feature map to obtain an attention-weighted channel feature map;

[0104] S444, the channel feature map after attention weighting is input into the spatial attention module, and the global maximum pooling and average pooling operations are performed on the channel feature map after attention weighting to obtain two 56×56×1 feature maps, and the two feature maps are spliced ​​in the channel dimension, and the dimension reduction is performed through a fully connected layer to obtain a 56×56×1 feature map, and the spatial attention feature is generated through the Sigmoid activation function; the spatial attention feature is multiplied with the input feature map to obtain the final spatial attention feature;

[0105] S445. Multiply the attention-weighted channel feature map and the final spatial attention feature element by element to obtain an attention-enhanced feature; wherein the attention-enhanced feature can retain key information and suppress noise and irrelevant information.

[0106] S45. Input the attention enhancement features into the deconvolution layer for processing to obtain a grasping quality map, a grasping angle map, and a grasping width map; train the adaptive multi-background suppression grasping network according to the grasping quality map, the grasping angle map, and the grasping width map to obtain a trained adaptive multi-background suppression grasping network.

[0107] S5. Obtain RGB-D image data of the object to be grasped.

[0108] S6. Input the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map, and a grasping width map.

[0109] Among them, the adaptive multi-background suppression grasping network can adapt to a variety of grasping backgrounds and make high-quality grasping posture predictions for unknown objects. While maintaining excellent performance, it does not increase the number of network parameters. This enables the adaptive multi-background suppression grasping network to achieve fast reasoning and high-precision grasping in a variety of backgrounds, achieving an ideal balance between efficient performance and precise grasping.

[0110] S7. Estimate grasping posture data according to the grasping quality map, the grasping angle map, and the grasping width map; perform motion planning according to the grasping posture data so that the robot performs the grasping task.

[0111] In a feasible implementation, the grasping posture data is estimated based on the grasping quality map, grasping angle map and grasping width map; wherein the grasping quality map represents the grasping confidence at each pixel point, the grasping confidence represents the grasping success rate at each pixel point, and the grasping angle map is the angle that the robot end effector needs to rotate; assuming that the angle to be rotated is a, the cosa value, sina value and tana value are calculated, and the rotation angle is obtained through arctana; the grasping width map is the size that the robot gripper needs to open, and the optimal grasping posture data is obtained by estimating the location of the optimal grasping confidence pixel point and the corresponding grasping angle and grasping width. Among them, the grasping posture in the camera coordinate system is converted into the grasping posture of the robot based on the coordinate system.

[0112] In a feasible implementation, during the robot's grasping task, the RGBD camera installed on the robot's end effector is responsible for analyzing the RGB-D image to detect the object and generate an accurate grasping posture; the posture information is initially defined in the camera coordinate system, and the posture information is unknown to the robot. When the camera completes the recognition and positioning of the object position, and converts this posture information from the camera coordinate system to the robot's base coordinate system, the robot can accurately know the position of the target object and perform the final grasping action. In order to achieve this conversion, the embodiment of the present invention uses the handeye-calib package to calibrate the hand-eye system. Through the calibration process, the conversion matrix from the camera coordinate system to the robot's base coordinate system can be obtained, so that the robot can understand the posture information provided by the camera and apply it to the actual grasping task. Through hand-eye calibration, it is ensured that the robot can accurately perform complex operations based on the data provided by the visual system, thereby improving the accuracy and reliability of the robot's operation.

[0113] in, Figure 6 It is a robot intelligent grasping system framework provided by an embodiment of the present invention; in a feasible implementation manner, an RGBD camera collects image data of a grasping scene; the image is preprocessed to obtain preprocessed image data; the preprocessed image data is input into a trained adaptive multi-background suppression based grasping network to generate a grasping quality image, a grasping angle image, and a grasping width image; grasping posture data is generated according to the grasping quality image, the grasping angle image, and the grasping width image, and inverse kinematics planning is performed through a ROS interface to obtain the robot's grasping path, and the robot performs a grasping action according to the grasping path.

[0114] in, Figure 7It is the robot grasping process architecture provided by the embodiment of the present invention; in a feasible implementation mode, the embodiment of the present invention builds a robot host computer grasping system platform, which is composed of a host computer, a Baxter robot and an RGBD camera. The Baxter robot adopts an open source robot operating system based on ROS and runs on a Linux platform. Users can read information or send instructions by interconnecting with the internal computer of the robot through the network; the host computer control system adopts the Ubuntu20.04 system, and the computer control platform and the physical robot are connected to the same IP through USB and router. The entire robot interaction system framework is based on the robot operating system ROS noetic version. The entire algorithm is programmed in Python. ROS uses independently running functional nodes, distributed processing subtasks, and completes information transmission between nodes through a variety of communication mechanisms, thereby improving algorithm reusability and writing efficiency. The entire robot intelligent grasping system consists of an adaptive multi-background suppression grasping network grasping posture detection module and a control grasping and inverse kinematics planning module.

[0115] Among them, the adaptive multi-background suppression grasping network grasping posture detection module is used to collect visual information of the grasping scene by using the RGBD camera, and can obtain the RGB-D image data of the grasped object; the adaptive multi-background suppression grasping network grasping posture detection module can effectively align the missing parts in the depth map to ensure the integrity of the data. After preprocessing the input image, the preprocessed image is input into the adaptive multi-background suppression grasping network to generate high-quality grasping posture. Among them, the control grasping and inverse kinematics planning module is used to convert the posture coordinates to the robot base coordinate system after the RGBD camera successfully obtains the grasping posture, perform inverse kinematics planning through the ROS interface, obtain the robot's grasping path, and the robot performs the grasping action according to the grasping path.

[0116] In a feasible implementation manner, the present invention provides an implementation case specifically including: first, the robot is connected to the host computer via USB or Ethernet, the RGBD camera is connected to the host computer via USB, and then the robot driver is started to establish communication between the host computer and the Baxter robot body. Once the host computer system successfully establishes a connection with the robot, the Baxter robot can be enabled, and after successful enabling, the grasping routine can be executed. While executing the grasping task in real time, the host computer can also obtain the joint status and the position information of the robot end effector in real time through ROS, which is crucial for monitoring the safety status of the robot.

[0117] In a feasible implementation manner, in order to verify the effectiveness of the VecGNet network and its related modules in the present invention, the following steps are taken: the VecGNet network designed by the present invention is trained on the Cornell dataset and the Jacquard dataset respectively. After the training is completed, the performance of the VecGNet network is visualized and analyzed on these two datasets to intuitively display its learning results. In addition, in order to further evaluate the performance of the network in practical applications, a robot grasping experiment was carried out in a real-world environment, which not only confirmed the theoretical superiority of the VecGNet network, but also demonstrated its feasibility and effectiveness in practical applications. A total of four groups of experiments were carried out to verify the performance of the present invention: grasping visualization on the dataset, real-world single-object grasping, real-world multi-object grasping, and grasping under multiple backgrounds in the real world.

[0118] (1) Crawl on the Cornell and Jacquard datasets: Figure 8 Shown are visual crawls on the Cornell and Jacquard datasets.

[0119] (2) Grasping in single-object scenes in the real world: In the single-object scene grasping experiment in the real world, a series of objects commonly seen in daily life were selected. These objects not only have different sizes and complex shapes, but also have rich texture information. Single-object grasping experiments were conducted on these objects to test the actual effectiveness of the present invention. Among the 500 grasping attempts, only 11 failed, and the grasping success rate was as high as 98%, which fully demonstrated the excellent performance of the present invention in handling single-object grasping scenarios. Fig. 9 As shown, a successful case of grasping and detecting a single object in a real-world single-object scene is demonstrated.

[0120] (3) Grasping in real-world multi-object scenes: 50 sets of scenes were designed in complex multi-object environments in the real world, each containing multiple objects of different sizes, shapes, and colors. The objects in the scenes vary in size, color, shape, and surface texture to ensure the diversity and challenge of the test environment. Fig.10 As shown in the figure, the above scenario reflects the complexity of grasping multiple objects in the real world. After testing, the average grasping success rate of the present invention in complex scenarios reached 95%, which not only confirmed the high performance of the proposed grasping detection network, but also demonstrated its excellent processing ability for multi-object grasping tasks in complex grasping environments.

[0121] (4) Grasping under different grasping backgrounds in the real world: Under backgrounds of different colors and different lighting conditions, the present invention has conducted extensive detection experiments, especially for single-object and multi-object scenes, including transparent objects. In the experiment, common challenges in the real world, such as backgrounds of different colors, dark light and strong light environments, were simulated. The system was tested in 30 sets of carefully designed complex scenes. The objects in each set of scenes differed in size, color, shape and surface texture to ensure the diversity and challenge of the test environment. After testing, the average grasping success rate of the present invention in complex scenes reached 94%, and the detection system of the present invention demonstrated excellent performance. In summary, the detection system shows efficient and accurate performance in the detection of single objects, multiple objects and transparent objects under different color backgrounds including dark light and strong light conditions, which fully demonstrates its applicability and robustness in different background environments. Fig.11 Shown are single- and multi-object grasping against a variety of backgrounds.

[0122] The embodiment of the present invention first obtains a grasping data set; preprocesses the grasping data set; obtains the preprocessed grasping data set; secondly, constructs an adaptive multi-background suppression grasping network; trains the adaptive multi-background suppression grasping network according to the preprocessed grasping data set to obtain a trained adaptive multi-background suppression grasping network; obtains RGB-D image data of the object to be grasped; inputs the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map and a grasping width map; finally, estimates grasping posture data according to the grasping quality map, the grasping angle map and the grasping width map; performs motion planning according to the grasping posture data, so that the robot performs a grasping task.

[0123] The adaptive multi-background suppression grasping network proposed in the present invention can accurately identify the importance of each channel in the feature map, and accordingly adjust the contribution of each channel in the feature map by weight; the adaptive multi-background suppression grasping network focuses on the features that are critical to the grasping task, and can ignore irrelevant features, ensuring excellent grasping detection effects under a variety of different backgrounds. The adaptive multi-background suppression grasping network proposed in the present invention solves the gradient vanishing problem in the network training process, simplifies the learning process, accelerates the convergence speed of the network, and improves the generalization ability of the network to objects in new positions; the present invention can dynamically adjust the channel attention and spatial attention through the CBAM module set, which can enhance the adaptability and complementary advantages of feature extraction and effectively deal with complex and changeable target objects; the adaptive multi-background suppression grasping network can be applied to the grasping detection tasks of objects with different sizes, colors, shapes and surface textures; the present invention constructs a ROS-based upper robot system platform, which can monitor the robot's motion state in real time, and integrate grasping detection and motion planning. The integrated design can achieve high-level automation and intelligence, and can detect the robot's state in real time to ensure that the robot is in a safe state.

[0124] Fig.12 is a block diagram of a robot grasping detection device for multiple backgrounds according to an exemplary embodiment, and the device is used for a robot grasping detection method for multiple backgrounds. Fig.12 The device includes a first acquisition unit 310, a preprocessing unit 320, a construction unit 330, a training unit 340, a second acquisition unit 350, a third acquisition unit 360 and an execution unit 370. Wherein:

[0125] A first acquisition unit 310 is used to acquire a captured data set;

[0126] A preprocessing unit 320, configured to preprocess the captured data set to obtain a preprocessed captured data set;

[0127] A construction unit 330, used to construct an adaptive multi-background suppression capture network;

[0128] A training unit 340 is used to train the adaptive multi-background suppression grabbing network according to the preprocessed grabbing data set to obtain a trained adaptive multi-background suppression grabbing network;

[0129] A second acquisition unit 350 is used to acquire RGB-D image data of the object to be grasped;

[0130] The third acquisition unit 360 is used to input the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map and a grasping width map;

[0131] The execution unit 370 is used to estimate the grasping posture data according to the grasping quality map, the grasping angle map and the grasping width map; and perform motion planning according to the grasping posture data to enable the robot to perform the grasping task.

[0132] Optionally, the preprocessing unit 320 is used to:

[0133] The captured data set is cleaned, converted and annotated.

[0134] Optionally, the adaptive multi-background suppression capture network includes: a convolution layer, a deconvolution layer, 5 residual modules, a SEGCM module and a CBAM module.

[0135] Optionally, the residual module includes: two 3×3 convolution kernels and a Mish activation function;

[0136] The residual module is used to perform a convolution operation on the input feature map, extract local features, and output a fused feature map;

[0137] Optionally, the SEGCM module includes: 3 1×1 convolution kernels, a 3×3 convolution kernel, a Mish activation function, a Sigmoid activation function and a fully connected layer;

[0138] The SEGCM module is used to extract the relationship between channels, optimize the feature representation through global context information, and output key features.

[0139] Optionally, the CBAM module includes: a channel attention module and a spatial attention module;

[0140] The channel attention module is used to perform a pooling operation on the input features to obtain attention-enhanced features;

[0141] The spatial attention module is used to perform pooling operations on the attention enhancement features to obtain spatial attention features.

[0142] Optionally, the training unit 340 is used to:

[0143] Input the preprocessed capture data set into an adaptive multi-background suppression capture network, perform feature extraction through a convolutional layer, and obtain preliminary features;

[0144] The preliminary features are input into the residual module and processed by a 3×3 convolution kernel to obtain local features. According to the residual mechanism, the local features are processed by normalization, Mish activation function and a 3×3 convolution kernel to output fused features. The fused features include the preliminary features of the original input and the features processed by the convolution layer.

[0145] The fused features are input into the SEGCM module, and processed by two 1×1 convolution kernels, Mish activation function and Sigmoid activation function to obtain a compressed feature vector. According to the compressed feature vector, it is processed by 3×3 convolution kernel, Mish activation function, Sigmoid activation function and fully connected layer to obtain the attention weight map.

[0146] The attention weight map is input into the CBAM module and processed through the channel attention module to obtain the attention-weighted channel features; the attention-weighted channel features are input into the spatial attention module to obtain the spatial attention features; the attention enhancement features are generated according to the spatial attention features and the attention-weighted channel features; the attention enhancement features are input into the deconvolution layer for processing to obtain the grasping quality map, the grasping angle map and the grasping width map; the adaptive multi-background suppression grasping network is trained according to the grasping quality map, the grasping angle map and the grasping width map to obtain the trained adaptive multi-background suppression grasping network.

[0147] The embodiment of the present invention first obtains a grasping data set; preprocesses the grasping data set; obtains the preprocessed grasping data set; secondly, constructs an adaptive multi-background suppression grasping network; trains the adaptive multi-background suppression grasping network according to the preprocessed grasping data set to obtain a trained adaptive multi-background suppression grasping network; obtains RGB-D image data of the object to be grasped; inputs the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map and a grasping width map; finally, estimates grasping posture data according to the grasping quality map, the grasping angle map and the grasping width map; performs motion planning according to the grasping posture data, so that the robot performs a grasping task.

[0148] The adaptive multi-background suppression grasping network proposed in the present invention can accurately identify the importance of each channel in the feature map, and accordingly adjust the contribution of each channel in the feature map by weight; the adaptive multi-background suppression grasping network focuses on the features that are critical to the grasping task, and can ignore irrelevant features, ensuring excellent grasping detection effects under a variety of different backgrounds. The adaptive multi-background suppression grasping network proposed in the present invention solves the gradient vanishing problem in the network training process, simplifies the learning process, accelerates the convergence speed of the network, and improves the generalization ability of the network to objects in new positions; the present invention can dynamically adjust the channel attention and spatial attention through the CBAM module set, which can enhance the adaptability and complementary advantages of feature extraction and effectively deal with complex and changeable target objects; the adaptive multi-background suppression grasping network can be applied to the grasping detection tasks of objects with different sizes, colors, shapes and surface textures; the present invention constructs a ROS-based upper robot system platform, which can monitor the robot's motion state in real time, and integrate grasping detection and motion planning. The integrated design can achieve high-level automation and intelligence, and can detect the robot's state in real time to ensure that the robot is in a safe state.

[0149] Fig.13 is a schematic diagram of a robot grasping detection device for various backgrounds provided by an embodiment of the present invention, such as Fig.13 As shown, the robot grasping detection device for various backgrounds may include the above Fig.12 The robot grasping detection device for multiple backgrounds is shown. Optionally, the robot grasping detection device 410 for multiple backgrounds may include a first processor 2001 .

[0150] Optionally, the robot grasping detection device 410 for various backgrounds may further include a memory 2002 and a transceiver 2003 .

[0151] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0152] Combine the following Fig.13 The components of the robot grasping detection device 410 for various backgrounds are specifically introduced:

[0153] The first processor 2001 is the control center of the robot grasping detection device 410 for various backgrounds, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0154] Optionally, the first processor 2001 can perform various functions of the robot grasping detection device 410 for various backgrounds by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0155] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Fig.13 CPU0 and CPU1 are shown in FIG.

[0156] In a specific implementation, as an embodiment, the robot grasping detection device 410 for various backgrounds may also include multiple processors, such as Fig.13 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0157] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0158] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through an interface circuit ( Fig.13 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0159] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0160] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Fig.13 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0161] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the robot grasping detection device 410 in various backgrounds through an interface circuit ( Fig.13 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0162] It should be noted that Fig.13 The structure of the robot grasping detection device 410 for various backgrounds shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0163] In addition, the technical effects of the robot grasping detection device 410 for various backgrounds can refer to the technical effects of the robot grasping detection method for various backgrounds described in the above method embodiment, which will not be repeated here.

[0164] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0165] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0166] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0167] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0168] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0169] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0170] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0171] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0172] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0175] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0176] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A robot grasping detection method for multiple backgrounds, characterized in that: The method comprises: S1, obtain the crawled data set; S2, preprocessing the captured data set to obtain a preprocessed captured data set; S3, build an adaptive multi-background suppression grasping network; S4, training the adaptive multi-background suppression grabbing network according to the preprocessed grabbing data set to obtain a trained adaptive multi-background suppression grabbing network; S5, obtaining RGB-D image data of the object to be grasped; S6, inputting the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map, and a grasping width map; S7. Estimate grasping posture data according to the grasping quality map, the grasping angle map, and the grasping width map; perform motion planning according to the grasping posture data to enable the robot to perform the grasping task.

2. The robot grasping detection method for multiple backgrounds according to claim 1 is characterized in that: The S2 preprocesses the captured data set, including: The captured data set is cleaned, converted and annotated.

3. The robot grasping detection method for multiple backgrounds according to claim 1 is characterized in that: The adaptive multi-background suppression capture network includes: a convolution layer, a deconvolution layer, 5 residual modules, a SEGCM module and a CBAM module.

4. The robot grasping detection method for multiple backgrounds according to claim 3 is characterized in that: The residual module includes: two 3×3 convolution kernels and a Mish activation function; The residual module is used to perform a convolution operation on the input feature map, extract local features, and output a fused feature map.

5. The robot grasping detection method for multiple backgrounds according to claim 3 is characterized in that: The SEGCM module includes: 3 1×1 convolution kernels, 3×3 convolution kernels, Mish activation function, Sigmoid activation function and a fully connected layer; The SEGCM module is used to extract the relationship between channels, optimize the feature representation through global context information, and output key features.

6. The robot grasping detection method for multiple backgrounds according to claim 3 is characterized in that: The CBAM module includes: a channel attention module and a spatial attention module; The channel attention module is used to perform a pooling operation on the input features to obtain attention-enhanced features; The spatial attention module is used to perform pooling operations on the attention enhancement features to obtain spatial attention features.

7. The robot grasping detection method for multiple backgrounds according to claim 3 is characterized in that: The step S4 of training the adaptive multi-background suppression grabbing network according to the preprocessed grabbing data set to obtain a trained adaptive multi-background suppression grabbing network includes: S41, inputting the preprocessed capture data set into an adaptive multi-background suppression capture network, performing feature extraction through a convolutional layer, and obtaining preliminary features; S42, inputting the preliminary features into the residual module, processing them through a 3×3 convolution kernel, and obtaining local features; according to the residual mechanism, processing the local features through normalization, Mish activation function, and a 3×3 convolution kernel, and outputting fused features; the fused features include the preliminary features of the original input and the features processed by the convolution layer; S43, input the fused features into the SEGCM module, process them through two 1×1 convolution kernels, Mish activation function and Sigmoid activation function to obtain a compressed feature vector; according to the compressed feature vector, process it through a 3×3 convolution kernel, Mish activation function, Sigmoid activation function and a fully connected layer to obtain an attention weight map; S44, input the attention weight map into the CBAM module, process it through the channel attention module, and obtain the attention-weighted channel features; input the attention-weighted channel features into the spatial attention module to obtain the spatial attention features; generate the attention enhancement features according to the spatial attention features and the attention-weighted channel features; S45. Input the attention enhancement features into the deconvolution layer for processing to obtain a grasping quality map, a grasping angle map, and a grasping width map; train the adaptive multi-background suppression grasping network according to the grasping quality map, the grasping angle map, and the grasping width map to obtain a trained adaptive multi-background suppression grasping network.

8. A robot grasping detection device for multiple backgrounds, the robot grasping detection device for multiple backgrounds is used to implement the robot grasping detection method for multiple backgrounds as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A first acquisition unit, used to acquire a captured data set; A preprocessing unit, used for preprocessing the captured data set to obtain a preprocessed captured data set; Construction unit, used to build an adaptive multi-background suppression grasping network; A training unit, used for training the adaptive multi-background suppression grabbing network according to the preprocessed grabbing data set to obtain a trained adaptive multi-background suppression grabbing network; A second acquisition unit, used to acquire RGB-D image data of the object to be grasped; A third acquisition unit is used to input the RGB-D image data of the object to be grasped into the trained adaptive multi-background suppression grasping network to obtain a grasping quality map, a grasping angle map and a grasping width map; The execution unit is used to estimate the grasping posture data according to the grasping quality map, the grasping angle map and the grasping width map; and perform motion planning according to the grasping posture data to enable the robot to perform the grasping task.

9. A robot grasping detection device for multiple backgrounds, characterized in that: The robot grasping detection device for various backgrounds includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images