Robot grasping method and system based on multifunctional grasping and object-independent framework

Abstract

The present invention provides a robot grasping method and system based on a multifunctional grasping and object-independent framework, and relates to the field of robot intelligent control. The method includes: preprocessing RGB image information in a training set to obtain processed RGB image information; building a grasping detection model based on the processed RGB image information and depth information; obtaining a loss calculation strategy based on IOU calculation; training the grasping detection model based on the loss calculation strategy to obtain a grasping detection model with optimal weight parameters; a controller establishes a conversion relationship between a grasping posture configuration in a camera coordinate system and a world coordinate system according to the grasping detection model with optimal weight parameters, and obtains a grasping posture in a three-dimensional world coordinate system; the robot transports the object to a specified target position according to the grasping posture in the three-dimensional world coordinate system, and the present invention improves the robot's ability to grasp objects.

Description

Technical Field

[0001] The present invention relates to the field of robot intelligent control, and in particular to a robot grasping method and system based on a multifunctional grasping and object-independent framework. Background Art

[0002] In the field of robotics research, robot intelligent grasping has always been one of the hot research directions and can be widely used in industrial manufacturing, logistics and transportation, home services and other fields. Traditional robot grasping mainly relies on researchers to design grasping features for different grasping scenarios, which is not only time-consuming but also requires researchers to have certain prior knowledge. Researchers plan the robot's motion trajectory in advance and obtain the grasping posture of the object through mathematical methods. This method is very time-consuming and inefficient to grasp objects. With the introduction of visual technology and the rapid development of artificial intelligence, people have put forward higher requirements for the grasping ability of robots. Modern grasping scenes are no longer limited to structured environments, but are closer to real life. Objects are placed in a disorderly manner, and even multiple objects are stacked. In this context, deep learning technology, especially the breakthrough of convolutional neural networks in the field of target detection, has brought new development opportunities for robot grasping technology. Domestic and foreign researchers widely use detection algorithms based on deep learning in robot grasping research. Through a large amount of data to train the network model, the network outputs the highest quality grasping posture, and the robot obtains the grasping posture to perform the grasping task. This method enables robots to achieve good grasping effects even in unstructured scenes. However, real-life grasping environments are often more complex, such as garbage sorting and parts sorting in large garbage dumps. The objects in these scenes are stacked and of various shapes, which poses an extremely difficult challenge to robot grasping.

[0003] The existing technology mainly has the following problems:

[0004] 1) The traditional IOU calculation strategy regards the central area of ​​the grasping rectangle and the areas at both ends of the gripper as equally important. Under this calculation strategy, the grasping posture predicted by the network will cause the robot's gripper to be unstable during grasping, and even cause a collision between the object and the gripper.

[0005] 2) IOU has limitations as a loss function in the case of non-overlapping grasping boxes. When the grasping box predicted by the network does not overlap with the true value, the IOU indicator will not be able to accurately measure the relative position relationship between them and will directly give a zero value. The gradient of the loss function with respect to the network parameters will also be 0. Ultimately, the network cannot update its weights through gradient backpropagation to optimize the output of the predicted grasping box.

[0006] 3) The existing grasp detection network lacks the perception of global information. When using convolution operations to extract features, the position information of the input data is lost, and the features cannot be globally modeled. Summary of the invention

[0007] In order to solve the technical problems existing in the prior art, the embodiment of the present invention provides a robot grasping method and system based on a multifunctional grasping and object-independent framework, which has strong adaptability to complex environments and different objects, and can maintain stable grasping performance in the presence of noise and interference. The technical solution is as follows:

[0008] A robot grasping method based on multifunctional grasping and object-independent framework, comprising:

[0009] S1, preprocessing the RGB image information in the training set to obtain processed RGB image information;

[0010] S2. Building a grasping detection model based on the processed RGB image information and depth information;

[0011] S3. Obtain loss calculation strategy based on IOU calculation;

[0012] S4. Based on the loss calculation strategy, the grasping detection model is trained to obtain a grasping detection model with optimal weight parameters;

[0013] S5. The controller establishes a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, and obtains the grasping posture in the three-dimensional world coordinate system;

[0014] S6. The robot transports the object to the specified target location according to the grasping posture in the three-dimensional world coordinate system.

[0015] Optionally, the step S1 of preprocessing the RGB image information in the training set to obtain the processed RGB image information includes:

[0016] S101, after scaling and rotating the RGB image information, randomly shift the RGB image information in the horizontal direction and the vertical direction to obtain the randomly shifted RGB image information;

[0017] S102, cutting an image block of a preset size based on the center of the randomly shifted RGB image information;

[0018] S103: normalize the image block and the corresponding label to obtain processed RGB image information.

[0019] Optionally, the step S2 of building a grasping detection model according to the processed RGB image information and depth information includes:

[0020] S201, the processed RGB image information and depth information are sequentially passed through a convolution layer and a normalization layer, and then subjected to nonlinear processing to obtain input features;

[0021] S202, the global information concatenation and local aggregation module obtains globally important key features according to the input features;

[0022] S203, based on the convolutional feedforward skip sub-network, converting the globally important key features into output features in the form of a two-dimensional feature image;

[0023] S204, the multi-dimensional multi-scale attention and adaptive feature fusion module processes the output features to enhance the attention of the features associated with grasping detection in the output features, and obtains a grasping detection model.

[0024] Optionally, the local information concatenation and local aggregation module of S202 obtains globally important key features according to the input features, including:

[0025] S2021, performing window segmentation on the input feature, and calculating the dimension of each window, where the dimension of the input feature is d;

[0026] S2022, each window independently performs attention calculation to obtain the query Q, key K and value matrix V corresponding to each window;

[0027] S2023. Calculate the dot product of the query Q, key K, and value matrix V corresponding to each window respectively, and divide the dot product of the query Q, key K, and value matrix V by the scaling factor to obtain the attention score of each head;

[0028] S2024. Convert the attention score of each head into a corresponding attention weight based on a softmax function, and perform weighted summation on the value matrix V based on the attention weight to obtain the output of each head;

[0029] S2025. Concatenate the output of each head along the last input dimension to obtain a tensor;

[0030] S2026. Transform the tensor into globally important key features through a linear layer, wherein the dimension of the globally important key features is equal to that of the input features.

[0031] Optionally, the loss calculation strategy obtained based on IOU calculation in S3 includes:

[0032] S301, based on the principle that when the distance between the predicted grabbing rectangle and the true value rectangular box increases, the area of ​​the region where the predicted grabbing box and the true value grabbing box intersect is 0, and the value of the loss function decreases accordingly, the loss function IOU is improved to obtain an improved loss function;

[0033] The calculation formula of the improved loss function is formula (1):

[0034]

[0035] a(A,B) is the minimum enclosing rectangular area of ​​the true value grab box and the predicted grab box, A is the minimum enclosing rectangular area of ​​the true value grab box, and B is the minimum enclosing rectangular area of ​​the predicted grab box;

[0036] η(A,B) is the union area of ​​the true value grasp box and the predicted grasp box;

[0037] EIOU is the improved loss function;

[0038] S302, define an EIOU loss function according to EIOU, wherein the formula of the EIOU loss function is formula (2):

[0039]

[0040] is the EIOU loss function and is the loss calculation strategy.

[0041] Optionally, defining an EIOU loss function according to EIOU in S302 includes:

[0042] Calculate the EIOU value when the true value grab box A and the predicted grab box B do not intersect, including formula (3):

[0043]

[0044] Where M = the area of ​​the minimum circumscribed rectangle containing the true value grab box A and the predicted grab box B - the union area of ​​the true value grab box A and the predicted grab box B;

[0045] Calculate the EIOU value when the true value grab box A and the predicted grab box B intersect, including formula (4):

[0046]

[0047] Where Q = the intersection area of ​​the true value grab box A and the predicted grab box B;

[0048] According to formula (3) and formula (4), the EIOU loss function is obtained.

[0049] Optionally, the step of training the grasping detection model based on the loss calculation strategy in S4 to obtain the grasping detection model with the optimal weight parameters includes:

[0050] S401. Based on the loss calculation strategy, a loss function of the grasping detection model is obtained, wherein the calculation formula of the total loss function of the grasping detection model is formula (5):

[0051]

[0052] in, is the total loss function of the grasp detection model; is the grasp confidence loss function, is the sum of the grasping angle loss functions, is the crawl width loss function;

[0053] S402. Based on the loss function of the grasping detection model, the grasping detection model is tested through a validation set to obtain a grasping detection model with optimal weight parameters.

[0054] Optionally, the controller of S5 establishes a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameter, and obtains the grasping posture in the three-dimensional world coordinate system, including:

[0055] S501, the controller calls the depth camera to obtain external scene information, and uses the grasping detection model with optimal weight parameters to generate multiple grasping postures under the external scene information;

[0056] S502, based on the internal reference Hci of the depth camera, a rotation matrix Hrc between the camera coordinate system and the robot end tool coordinate system is obtained by using the Halcon calibration method;

[0057] S503, based on the camera intrinsic parameter Hci and the rotation matrix Hrc, the grasping posture Gw in the three-dimensional world coordinate system is calculated, including formula (6):

[0058] Gw = Hrc(Hci(g)); (6).

[0059] Optionally, the robot of S6 transports the object to a specified target position according to the grasping posture in the three-dimensional world coordinate system, including:

[0060] The robot receives the grasping posture in the three-dimensional world coordinate system. Based on the use of the Moveit interface and inverse kinematics solution, the robot moves to the target position to grasp the object and transports the object to the specified target position.

[0061] A robot grasping system based on a multifunctional grasping and object-independent framework is applied to the above-mentioned robot grasping based on a multifunctional grasping and object-independent framework, and the system comprises:

[0062] A preprocessing module is used to preprocess the RGB image information in the training set to obtain processed RGB image information;

[0063] A building module, used to build a grasping detection model according to the processed RGB image information and depth information;

[0064] Strategy module, used to obtain loss calculation strategy based on IOU calculation;

[0065] A training module, used for training the grasping detection model according to the loss calculation strategy to obtain a grasping detection model with optimal weight parameters;

[0066] A coordinate system conversion module is used for the controller to establish a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, so as to obtain the grasping posture in the three-dimensional world coordinate system;

[0067] The driving module is used to drive the robot to transport the object to the specified target position according to the grasping posture in the three-dimensional world coordinate system.

[0068] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0069] The present invention improves the robot's ability to grasp objects, from being able to only grasp specific objects in fixed positions to being able to recognize and grasp various types of objects in any position, achieving accurate mapping from object observation data to precise grasping position and posture.

[0070] The present invention solves the problem that it is difficult to ensure the real-time performance of grasping. The method designed by the present invention generates multiple high-quality grasping postures in a complex scene at a speed of milliseconds, and has an extremely high grasping success rate while meeting the real-time performance.

[0071] The present invention can be applied to various unstructured scenarios, solving the problem that traditional robot grasping technology relies on the simplification of physical models and the assumption of a fully observable environment. Given an unknown complex working scene, the method of the present invention can solve the optimal grasping configuration under the working scene, which is intelligent.

[0072] The present invention builds an integrated robot grasping software platform system to enhance human-computer interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 is a flow chart of the method provided by an embodiment of the present invention;

[0075] Figure 2 It is the visualization of feasible crawling and infeasible crawling in the prior art;

[0076] Figure 3 is a network architecture diagram provided by an embodiment of the present invention;

[0077] Figure 4 It is a double 0 type representation model provided by an embodiment of the present invention;

[0078] Figure 5 It is a visualization diagram of calculating EIOU when the predicted grabbing rectangle and the true grabbing rectangle do not intersect, provided by an embodiment of the present invention;

[0079] Figure 6 It is a visualization diagram of calculating EIOU when the predicted grabbing rectangle intersects with the true grabbing rectangle provided by an embodiment of the present invention;

[0080] Figure 7 is a crawl visualization of the detection model on the crawl dataset;

[0081] Figure 8 It is a robotic system;

[0082] Fig. 9 It is the grasping posture reasoning of a single object by a robot in a real environment;

[0083] Fig.10 It is the robot's grasping posture reasoning of multiple objects in real complex stacking scenes;

[0084] Fig.11 The robot performs grasping posture reasoning for the entire grasping scene;

[0085] Fig.12 is a schematic diagram of a system provided by an embodiment of the present invention;

[0086] Fig.13 Schematic diagram of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "one" or "the" do not indicate a quantitative limitation, but indicate the existence of at least one. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0089] It should be noted that the terms "up", "down", "left", "right", "front", "back", etc. used in the present invention are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0090] like Figure 2As shown in , with the vigorous development of deep learning technology, researchers can efficiently and accurately extract the required object features from images, and using deep learning to determine the grasping posture of objects has also become one of the hot research fields at home and abroad. However, the existing grasping detection methods still have many limitations and challenges. First, in complex scenes, different objects have different sizes and shapes, and there may be mutual interference and dependencies between them. However, for grasping detection tasks, it is necessary to consider both the local feature information of each object itself and the global information of the relative position and relationship between multiple objects. Existing grasping detection networks usually increase the receptive field and reduce the resolution by stacking convolutional layers and pooling layers multiple times. Although this approach helps to capture local details, it often leads to the loss of global information, which in turn affects the detection performance in complex scenes. Secondly, in previous grasping detection evaluation methods, when the intersection over union (IOU) between the predicted grasping rectangle and the ground truth exceeds the set threshold, the grasping detection result is considered correct. However, a high IOU value does not mean a successful grasp with high quality and no collision. As Figure 1 As shown in the first row of , although these predicted grasping rectangles have high IOU values, when the robot performs grasping in a real environment, the fingers on one side of the gripper at the end of the robot will collide with the object itself, resulting in an unstable grasp. What we need is Figure 1 The grasping posture configuration shown in the second row of Figure 2. This grasping posture configuration not only has a high IOU value, but also avoids the collision between the gripper at the end of the robot and the object, causing the failure of the grasping task.

[0091] Previous evaluation strategies weakened the match between predictions and reality. High IOU values ​​predicted in grasping datasets do not linearly correlate with high success rates of robot grasping in real environments. Traditional methods treat the center area of ​​the predicted grasping rectangle and the key area near the fingertips as equally important when calculating IOU, but in actual grasping tasks, the area near the fingertips is usually more important for successful grasping.

[0092] like Figures 1 to 13 Based on the above problems, the present invention provides a robot grasping method based on a multifunctional grasping and object-independent framework, the method comprising:

[0093] S1, preprocessing the RGB image information in the training set to obtain processed RGB image information;

[0094] S2. Building a grasping detection model based on the processed RGB image information and depth information;

[0095] S3. Obtain loss calculation strategy based on IOU calculation;

[0096] S4. Based on the loss calculation strategy, the grasping detection model is trained to obtain a grasping detection model with optimal weight parameters;

[0097] S5. The controller establishes a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, and obtains the grasping posture in the three-dimensional world coordinate system;

[0098] S6. The robot transports the object to the specified target location according to the grasping posture in the three-dimensional world coordinate system.

[0099] The specific implementation methods of this method are mainly as follows:

[0100] S1. Preprocess the RGB image information in the training set to obtain processed RGB image information, specifically including:

[0101] S101, after scaling and rotating the RGB image information, randomly shift the RGB image information in the horizontal direction and the vertical direction to obtain the randomly shifted RGB image information;

[0102] S102, cutting an image block of a preset size based on the center of the randomly shifted RGB image information;

[0103] S103: normalize the image block and the corresponding label to obtain processed RGB image information.

[0104] Step S1 is mainly for data preprocessing. First, data enhancement is performed on the data in the training set. The input image is scaled within a certain scale and randomly rotated between [-25°, 25°]. In order to better simulate the actual situation in the real environment, noise is added on this basis. The images will then undergo the following three processing steps. First, they will be randomly displaced in the horizontal and vertical directions. Then, based on the center of the image, a 224×224 image block is cropped. Finally, in order to optimize model training and accelerate calculations, these images and corresponding labels will be normalized.

[0105] S2. Building a grab detection model based on the processed RGB image information and depth information, specifically including:

[0106] S201, the processed RGB image information and depth information are sequentially passed through a convolution layer and a normalization layer, and then subjected to nonlinear processing to obtain input features;

[0107] S202, the global information concatenation and local aggregation module obtains global important key features according to the input features. Step S202 includes:

[0108] S2021, performing window segmentation on the input feature, and calculating the dimension of each window, where the dimension of the input feature is d;

[0109] S2022, each window independently performs attention calculation to obtain the query Q, key K and value matrix V corresponding to each window;

[0110] S2023. Calculate the dot product of the query Q, key K, and value matrix V corresponding to each window respectively, and divide the dot product of the query Q, key K, and value matrix V by the scaling factor to obtain the attention score of each head;

[0111] S2024. Convert the attention score of each head into a corresponding attention weight based on a softmax function, and perform weighted summation on the value matrix V based on the attention weight to obtain the output of each head;

[0112] S2025. Concatenate the output of each head along the last input dimension to obtain a tensor;

[0113] S2026. Transform the tensor into globally important key features through a linear layer, wherein the dimension of the globally important key features is equal to that of the input features.

[0114] S203, based on the convolutional feedforward skip sub-network, converting the globally important key features into output features in the form of a two-dimensional feature image;

[0115] S204, the multi-dimensional multi-scale attention and adaptive feature fusion module processes the output features to enhance the attention of the features associated with grasping detection in the output features, and obtains a grasping detection model.

[0116] Step S2 is mainly to build a crawling detection model. The entire network architecture is as follows Figure 3 As shown in the figure, it mainly includes a global information concatenation and local aggregation module and a multi-dimensional multi-scale attention and adaptive feature fusion module. The whole network consists of downsampling, a global information concatenation and local aggregation module, a multi-dimensional multi-scale attention and adaptive feature fusion module, a deconvolution and parallel decoding part.

[0117] First, the depth camera collects the RGB and depth information of the target object, processes it, and then sends it to the network to predict the grasping pose. First, a convolution layer is used to preliminarily extract key features and reduce the interference of irrelevant pixels. In order to stabilize the training and reduce the internal covariate shift, a batch normalization (BN) layer is then applied. In order to stabilize the training and reduce the internal covariate shift, a batch normalization (BN) layer is then applied (preprocessing stage), which maps the input features to a uniform numerical range, reducing their divergence without changing the dimension of the feature map. After batch normalization, these regularized feature maps are sent to the Mish activation function for further nonlinearization. Throughout the process, the dimension of the feature map remains unchanged after passing through the convolution layer and activation function, that is, it is the same as the feature dimension output by the convolution layer.

[0118] The features after convolution and other operations are sent to the designed global information concatenation and local aggregation module. This module divides the input features into multiple independent windows, and then independently calculates the attention weights in each window to capture the contextual relationship between the features in the window. Then, the results of the attention mechanism output are aggregated through the convolutional aggregation network. This process realizes the effective exchange and fusion of information in the local area, thereby enhancing the effect of feature extraction. Specifically, the input feature x is first divided into multiple windows, the dimension of x is d, and the dimension of the window is d k =d v =d / h (where d k is the dimension of the key, d vis the dimension of value). For each window, self-attention calculation is performed independently, which includes computing the dot product scores of query, key, and value, applying the softmax function to obtain the attention weights, and weighted summing the values ​​using these weights. For each window, the input embedding is linearly transformed to generate query, key, and value matrices. The attention scores for each head are obtained by computing the dot product of the query and key and then dividing by a scaling factor. Next, the softmax function is applied to convert these scores into attention weights. The values ​​are weighted summed using the attention weights to obtain the output of each head. The outputs of all heads are concatenated along the last input dimension to form a larger tensor. This tensor is then transformed back to the original input dimension d by a linear layer. Of particular note, we introduce residual connections, which allow the model to directly access and learn the residual between input and output during training, thereby effectively preventing information loss and network performance degradation. In order to better aggregate local features, a convolutional feedforward skip network is embedded in the global information concatenation and local aggregation module to realize the information flow of local information areas. The convolutional feedforward skip network reshapes the previous output and converts it into a two-dimensional feature image. Subsequently, in order to aggregate information between channels, a 1×1 convolution operation is applied to map the features to a high-dimensional space. Next, in order to further capture the local feature information of the image, a 3×3 depth-separable convolution is used for processing. Finally, a 1×1 convolution operation is used again to reduce the dimension of the feature.

[0119] In order to strengthen the connection between features and grasping parameter space, improve the effectiveness of features, and enable the network to focus more on the parts that have a positive impact on grasping tasks, a multi-dimensional multi-scale attention and adaptive feature fusion module is designed. The core goal of this design is to enhance features that are closely related to grasping detection, while weakening features that are irrelevant to grasping detection. Specific operation: After three identical downsampling and feature processing of global information concatenation and local aggregation modules, the features are sent to the multi-dimensional multi-scale attention and adaptive feature fusion module for the next step. The channel attention part in the multi-dimensional multi-scale attention and adaptive feature fusion module first passes the input feature map through two 3×3 convolutions to extract local spatial information on the feature map. Each 3×3 convolution is followed by a normalized and parameterized ReLu activation function to introduce nonlinearity in the feature map. After that, a global average pooling layer is performed, which averages the values ​​of all spatial positions of each feature map into a single value, aggregating information from the spatial dimension to generate channel-level attention weights in subsequent steps. Then, two fully connected layers are applied to generate attention weights at the channel level. The first fully connected layer compresses the channels, and the second fully connected layer expands them to the original channels to match the number of channels in the original feature map. Finally, the attention weights are multiplied with the input feature map to generate a weighted feature map, in which the more critical channels are emphasized. The features obtained after channel attention are sent to the multi-scale pixel attention and adaptive feature fusion part, and two 3×3 convolutions are first used for feature extraction. The obtained feature maps are processed using dilated convolutions with dilation rates of 3, 5, and 7 respectively. Different dilation rates are used to capture spatial information at different scales and obtain feature information of multiple receptive fields. Then, they are concatenated with the features obtained by the previous two 3×3 convolutions in the channel dimension to obtain feature map T, and then two 1×1 convolutions are used to process feature map T to obtain an important feature map representing the key to grasp detection. Finally, the generated important feature map is multiplied with the original feature map in the multi-scale pixel attention and adaptive feature fusion part to highlight the important areas in the image and suppress irrelevant areas.

[0120] When performing pixel-by-pixel grasp prediction, in order to ensure that the prediction result can match the size of the input image, it is usually necessary to enlarge the output feature map. However, due to the large number of parameters involved in grasp prediction and the small proportion of the actual grasp area in the entire image, directly applying a simple upsampling method often cannot obtain accurate grasp prediction results. In order to solve this problem, the present invention adopts a transposed convolution method to more finely restore the size of the feature map. This method can more effectively capture the detailed information of the image, thereby obtaining more accurate grasp prediction results. Finally, the target features are sent to the parallel decoding network, and the network decodes the grasp parameters pixel by pixel: grasp quality Q, grasp angle θ and grasp width W. Among them, the grasp angle is not output through direct network regression. The network first outputs sinθ and cosθ, and then uses Finally, the network selects the highest quality grasping points and selects the corresponding grasping width and grasping angle to form a set of optimal grasping configurations s = (x, y, θ, w). Based on this, the grasping detection model is obtained.

[0121] S3. The loss calculation strategy is obtained based on IOU calculation, including:

[0122] S301, based on the principle that when the distance between the predicted grabbing rectangle and the true value rectangular box increases, the area of ​​the region where the predicted grabbing box and the true value grabbing box intersect is 0, and the value of the loss function decreases accordingly, the loss function IOU is improved to obtain an improved loss function;

[0123] The calculation formula of the improved loss function is formula (1):

[0124]

[0125] a(A,B) is the minimum enclosing rectangular area of ​​the true value grab box and the predicted grab box, A is the minimum enclosing rectangular area of ​​the true value grab box, and B is the minimum enclosing rectangular area of ​​the predicted grab box;

[0126] η(A,B) is the union area of ​​the true value grasping box and the predicted grasping box;

[0127] EIOU is the improved loss function;

[0128] S302, define an EIOU loss function according to EIOU, wherein the formula of the EIOU loss function is formula (2):

[0129]

[0130] is the EIOU loss function and is the loss calculation strategy.

[0131] Wherein, the definition of the EIOU loss function according to EIOU in S302 includes:

[0132] like Figure 5 , calculate the EIOU value when the true value grab box A and the predicted grab box B do not intersect, including formula (3):

[0133]

[0134] Where M = the minimum circumscribed rectangle area containing the true value grab box A and the predicted grab box B - the union area of ​​the true value grab box A and the predicted grab box B;

[0135] like Figure 6 , calculate the EIOU value when the true value grab box A and the predicted grab box B intersect, including formula (4):

[0136]

[0137] Where Q = the intersection area of ​​the true value grab box A and the predicted grab box B;

[0138] According to formula (3) and formula (4), the EIOU loss function is obtained.

[0139] In step S3, the main purpose is to establish an IOU calculation representation model and a loss calculation strategy. In the previous grasping detection method, the edge areas on both sides of the gripper are not given enough attention, resulting in collision between the gripper and the object during grasping. The present invention proposes a new IOU calculation representation model, which is configured as a "point-to-point" grasping representation model, such as Figure 4 As shown, this representation pays more attention to the area on both sides of the gripper and reduces the influence of the central area.

[0140] The ordinary IOU loss function has a significant limitation: when the predicted grasping rectangle and the true rectangle do not overlap at all, the IOU value is 0, resulting in the gradient of the loss function being 0. To overcome this weakness, an improved loss function (EIOU) is designed, which not only considers the overlapping area between the grasping prediction box and the true box, but also the non-overlapping area between them, especially the relative position and distance between the two boxes. Specifically, when the distance between the predicted grasping rectangle and the true value rectangle increases, even if the area of ​​the intersection between the two is 0, the value of EIOU will decrease accordingly, and the gradient can be back-propagated. The calculation method of EIOU is roughly as follows: first calculate the IOU value between the predicted box and the true box, then consider the minimum bounding rectangle between the two boxes, and calculate the area of ​​the minimum bounding rectangle that is not covered by the two boxes. EIOU is obtained by subtracting a penalty term related to the uncovered area from the IOU, which reflects the relative distance between the two boxes. Therefore, when the distance between the two boxes increases, the value of EIOU decreases, providing a more effective training signal to guide the optimization of the model. The calculation formula of EIOU is formula (1).

[0141] according to Figure 5 and Figure 6 There are two cases, among which, Figure 5 This is a visualization of the EIOU when the predicted grab rectangle does not intersect with the true grab rectangle. When A and B do not intersect, the value of EIOU is formula (3). When A and B are closer, M is smaller, so the larger the EIOU is, the more it can provide the gradient required for model convergence. Figure 6 A visualization of the calculation of EIOU when the predicted grab rectangle intersects with the true grab rectangle is given. When A and B intersect, the value of EIOU is formula (4). When A and B are close to each other, Q will become larger and M will become smaller, so EIOU will become larger and can provide the required gradient value. In this EIOU calculation method, regardless of whether A and B have some kind of intersection, the model can always benefit from the supervision of EIOU loss.

[0142] Since the IOU value between the double-pointed grasping rectangle and the true value rectangle does not exceed 0.5, the EIOU loss function is defined as formula (2).

[0143] S4. Based on the loss calculation strategy, the grasping detection model is trained to obtain a grasping detection model with optimal weight parameters, specifically including:

[0144] S401. Based on the loss calculation strategy, a loss function of the grasping detection model is obtained, wherein the calculation formula of the total loss function of the grasping detection model is formula (5):

[0145]

[0146] in, is the total loss function of the grasp detection model; is the grasp confidence loss function, is the sum of the grasping angle loss functions, is the crawl width loss function;

[0147] S402. Based on the loss function of the grasping detection model, the grasping detection model is tested through a validation set to obtain a grasping detection model with optimal weight parameters.

[0148] Step S4 mainly uses the training data to train the grasping detection model and optimizes it using the loss function. During the training process, the back propagation algorithm is used to optimize the gradient of the objective function to ensure that the difference between the grasping box output by the detection model and the true value can be gradually reduced to a minimum. The loss function calculation formula is formula (5). At the same time, in order to improve the training effect of the model and prevent overfitting, the grasping detection model is tested using the validation set. Through the performance of the validation set, the learning rate in the model training process is adjusted in time to optimize the training efficiency and generalization ability of the model, thereby avoiding the overfitting phenomenon of the grasping detection model to a certain extent. In this way, a grasping detection model with optimal weight parameters is obtained.

[0149] S5. The controller establishes a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model with the optimal weight parameters, and obtains the grasping posture in the three-dimensional world coordinate system, specifically including:

[0150] S501, the controller calls the depth camera to obtain external scene information, and uses the grasping detection model with optimal weight parameters to generate multiple grasping postures under the external scene information;

[0151] S502, based on the internal reference Hci of the depth camera, a rotation matrix Hrc between the camera coordinate system and the robot end tool coordinate system is obtained by using the Halcon calibration method;

[0152] S503, based on the camera intrinsic parameter Hci and the rotation matrix Hrc, the grasping posture Gw in the three-dimensional world coordinate system is calculated, including formula (6):

[0153] Gw = Hrc(Hci(g)); (6).

[0154] Step S5 mainly uses the controller to carry a grasping detection model to establish the conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration. The controller calls the depth camera to obtain external scene information, and uses the built grasping detection model to generate multiple high-quality grasping postures in the scene. In order to transform the grasping posture in the camera coordinate system into a grasping posture configuration that the robot can understand in the world coordinate system, a physical space correspondence between the camera and the robot is established, so that the robot understands the position of the target object in its own coordinate system. Through the intrinsic reference Hci of the depth camera, combined with the rotation matrix Hrc between the camera coordinate system and the robot end tool coordinate system obtained by the Halcon calibration method, the camera intrinsic reference Hci and the rotation matrix Hrc are used to calculate the grasping posture Gw of the grasping point in the three-dimensional world.

[0155] S6. The robot transports the object to the specified target location according to the grasping posture in the three-dimensional world coordinate system, mainly including:

[0156] The robot receives the grasping posture in the three-dimensional world coordinate system. Based on the use of the Moveit interface and inverse kinematics solution, the robot moves to the target position to grasp the object and transports the object to the specified target position.

[0157] The present invention proposes a new IOU calculation and representation strategy, so that a high IOU value and a high grasping success rate show a corresponding linear relationship. This calculation strategy pays attention to the edge areas at both ends of the gripper to avoid collisions between the gripper at the end of the robot and the object in the real world. The grasping detection model proposed in the present invention can capture the local context information and detailed features of the graspable object, model the long-term dependencies between distant pixels, and pay attention to the overall contour information of the object while paying attention to local information. Two newly designed modules in the network: the global information concatenation and local aggregation module and the multi-dimensional multi-scale attention and adaptive feature fusion module, realize the flow of information between windows, enhance the extraction capabilities of local and global information, and correlate the features with the grasping parameter space, thereby improving the detection accuracy. The present invention proposes a new loss function, optimizes the IOU loss term in the loss function, and solves the problem of gradient disappearance caused by IOU in the case of non-overlapping grasping frames.

[0158] Figure 7 This is the visualization of the grasping detection model of the present invention on the grasping detection dataset. It can be seen that when the designed EIOU calculation representation model is used, the network pays more attention to the areas at both ends of the gripper. Due to the enhanced perception and modeling of global information, the network pays attention to the details while taking into account the overall contour information of the object.

[0159] Figure 8 The robot system of the present invention is equipped with a depth camera to perceive external scene information in real time.

[0160] like Fig.12 A robot grasping system based on a multifunctional grasping and object-independent framework is applied to the above-mentioned robot grasping based on a multifunctional grasping and object-independent framework, and the system comprises:

[0161] A preprocessing module is used to preprocess the RGB image information in the training set to obtain processed RGB image information;

[0162] A building module, used to build a grasping detection model according to the processed RGB image information and depth information;

[0163] Strategy module, used to obtain loss calculation strategy based on IOU calculation;

[0164] A training module, used for training the grasping detection model according to the loss calculation strategy to obtain a grasping detection model with optimal weight parameters;

[0165] A coordinate system conversion module is used for the controller to establish a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, so as to obtain the grasping posture in the three-dimensional world coordinate system;

[0166] The driving module is used to drive the robot to transport the object to the specified target position according to the grasping posture in the three-dimensional world coordinate system.

[0167] The present invention improves the robot's ability to grasp objects, from being able to grasp only specific objects in fixed positions to being able to identify and grasp various types of objects in any position, and achieving accurate mapping from object observation data to precise grasping position and posture. The present invention solves the problem that the real-time performance of grasping is difficult to ensure. The method designed by the present invention generates multiple high-quality grasping postures in a complex scene at a speed of milliseconds, and has a very high grasping success rate while meeting the real-time performance. The present invention can be applied to various unstructured scenes, solving the problem that traditional robot grasping technology relies on the simplification of physical models and the assumption of a fully observable environment. Given an unknown complex working scene, the method of the present invention can solve the optimal grasping configuration under the working scene, and is intelligent. The present invention builds an integrated robot grasping software platform system to enhance human-computer interactivity.

[0168] Fig.13 is a schematic diagram of the structure of a robot grasping device based on a multifunctional grasping and object-independent framework provided by an embodiment of the present invention, such as Fig.13 As shown, the robot grasping device based on the multifunctional grasping and object-independent framework may include the above Fig.12The robot grasping system based on the multifunctional grasping and object-independent framework is shown. Optionally, the robot grasping device 410 based on the multifunctional grasping and object-independent framework may include a first processor 2001 .

[0169] Optionally, the robot grasping device 410 based on the multifunctional grasping and object-independent framework may further include a memory 2002 and a transceiver 2003 .

[0170] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0171] Combine the following Fig.13 The components of the robot grasping device 410 based on the multifunctional grasping and object-independent framework are specifically introduced:

[0172] The first processor 2001 is the control center of the robot grasping device 410 based on the multifunctional grasping and object-independent framework, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0173] Optionally, the first processor 2001 can perform various functions of the robot grasping device based on the multifunctional grasping and object-independent framework by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0174] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Fig.13 CPU0 and CPU1 are shown in FIG.

[0175] In a specific implementation, as an embodiment, the robot grasping device 410 based on the multifunctional grasping and object-independent framework may also include multiple processors, such as Fig.13The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0176] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0177] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Fig.13 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0178] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0179] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Fig.13 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0180] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Fig.13(not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0181] It should be noted that Fig.13 The structure of the robot grasping device 410 based on the multifunctional grasping and object-independent framework shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0182] In addition, the technical effects of the robot grasping device 410 based on the multifunctional grasping and object-independent framework can refer to the technical effects of the robot grasping method based on the multifunctional grasping and object-independent framework described in the above method embodiment, which will not be repeated here.

[0183] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0184] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0185] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0186] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0187] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0188] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0189] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0190] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0191] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0194] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program codes.

[0195] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A robot grasping method based on multifunctional grasping and object-independent framework, characterized in that: include: S1, preprocessing the RGB image information in the training set to obtain processed RGB image information; S2. Building a grasping detection model based on the processed RGB image information and depth information; S3. Obtain loss calculation strategy based on IOU calculation; S4. Based on the loss calculation strategy, the grasping detection model is trained to obtain a grasping detection model with optimal weight parameters; S5. The controller establishes a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, and obtains the grasping posture in the three-dimensional world coordinate system; S6, the robot transports the object to the specified target location according to the grasping posture in the three-dimensional world coordinate system; The step S1 preprocesses the RGB image information in the training set to obtain the processed RGB image information including: S101, after scaling and rotating the RGB image information, randomly shift the RGB image information in the horizontal direction and the vertical direction to obtain the randomly shifted RGB image information; S102, cutting an image block of a preset size based on the center of the randomly shifted RGB image information; S103, normalizing the image block and the corresponding label to obtain processed RGB image information; The step S2 of building a grasping detection model according to the processed RGB image information and depth information includes: S201, the processed RGB image information and depth information are sequentially passed through a convolution layer and a normalization layer, and then subjected to nonlinear processing to obtain input features; S202, the global information concatenation and local aggregation module obtains globally important key features according to the input features; S203, based on the convolutional feedforward skip sub-network, converting the globally important key features into output features in the form of a two-dimensional feature image; S204, a multi-dimensional multi-scale attention and adaptive feature fusion module processes the output features to enhance the attention of features associated with grasping detection in the output features, thereby obtaining a grasping detection model; The global information concatenation and local aggregation module of S202 obtains the globally important key features according to the input features, including: S2021, performing window segmentation on the input feature, and calculating the dimension of each window, where the dimension of the input feature is d; S2022, each window independently performs attention calculation to obtain the query Q, key K and value matrix V corresponding to each window; S2023. Calculate the dot product of the query Q, key K, and value matrix V corresponding to each window respectively, and divide the dot product of the query Q, key K, and value matrix V by the scaling factor to obtain the attention score of each head; S2024. Convert the attention score of each head into a corresponding attention weight based on a softmax function, and perform weighted summation on the value matrix V based on the attention weight to obtain the output of each head; S2025. Concatenate the output of each head along the last input dimension to obtain a tensor; S2026. Transform the tensor into a globally important key feature through a linear layer, where the dimension of the globally important key feature is equal to that of the input feature; The loss calculation strategy of S3 based on IOU calculation includes: S301, based on the principle that when the distance between the predicted grabbing rectangle and the true value rectangular box increases, the area of ​​the region where the predicted grabbing box and the true value grabbing box intersect is 0, and the value of the loss function decreases accordingly, the loss function IOU is improved to obtain an improved loss function; The calculation formula of the improved loss function is formula (1): a(A,B) is the minimum enclosing rectangular area of ​​the true value grab box and the predicted grab box, A is the minimum enclosing rectangular area of ​​the true value grab box, and B is the minimum enclosing rectangular area of ​​the predicted grab box; η(A,B) is the union area of ​​the true value grasping box and the predicted grasping box; EIOU is the improved loss function; S302, define an EIOU loss function according to EIOU, wherein the formula of the EIOU loss function is formula (2): is the EIOU loss function, is the loss calculation strategy; The step of defining the EIOU loss function according to EIOU in S302 includes: Calculate the EIOU value when the true value grab box A and the predicted grab box B do not intersect, including formula (3): Where M = the area of ​​the minimum circumscribed rectangle containing the true value grab box A and the predicted grab box B - the union area of ​​the true value grab box A and the predicted grab box B; Calculate the EIOU value when the true value grab box A and the predicted grab box B intersect, including formula (4): Where Q = the intersection area of ​​the true value grab box A and the predicted grab box B; According to formula (3) and formula (4), the EIOU loss function is obtained.

2. The robot grasping method based on multifunctional grasping and object-independent framework according to claim 1 is characterized in that: The S4, based on the loss calculation strategy, trains the grasping detection model to obtain a grasping detection model with optimal weight parameters, including: S401. Based on the loss calculation strategy, a loss function of the grasping detection model is obtained, wherein the calculation formula of the total loss function of the grasping detection model is formula (5): in, is the total loss function of the grasp detection model; is the grasp confidence loss function, is the sum of the grasping angle loss functions, is the crawl width loss function; S402. Based on the loss function of the grasping detection model, the grasping detection model is tested through a validation set to obtain a grasping detection model with optimal weight parameters.

3. The robot grasping method based on multifunctional grasping and object-independent framework according to claim 2 is characterized in that: The controller of S5 establishes a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, and obtains the grasping posture in the three-dimensional world coordinate system, including: S501, the controller calls the depth camera to obtain external scene information, and uses the grasping detection model with optimal weight parameters to generate multiple grasping postures under the external scene information; S502, based on the internal reference Hci of the depth camera, a rotation matrix Hrc between the camera coordinate system and the robot end tool coordinate system is obtained by using the Halcon calibration method; S503, based on the camera intrinsic parameter Hci and the rotation matrix Hrc, the grasping posture Gw in the three-dimensional world coordinate system is calculated, including formula (6): Gw = Hrc(Hci(g)); (6).

4. The robot grasping method based on multifunctional grasping and object-independent framework according to claim 3 is characterized in that: The S6 robot transports the object to the specified target location according to the grasping posture in the three-dimensional world coordinate system, including: The robot receives the grasping posture in the three-dimensional world coordinate system. Based on the use of the Moveit interface and inverse kinematics solution, the robot moves to the target position to grasp the object and transports the object to the specified target position.

5. A robot grasping system based on a multifunctional grasping and object-independent framework, characterized in that: The robot grasping method based on multifunctional grasping and object-independent framework as described in any one of claims 1 to 4, the system comprising: A preprocessing module is used to preprocess the RGB image information in the training set to obtain processed RGB image information; A building module, used to build a grasping detection model according to the processed RGB image information and depth information; Strategy module, used to obtain loss calculation strategy based on IOU calculation; A training module, used for training the grasping detection model according to the loss calculation strategy to obtain a grasping detection model with optimal weight parameters; A coordinate system conversion module is used for the controller to establish a conversion relationship between the camera coordinate system and the world coordinate system for the grasping posture configuration according to the grasping detection model carrying the optimal weight parameters, so as to obtain the grasping posture in the three-dimensional world coordinate system; The driving module is used to drive the robot to transport the object to the specified target location according to the grasping posture in the three-dimensional world coordinate system; The preprocessing of the RGB image information in the training set to obtain the processed RGB image information includes: S101, after scaling and rotating the RGB image information, randomly shift the RGB image information in the horizontal direction and the vertical direction to obtain the randomly shifted RGB image information; S102, cutting an image block of a preset size based on the center of the randomly shifted RGB image information; S103, normalizing the image block and the corresponding label to obtain processed RGB image information; The step of building a grasping detection model according to the processed RGB image information and depth information includes: S201, the processed RGB image information and depth information are sequentially passed through a convolution layer and a normalization layer, and then subjected to nonlinear processing to obtain input features; S202, the global information concatenation and local aggregation module obtains globally important key features according to the input features; S203, based on the convolutional feedforward skip sub-network, converting the globally important key features into output features in the form of a two-dimensional feature image; S204, a multi-dimensional multi-scale attention and adaptive feature fusion module processes the output features to enhance the attention of features associated with grasping detection in the output features, thereby obtaining a grasping detection model; The global information concatenation and local aggregation module of S202 obtains the globally important key features according to the input features, including: S2021, performing window segmentation on the input feature, and calculating the dimension of each window, where the dimension of the input feature is d; S2022, each window independently performs attention calculation to obtain the query Q, key K and value matrix V corresponding to each window; S2023. Calculate the dot product of the query Q, key K, and value matrix V corresponding to each window respectively, and divide the dot product of the query Q, key K, and value matrix V by the scaling factor to obtain the attention score of each head; S2024. Convert the attention score of each head into a corresponding attention weight based on a softmax function, and perform weighted summation on the value matrix V based on the attention weight to obtain the output of each head; S2025. Concatenate the output of each head along the last input dimension to obtain a tensor; S2026. Transform the tensor into a globally important key feature through a linear layer, where the dimension of the globally important key feature is equal to that of the input feature; The loss calculation strategy based on IOU calculation includes: S301, based on the principle that when the distance between the predicted grabbing rectangle and the true value rectangular box increases, the area of ​​the region where the predicted grabbing box and the true value grabbing box intersect is 0, and the value of the loss function decreases accordingly, the loss function IOU is improved to obtain an improved loss function; The calculation formula of the improved loss function is formula (1): a(A,B) is the minimum enclosing rectangular area of ​​the true value grab box and the predicted grab box, A is the minimum enclosing rectangular area of ​​the true value grab box, and B is the minimum enclosing rectangular area of ​​the predicted grab box; η(A,B) is the union area of ​​the true value grasping box and the predicted grasping box; EIOU is the improved loss function; S302, define an EIOU loss function according to EIOU, wherein the formula of the EIOU loss function is formula (2): is the EIOU loss function, is the loss calculation strategy; The step of defining the EIOU loss function according to EIOU in S302 includes: Calculate the EIOU value when the true value grab box A and the predicted grab box B do not intersect, including formula (3): Where M = the area of ​​the minimum circumscribed rectangle containing the true value grab box A and the predicted grab box B - the union area of ​​the true value grab box A and the predicted grab box B; Calculate the EIOU value when the true value grab box A and the predicted grab box B intersect, including formula (4): Where Q = the intersection area of ​​the true value grab box A and the predicted grab box B; According to formula (3) and formula (4), the EIOU loss function is obtained.

Images