Method and system for detecting target grabbed by mechanical arm

Abstract

The invention relates to a method and system for detecting a target grabbed by a mechanical arm, and the method comprises the steps: collecting a target material image, and obtaining an initial data set; preprocessing the initial data set by adopting a data enhancement mode to obtain a target data set; taking the generative residual convolutional neural network model as a reference model to construct an improved generative residual convolutional neural network model as a mechanical arm grabbing target detection network model; calculating each output loss and total loss by using an improved loss calculation method, obtaining a difference between the output loss and the total loss and practical feasible capture information, and obtaining a training weight; and updating the weight of the model by using a loss calculation result to obtain an optimal training weight, and inputting a test set into the trained mechanical arm grabbing target detection network model to obtain a mechanical arm grabbing target detection result. According to the improved mechanical arm grabbing target detection network model, the grabbing target detection precision is improved, and the grabbing target detection speed is increased.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a method and system for detecting a target captured by a robot arm. Background Art

[0002] In recent years, robotic grasping technology has gradually become a research hotspot and has been widely used in fields such as home assistance, human-machine collaboration, industrial manufacturing, and space debris cleaning. In the interaction between robots and the real environment, stable and efficient grasping is the key link to achieve reliable operation, which also puts higher requirements on the robot's grasping effect and stability.

[0003] At present, Pinto et al. applied convolutional neural network (CNN) to accurately predict the grasping posture of objects, which greatly improved the overall generalization performance of the network. Asif et al. fused the multi-layer features of CNN layer by layer and selected the most credible grasping configuration as output at the global, local and pixel levels, thus overcoming the receptive field limitation. Kumra et al. further proposed GR-ConvNet based on RGB-D input, whose output includes grasping quality, rotation angle and opening width.

[0004] However, since the network encoder consists of only three layers of convolution, the feature extraction capability is limited, and the encoder uses a large convolution kernel, resulting in too much initial computation, insufficient backbone network depth, and difficulty in fully extracting image features. In addition, when processing the output, GR-ConvNet simply adds the outputs of grasping quality, rotation angle, and opening width, and fails to reasonably allocate weights. In response to the above shortcomings, this paper proposes an improved grasping detection model FusedCNN, which comprehensively optimizes the original model, including introducing a more efficient convolution structure and improving the loss calculation and prediction mechanism. The model improves feature extraction capabilities, computational efficiency, and generalization, enabling it to show higher accuracy and stability in complex grasping tasks and computer vision applications.

[0005] Therefore, based on the current grasping algorithm, a method and system for detecting target information for robot grasping is proposed. Summary of the invention

[0006] Based on the above-mentioned problems existing in the prior art, the purpose of an embodiment of the present invention is to provide a method and system for detecting a target by grasping a robot arm, which improves the accuracy of predicting grasping information by correcting the original loss calculation method and performing weighted calculation on each part of the total loss, thereby improving the accuracy and speed of the model's grasping target detection.

[0007] To achieve the above object, the technical solution adopted by the present invention is: a method for detecting a target grasped by a robot arm, comprising:

[0008] S1, collect the target material image and obtain the initial data set;

[0009] S2, preprocess the initial data set using data enhancement to obtain the target data set;

[0010] S3, taking the generative residual convolutional neural network model as the baseline model, an improved generative residual convolutional neural network model is constructed as the robot arm grasping target detection network model;

[0011] S4, using the improved loss calculation method to calculate each output loss and the total loss, and obtain the difference between the actual feasible crawling information;

[0012] S5. Use the loss calculation result to update the model weight to obtain the optimal training weight;

[0013] S6, inputting the test set into the trained robot arm grasping target detection network model to obtain the robot arm grasping target detection result.

[0014] Furthermore, in S3, an improved generative residual convolutional neural network model is constructed with the generative residual convolutional neural network model as a benchmark model as a robot arm grasping target detection network model, including:

[0015] Step S31, construct a new encoder module to replace the original encoder, and input the training set image into the encoder for downsampling and preliminary feature extraction;

[0016] Step S32, building a backbone feature extraction network to replace the original 5-layer residual module, inputting the feature map output by the encoder into the backbone feature extraction network to perform deep feature extraction;

[0017] Step S33, the decoder upsamples the output feature map of the backbone extraction network, and uses the upsampled output to predict the target's grasping information including the grasping position, the sine value of the grasping angle, the cosine value of the grasping angle, and the grasping width.

[0018] Furthermore, the encoder is composed of three layers of dilated convolution layers DconvBNact, which gradually process the input graphic parameters and then output them. The dilated convolution layer DconvBNact includes: a dilated convolution DilatedConv2d with a convolution kernel size of 3×3, a step size of 1 or 2, and dilation factors of [1, 2, 3] respectively, a data batch normalization BN layer and a SiLU activation function. The dilated convolution layer DconvBNact is used to downsample the input captured target image and perform preliminary feature extraction.

[0019] Furthermore, the calculation rule of the data batch normalization BN layer is:

[0020]

[0021] Among them, x i The i-th dimension sample data of the batch data output by the convolution DilatedConv2d, μ B For x i The average value of For x i The variance of For x i After normalization, ε is a positive number much smaller than 1 to avoid the denominator being zero. i for The linear transformation output of , γ and β are learnable parameters for scaling and translation

[0022] The calculation rule of the SiLU function is:

[0023] f(x)=x*σ(x)

[0024]

[0025] Among them, f(x) is the output after SiLU activation function processing, x is the feature map input to SiLU activation function, and σ(x) is the sigmoid function.

[0026] Furthermore, the downsampling and preliminary feature extraction process of the encoder includes:

[0027] Step S311, the input image size is 224*224*4, and it passes through the first layer of dilated convolution layer. The first layer of dilated convolution layer includes: DilatedConv2d with a convolution kernel size of 3*3, a step size of 1, a dilation factor of 1, a batch normalization BN layer, and a SiLU activation function; the dilated convolution layer is used to perform preliminary downsampling on the input target object image. After downsampling, the feature map size of the output of this layer is 224*224*32.

[0028] Step S312, through the second layer of dilated convolution layer, which includes: DilatedConv2d with a convolution kernel size of 3*3, a step size of 2, a dilation factor of 2, a batch normalization BN layer and a SiLU activation function; the dilated convolution layer is used to further downsample the input target object image, and after downsampling, the feature map size of the output of this layer is 112*112*64.

[0029] Step S313, through the third dilated convolution layer, which includes: a DilatedConv2d with a convolution kernel size of 3*3, a step size of 2, and a dilation factor of 3, a batch normalization BN layer, and a SiLU activation function; the dilated convolution layer is used to downsample the input target object image, and after downsampling, the feature map size of the output of this layer is 56*56*128.

[0030] Furthermore, the backbone feature extraction network includes two different convolution structures according to different expansion factors. The first convolution structure is used when the expansion factor is 1. The input feature map extracts spatial position information through the CA attention module, and then passes through a 1x1 projection convolution layer and batch normalization, and then passes through the Dropout layer, and then performs an additive residual connection with the input to obtain the output; the second convolution structure is used when the expansion factor is not 1. The input first passes through a 3x3 expansion convolution layer, followed by batch normalization and SiLU activation, and then passes through a 1x1 projection convolution layer and batch normalization, and then passes through the Dropout layer, and then performs an additive residual connection with the input to generate the output.

[0031] Furthermore, the structure of the decoder consists of three layers of transposed convolutional layers ConvT2D with different convolution kernel sizes, and the first and second layers are activated using a batch normalization BN layer and a ReLU activation function.

[0032] Further, in S4, the improved loss calculation method calculates each output loss and the total loss including:

[0033] Step S41: The predicted value output by the robot arm target detection network model is G c = {G p ,G sin ,G cos ,G w}, the predicted value consists of four parts, namely the grasping position G p , Sine value of the grab angle G sin , grab angle cosine value G cos , Grab width G w ;

[0034] Step S42, the actual value of the capture obtained from the image annotation is Regression loss estimation of the graspable area’s grasp position, angle, and width: loss p 、loss cos 、loss sin , and loss w , except for the width loss, the other losses use the SmoothL1 function to calculate the average value of n training samples;

[0035] Step S43, for the width loss lossw , make a certain bias in the calculation of the loss, increase the loss value when the width is smaller than the actual width, and adjust the loss calculation based on SmoothL1;

[0036] Step S44, calculate the total network loss Loss all .

[0037] Furthermore, in S5, the weights of the model are updated using the result of the loss calculation to obtain the optimal training weights, including: inputting the verification set images into the robotic arm grasping target detection network model after the weights are updated, detecting the performance of the current model according to the performance indicators on the verification set, and comparing it with the performance of the current optimal model, and recording the training weights of the model with higher performance as the optimal training weights for the current stage.

[0038] A robot arm grasping target detection system is applied to the above-mentioned robot arm grasping target detection method, and the system comprises:

[0039] A data set acquisition module is used to acquire images of target objects and obtain an initial data set;

[0040] The data set preprocessing module is used to preprocess the initial data set by using data enhancement to obtain the target data set;

[0041] An improved modeling module is used to construct an improved generative residual convolutional neural network model as a robot arm grasping target detection network model based on the generative residual convolutional neural network model as a baseline model;

[0042] A weight training module is used to input the training set into the robot arm grasping target detection network model for model training, and improve the parameters of the robot arm grasping target detection network model based on the training results to obtain the trained robot arm grasping target detection network model;

[0043] A weight updating module, used for inputting the verification set into the trained robot arm grasping target detection network model for model evaluation;

[0044] The detection result generation module is used to input the test set into the trained robot arm grasping target detection network model to obtain the robot arm grasping target detection result.

[0045] The beneficial effects of the present invention are:

[0046] A method for detecting a target grasped by a robot arm of the present invention comprises collecting a target object image to obtain an initial data set; preprocessing the initial data set by data enhancement to obtain a target data set; dividing the target data set into a training set and a validation set, and randomly extracting some sample images from the target data set as a test set; constructing an improved generative residual convolutional neural network model as a robot grasping target detection network model with a generative residual convolutional neural network model as a benchmark model; inputting the training set into the robot grasping target detection network model for model training, and improving the parameters of the robot grasping target detection network model based on the training result to obtain a trained robot grasping target detection network model; inputting the validation set into the trained robot grasping target detection network model for model evaluation; and inputting the test set into the trained robot grasping target detection network model to obtain a robot grasping target detection result. The present invention solves the problem of excessive early calculation of the network model by replacing the encoder of the generative residual convolutional neural network model with an encoder constructed by a dilated convolution module, thereby achieving lightweighting of the model. By building a backbone feature extraction network to replace the backbone extraction network of the generative residual convolutional neural network model, richer feature information is extracted, and the model feature extraction capability is improved. By correcting the original loss calculation method and performing weighted calculation on each part of the total loss, the accuracy of the predicted capture information is improved, and the accuracy and speed of the capture target detection are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] In the figure:

[0049] Figure 1 A flowchart of an improved method for detecting a target grasped by a robotic arm provided in Embodiment 1 of the present invention;

[0050] Figure 2 A schematic diagram of the overall structure of an improved method for detecting a target grasped by a robotic arm provided in the first embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the structure of an improved encoder provided in Embodiment 1 of the present invention;

[0052] Figure 4 A schematic diagram of the structure of an improved backbone feature extraction network provided in the first embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the structure of a CA module provided in Embodiment 1 of the present invention;

[0054] Figure 6 A schematic diagram of the structure of a decoder provided in the first embodiment of the present invention;

[0055] Figure 7 A schematic diagram for comparing the model detection effects provided in the first embodiment of the present invention;

[0056] Figure 8 A schematic diagram of a module of a robot arm grasping target detection system provided in the second embodiment of the present invention;

[0057] Fig. 9 It is a structural diagram of a network-side server provided according to a third embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] In order to facilitate subsequent understanding, the professional terms appearing in the following embodiments are explained here:

[0060] Encoder: Encoder is a component in deep learning that is used to gradually reduce the spatial dimension of feature maps.

[0061] Decoder: Decoder is a component in deep learning that is used to gradually increase the spatial dimension of feature maps.

[0062] Backbone Extraction Network: BackboneNetwork, a core component for feature extraction in deep learning.

[0063] DconvBNact module: DilatedConvolutionwithBatchNormalizationandActivation, is a module used for downsampling in deep learning.

[0064] FusedConv module: FusedConvolution is a module used for feature extraction in deep learning

[0065] Conv2d module: It is a two-dimensional convolutional layer (2DConvolutionalLayer) in deep learning, mainly used to process image data.

[0066] DilatedConv2d module: Also called dilated convolution or hole convolution, it is a special two-dimensional convolution operation in deep learning.

[0067] Transposed convolution layer ConvT2d: Transposed2DConvolutionLayer is a convolution operation used for upsampling in neural networks.

[0068] CA attention module: Coordinateattention is an attention mechanism for enhancing feature representation.

[0069] ExpandConv: ExpandConvolution is a convolution operation used to expand the number of feature channels.

[0070] Projection convolution layer ProjectConv: ProjectConvolution is a convolution operation used to restore the number of expanded feature channels.

[0071] Data batch normalization BN layer: BatchNormalization, referred to as BN, is a structure used in deep learning to accelerate the training process and improve the stability of the model.

[0072] Dropout layer: It is a regularization technique commonly used in deep learning.

[0073] SiLU activation function: SigmoidLinearUnit, is a self-gated activation function.

[0074] ReLU activation function: RectifiedLinearUnit, is a nonlinear activation function widely used in deep learning.

[0075] SmoothL1 function: a loss function commonly used in deep learning

[0076] First embodiment:

[0077] A method for detecting a target grasped by a robot arm of the present invention comprises collecting a target object image to obtain an initial data set; preprocessing the initial data set by means of data enhancement to obtain a target data set; dividing the target data set into a training set and a validation set, and randomly extracting some sample images from the target data set as a test set; constructing an improved generative residual convolutional neural network model as a robot grasping target detection network model with a generative residual convolutional neural network model as a benchmark model; inputting the training set into the robot grasping target detection network model for model training, and improving the parameters of the robot grasping target detection network model based on the training result to obtain a trained robot grasping target detection network model; inputting the validation set into the trained robot grasping target detection network model for model evaluation; and inputting the test set into the trained robot grasping target detection network model to obtain a robot grasping target detection result.

[0078] A first embodiment of the present invention provides a method for detecting a target grasped by a robot arm, comprising:

[0079] The following is a detailed description of the implementation details of a method for detecting a target grasped by a robot arm in this embodiment. The following content is only provided for the convenience of understanding the implementation details and is not necessary for the implementation of this solution. The specific process of this embodiment is as follows: Figure 1 As shown, this embodiment is applied to a method for detecting a target grasped by a robot arm.

[0080] S1, collect target material images to obtain an initial data set.

[0081] Specifically, a depth camera KinectV2 is used to photograph the target object. The depth camera KinectV2 is set above the target object, and the target object is placed in a storage box. The depth camera KinectV2 is equipped with a depth camera and a high-resolution color camera so that it can clearly capture the image of the target object used in the grasping test; the depth camera captures 885 images of the target object used in the robot grasping test, and then all the collected images are transmitted to a local storage device or server to obtain an initial data set.

[0082] Step S2: preprocess the initial data set by data enhancement to obtain the target data set.

[0083] Specifically, the target object is placed in the storage box, and the depth camera will take pictures in different states during the shooting process. In order to enrich the diversity of the target data set, the image is preprocessed by data enhancement, including: cropping the image size, flipping the horizontal direction of the image, adjusting the image hue and saturation, adjusting the image rotation angle, and adding Gaussian noise. The target data set is split into a training set and a validation set in a ratio of 8:2.

[0084] Step S3, using the generative residual convolutional neural network model as a baseline model to construct an improved generative residual convolutional neural network model as a robotic arm grasping target detection network model.

[0085] The improved generative residual convolutional neural network model as a robot arm grasping target detection network model includes:

[0086] Step S31, construct a new encoder module to replace the original encoder, and input the training set image into the encoder for downsampling and preliminary feature extraction.

[0087] Furthermore, the encoder is stored in an industrial computer as part of the program. The original encoder is a three-layer ordinary convolution module with a convolution kernel size of 9, 4, 4. The original three-layer ordinary convolution module is replaced with a three-layer hole convolution module with a convolution kernel size of 3, 3, 3. By replacing the convolution module, the computational complexity of the encoder can be reduced.

[0088] See also Figure 2 The encoder consists of three layers of dilated convolutional layers DconvBNact, which process the input graphic parameters step by step and then output them. The dilated convolutional layer DconvBNact includes: a dilated convolution DilatedConv2d with a convolution kernel size of 3×3, a stride of 1 or 2, and dilation factors of [1, 2, 3], a batch normalization BN layer, and a SiLU activation function. The dilated convolutional layer DconvBNact is used to downsample the input captured target image and perform preliminary feature extraction;

[0089] Specifically, the calculation rule of the data batch normalization BN layer is:

[0090]

[0091] Among them, x i The i-th dimension sample data of the batch data output by the convolution DilatedConv2d, μ B For x i The average value of For x i The variance of For x i After normalization, ε is a positive number much smaller than 1 to avoid the denominator being zero. i for The linear transformation output of , γ and β are learnable parameters for scaling and translation

[0092] Specifically, the calculation rule of the SiLU function is:

[0093] f(x)=x*σ(x)

[0094]

[0095] Among them, f(x) is the output after SiLU activation function processing, x is the feature map input to SiLU activation function, and σ(x) is the sigmoid function.

[0096] Specifically, the specific steps of inputting the training set image into the encoder for downsampling and preliminary feature extraction include:

[0097] Step S311, the input image size is 224*224*4, and passes through the first layer of dilated convolution layer, which includes: DilatedConv2d with a convolution kernel size of 3*3, a step size of 1, a dilation factor of 1, a batch normalization BN layer, and a SiLU activation function; the dilated convolution layer is used to perform preliminary downsampling on the input target object image, and after downsampling, the feature map size of the output of this layer is 224*224*32.

[0098] Step S312, through the second layer of dilated convolution layer, which includes: DilatedConv2d with a convolution kernel size of 3*3, a step size of 2, a dilation factor of 2, a batch normalization BN layer and a SiLU activation function; the dilated convolution layer is used to further downsample the input target object image, and after downsampling, the feature map size of the output of this layer is 112*112*64.

[0099] Step S313, through the third dilated convolution layer, which includes: a DilatedConv2d with a convolution kernel size of 3*3, a step size of 2, and a dilation factor of 3, a batch normalization BN layer, and a SiLU activation function; the dilated convolution layer is used to downsample the input target object image, and after downsampling, the feature map size of the output of this layer is 56*56*128.

[0100] Step S32: build a backbone feature extraction network to replace the original 5-layer residual module, input the feature map output by the encoder into the backbone feature extraction network, and perform deep feature extraction.

[0101] See also Figure 3 ,The backbone feature extraction network includes two different convolutional structures according to the different ,expansion factors.

[0102] Specifically, the first convolution structure is used when the expansion factor is 1. The input feature map extracts spatial position information through the CA attention module, then passes through a 1x1 projection convolution layer (with a step size of 1) and batch normalization, and then passes through the Dropout layer, and then performs an additive residual connection with the input to obtain the output.

[0103] like Figure 4As shown in the figure, the CA attention module aims to make the model pay attention to the spatial location information. The CA attention module encodes channel relationships and long-range dependencies with precise location information through two steps: coordinate information embedding and coordinate attention generation.

[0104] The input information size of the CA module is C×H×W. In order to capture long-range spatial interactions with precise location information, the global average pooling is decomposed along the horizontal coordinate and the vertical coordinate, respectively, as shown in the following formula:

[0105]

[0106] Among them, given an input X, a total of C channels, the height of the input image is H, the width is W, i and j are variables that traverse H and W respectively, and the input x of the cth channel of the input image is c Average pooling is performed along the width W at height h. The result of average pooling is Similarly, is the input x of the cth channel c The output of average pooling is performed along the height H direction at the bandwidth w. These two parallel operations preserve long-range dependencies in one direction while also retaining sufficient positional information along the other.

[0107] Then connect them together and then perform F 1 Operation (using 1*1 convolution kernel for dimensionality reduction) and activation operation to generate feature maps This part of the formula is expressed as follows:

[0108] f=δ(F 1 ([z h ,z w ]))

[0109] Among them, [·,·] is the cascade operation along the spatial dimension, z h is the average pooling result of input X at height h, z w is the average pooling result of input X at and width w, δ is the nonlinear activation function, F 1 is a 1*1 convolution transformation function, f is an intermediate feature map encoding horizontal and vertical spatial information, and r is the reduction rate used to control the block size.

[0110] Then, f is split along the spatial dimension into and Then use 1×1 convolution to increase the dimension, and then combine the sigmoid activation function to get the final attention vector and The specific calculation formula is:

[0111]

[0112] Among them, y c (i,j) is the output of the CA attention module, x c (i,j) is the initial input of the CA attention module, is the output in the h direction after being processed by the CA attention module, is the output in the w direction after being processed by the CA attention module.

[0113] Through the encoding setting of this step, our coordinate attention can more accurately locate the exact position of the object of interest, thereby helping the entire model to better identify the target object.

[0114] Furthermore, the calculation formula for the output obtained by adding residual connections is as follows:

[0115] Output=Input+F(Input)

[0116] Among them, Input is the feature map of the first convolution structure, F(Input) is the feature map output after the nonlinear transformation (such as convolution, activation function, etc.) of the first convolution structure, and Output is the final output feature map of the first convolution structure.

[0117] Specifically, the second convolution structure is used when the expansion factor is not 1. The input first passes through a 3x3 expansion convolution layer (with a step size of 1 or 2), followed by batch normalization and SiLU activation, and then passes through a 1x1 projection convolution layer (with a step size of 1) and batch normalization. After passing through the Dropout layer, the input is added with a residual connection to generate the output.

[0118] like Figure 2 As shown in FIG. 1 , the backbone feature extraction network is composed of six FusedConv modules, namely, FusedConv1 to FusedConv6. The structure is the same as the above two different convolution structures, but the parameter settings are different.

[0119] As an example, the specific steps of the backbone feature extraction network processing include:

[0120] The feature map size of the encoder output is 56*56*128, which is input into the FusedConv1 layer. The FusedConv1 layer includes: an expanded convolution with a convolution kernel size of 3*3, a step size of 1, and an expansion factor of 3, a batch normalization BN layer, and a SiLU activation function. After the convolution, the feature map size becomes 56*56*512; a projection convolution layer with a convolution kernel size of 1x1, through a batch normalization BN layer, after the convolution, the feature map size becomes 56*56*128; a Dropout layer, through which branches are randomly dropped, after passing through this layer, the feature map size becomes 56*56*128; the feature map input to the FusedConv1 layer and the output after the Dropout layer are added as the output of the FusedConv1 layer, and the output feature map size is 56*56*128.

[0121] The output feature map size of the FusedConv1 layer is 56*56*128, which is input into the FusedConv2 layer, which includes: an expanded convolution with a convolution kernel size of 3*3, a stride of 1, and an expansion factor of 3, a batch normalization BN layer, and a SiLU activation function. After the convolution, the feature map size becomes 56*56*512; a projection convolution layer with a convolution kernel size of 1x1, through a batch normalization BN layer, after the convolution, the feature map size becomes 56*56*128; a Dropout layer, through which branches are randomly dropped, after passing through this layer, the feature map size becomes 56*56*128; the feature map input to the FusedConv2 layer and the output after the Dropout layer are added as the output of the FusedConv2 layer, and the output feature map size is 56*56*128.

[0122] The output feature map size of the FusedConv2 layer is 56*56*128, which is input into the FusedConv3 layer, which includes: an expanded convolution with a convolution kernel size of 3*3, a stride of 1, and an expansion factor of 3, a batch normalization BN layer, and a SiLU activation function. After the convolution, the feature map size becomes 56*56*512; a projection convolution layer with a convolution kernel size of 1x1, through a batch normalization BN layer, after the convolution, the feature map size becomes 56*56*128; a Dropout layer, through which branches are randomly dropped, after which the feature map size becomes 56*56*128; the feature map input to the FusedConv3 layer and the output after the Dropout layer are added as the output of the FusedConv3 layer, and the output feature map size is 56*56*128.

[0123] The output feature map size of the FusedConv3 layer is 56*56*128, which is input into the FusedConv4 layer, which includes: an expanded convolution with a convolution kernel size of 3*3, a stride of 1, and an expansion factor of 2, a batch normalization BN layer, and a SiLU activation function. After the convolution, the feature map size becomes 56*56*256; a projection convolution layer with a convolution kernel size of 1x1, through a batch normalization BN layer, after the convolution, the feature map size becomes 56*56*128; a Dropout layer, through which branches are randomly dropped, after which the feature map size becomes 56*56*128; the feature map input to the FusedConv4 layer and the output after the Dropout layer are added as the output of the FusedConv4 layer, and the output feature map size is 56*56*128.

[0124] The output feature map size of the FusedConv4 layer is 56*56*128, which is input into the FusedConv5 layer, which includes: an expanded convolution with a convolution kernel size of 3*3, a stride of 1, and an expansion factor of 2, a batch normalization BN layer, and a SiLU activation function. After the convolution, the feature map size becomes 56*56*256; a projection convolution layer with a convolution kernel size of 1x1, through a batch normalization BN layer, after the convolution, the feature map size becomes 56*56*128; a Dropout layer, through which branches are randomly dropped, after which the feature map size becomes 56*56*128; the feature map input to the FusedConv5 layer and the output after the Dropout layer are added as the output of the FusedConv5 layer, and the output feature map size is 56*56*128.

[0125] The output feature map size of the FusedConv5 layer is 56*56*128, which is input into the FusedConv6 layer. This layer includes: a CA module, after which the feature map size becomes 56*56*128; a projection convolution layer with a convolution kernel size of 1x1, after which the feature map size becomes 56*56*128 through a batch normalization BN layer; a Dropout layer, through which branches are randomly dropped, after which the feature map size becomes 56*56*128; the feature map input to the FusedConv6 layer and the output after the Dropout layer are added as the output of the FusedConv6 layer, and the output feature map size is 56*56*128.

[0126] Step S33, the decoder upsamples the output feature map of the backbone extraction network, and uses the upsampled output to predict the grasping information of the target, where the grasping information includes the grasping position, the sine value of the grasping angle, the cosine value of the grasping angle, and the grasping width.

[0127] Furthermore, the decoder, like the encoder, is only stored in the industrial computer as part of the program.

[0128] The decoder convolution layer still uses the decoder convolution layer of the generative residual convolutional neural network model without replacement or modification. In the original generative residual convolutional neural network model, the generative residual convolutional neural network model removes BatchNorm and ReLU in the last convolutional layer. The decoder in this embodiment has the same setting as the last layer of the encoder of the generative residual convolutional neural network model.

[0129] See also Figure 5 ,The structure of the decoder consists of three layers of transposed convolutional layers ConvT2D with different convolution kernel sizes, and the first and second layers are activated using batch normalization BN layer and ReLU activation function;

[0130] As an example, the specific steps of processing three layers of transposed convolution layers ConvT2D with different convolution kernel sizes include:

[0131] The first transposed convolution layer receives the feature map 56*56*128 input by the backbone extraction network and performs upsampling. This layer includes: a transposed convolution layer ConvT2d with a convolution kernel size of 4*4 and a stride of 2. After the batch normalization BN layer and SiLU activation, the feature map size becomes 112*112*64 after the transposed convolution.

[0132] The feature map size of the output of the first transposed convolution layer becomes 112*112*64, and the feature map is used as the input of the second transposed convolution layer. This layer includes: a transposed convolution layer ConvT2d with a convolution kernel size of 4*4 and a stride of 2. After being activated by the batch normalization BN layer and SiLU, the feature map size becomes 224*224*32 after the transposed convolution.

[0133] The feature map size output by the second transposed convolution layer becomes 224*224*32, and the feature map is used as the input of the third transposed convolution layer. This layer includes: a transposed convolution layer ConvT2d with a convolution kernel size of 9*9 and a stride of 1. After the transposed convolution, the feature map size becomes 224*224*32.

[0134] Among them, the feature map after the third layer of transposed convolution is used to output the grasping information of the predicted target. Specifically, the output of the model, the grasping position, the sine value of the grasping angle, the cosine value of the grasping angle and the grasping width are obtained through four parallel convolution layers Conv2d with a convolution kernel size of 2*2.

[0135] Among them, the calculation rule of the ReLU activation function is:

[0136] ReLU=max(0,x)

[0137] Among them, x is the input of the activation function.

[0138] The output of the ReLU activation function is the larger value between 0 and x.

[0139] Step S4, using the improved loss calculation method to calculate each output loss and the total loss, and obtain the difference between the actual feasible crawling information.

[0140] Specifically, the original loss calculation method is to simply add the four calculated losses, which is improved to weighting the four outputs. The loss of each output and the total loss are calculated through the improved loss calculation method to obtain the difference between the actual feasible crawling information.

[0141] Going a step further, the four output losses before improvement p 、loss cos 、loss sin , and loss w The function SmoothL1 is used to calculate the average value of n training samples.

[0142] The total loss is calculated as follows:

[0143] Loss all =loss p +loss sin +loss cos +loss w

[0144] Among them, loss p is the position loss, loss cos is the cosine loss of the rotation angle, loss sin is the rotation angle sine loss, loss w is the width loss, loss all is the total loss,

[0145] The original calculation method of the total loss is to simply add the four calculated losses, which cannot reflect the importance of the four outputs. Therefore, the four outputs are weighted to enhance the importance of the grasping position.

[0146] Original calculation loss w When the predicted width is greater than the actual width, the calculated loss is the same as when the predicted width is less than the actual width, and it is only related to the absolute value of the difference between the predicted width and the actual width. It is unable to reflect the negative impact of the grasping width being less than the actual width on the grasping, so the calculation of the loss is corrected, and the loss value is increased when the grasping width is less than the actual width, so that the width loss lossw Try to be on the side larger than the actual width to reduce the negative impact on grasping. Through two improvements in the loss function calculation, the loss calculation is more reasonable and the loss function converges faster. After using the improved loss function calculation method, the detection accuracy will be higher with the same number of training times.

[0147] Specifically, the specific steps of calculating each output loss and the total loss in the improved loss calculation method include:

[0148] Step S41: The predicted value output by the robot arm target detection network model is G c = {G p ,G sin ,G cos ,G w}, the predicted value consists of four parts, namely the grasping position G p , Sine value of the grab angle G sin , grab angle cosine value G cos , Grab width G w .

[0149] Step S42, the actual value of the capture obtained from the image annotation is Regression loss estimation of the graspable area’s grasp position, angle, and width: loss p 、loss cos 、loss sin , and loss w Except for the width loss, the other losses use the SmoothL1 function to calculate the average value of n training samples.

[0150] Among them, the formula of SmoothL1 is as follows:

[0151]

[0152] Among them, G c is the predicted value output by the model network, Indicates the actual value of the grab target.

[0153] Among them, z i The definition function calculation formula is:

[0154]

[0155] Among them, z i is the loss value of the i-th sample, G ic is the predicted output of the ith sample, is the actual output of the ith sample.

[0156] Step S43, for the width loss loss w, a certain bias is made in the calculation of the loss, the loss value is increased when the width is smaller than the actual width, and the loss calculation is adjusted based on SmoothL1. The adjusted z i As shown in the following formula: where η is a constant greater than 1. In this experiment, η is set to 1.2.

[0157]

[0158] Among them, z i is the loss value of the i-th sample, G ic is the predicted output of the i-th sample, is the actual output of the i-th sample.

[0159] Step S44, calculate the total network loss Loss all , the calculation formula is as follows:

[0160] Loss all =αloss p +β(loss sin +loss cos )+γloss w

[0161] Among them, Loss all is the total network loss, loss p is the position loss, loss cos is the cosine loss of the rotation angle, loss sin is the rotation angle sine loss, loss w is the width loss, α, β, γ are all constants, α is the weight of the position loss, β is the weight of the rotation angle cosine loss and the rotation angle sine loss, and γ is the weight of the width loss.

[0162] Furthermore, the weight α of the grasping position loss is set to the largest constant, the weight γ of the grasping width loss is set to a smaller constant, and the weight 2β of the grasping angle loss is between the two. Considering that both the sine value and the cosine value reflect the grasping angle, the weights of the two are set to the same, which is β. In this experiment, the weights are set to α=1.15, β=0.55, and γ=0.95.

[0163] Step S5, using the result of loss calculation to update the weight of the model to obtain the optimal training weight.

[0164] Specifically, the results of the loss calculation are used to update the weights of the model, thereby improving the matching accuracy between the predicted grasping information output value and the actual grasping information in step S4. The verification set images are input into the robotic arm grasping target detection network model after the weights are updated. The performance of the current model is tested according to the performance indicators on the verification set and compared with the performance of the current optimal model. The training weights of the model with higher performance are recorded as the optimal training weights at the current stage.

[0165] Furthermore, for all deep models, the training process can be summarized as

[0166] Forward propagation: pass the input data through the network to calculate the output.

[0167] Calculate loss: Calculate the loss value based on the model's predicted output and the true label.

[0168] Backpropagation: Calculate the gradient of the loss function with respect to each parameter.

[0169] Parameter update: Update the model parameters according to the optimization algorithm and the calculated gradients.

[0170] The loss function is used to measure the gap between the model's predicted value and the true value. The model's performance is then improved through back propagation and parameter updates.

[0171] The best model is selected as the final model based on the performance indicators on the validation set, and the corresponding weight setting weights of each convolution are the optimal training weights.

[0172] For the measurement of detection effect, angle difference and Jaccard index are two commonly used criteria for evaluating grasping performance. According to the angle difference standard, if the angle deviation between the predicted grasping area and its corresponding label is less than 30°, it is judged as a successful grasp. According to the Jaccard index standard, if the overlap between the predicted grasping area and the marked grasping area exceeds 25%, it is considered a successful grasp. The mathematical expression of the Jaccard index is as follows:

[0173]

[0174] Among them, g o is the predicted grasping area, g t is the actual grab area value.

[0175] Furthermore, the intersection of the predicted grasping region and the grasping region label is given by g p ∩g t , and the union of the predicted grasping area and the grasping area label is represented by g p ∪g t express.

[0176] Step S6, inputting the test set into the trained robot arm grasping target detection network model to obtain the detection results of accuracy and detection speed.

[0177] Furthermore, the weights in the robotic arm grasping target detection network model are updated using the final optimal training weights, and the test set is input into the robotic arm grasping target detection network model after the training weights are changed to obtain the robotic arm grasping target detection results. The test set is a picture randomly selected from the target training set.

[0178] See also Figure 6 , the output detection results, from top to bottom, are the generated grasping quality map Quality, grasping angle map Angle, grasping width map Width and target object grasping frame map Grasp. It can be seen that the proposed robot arm grasping target detection network model obtains clear feature information and can generate a grasping posture with higher confidence in the quality heat map. The success rate of the robot arm grasping target detection network model and the generative residual convolutional neural network model on the test set are compared. The comparison results are shown in Table 1.

[0179] Table 1 Test set test results statistics

[0180]

[0181] According to Table 1, the improved robot arm grasping target detection network model achieved an accuracy of 95.5% in 177 test images, which is 3.4% higher than the generative residual convolutional neural network model and shows a faster detection speed. The improved model achieves improved accuracy and detection speed for grasping target detection. The above results fully verify the effectiveness of the method of the present invention.

[0182] In the experimental process of this embodiment, the implementation case selected the generative residual convolutional neural network model before improvement and the robot arm grasping target detection network model of this article for training and comparison under the same conditions.

[0183] The model training and testing configuration is Windows 11 system, combined with Pytorch deep learning framework as the software platform. The experimental equipment configuration includes 12th Gen Intel (R) Core (TM) i9-12900KF processor and NVIDIA GeForce RTX3080Ti graphics card. All experiments are implemented in PyCharm 2022.1.3 version and Cuda 11.6 environment.

[0184] The sample images used in the implementation case are made from color images and corresponding depth images taken by the depth camera KinectV2, and are annotated according to the format of the Cornell dataset. The basic facts of each image are annotated with grab rectangles, representing the grab possibility of each object.

[0185] During the model training process, the training parameters were set to 50 epochs, and each epoch processed 1000 batches, each batch containing 8 samples. The sample size was adjusted to 224×224 when input into the model to meet the input requirements of the network. The target dataset was split into a training set and a test set in a ratio of 8:2, and some sample images were randomly selected from the target dataset as the test set. The training set was then augmented and expanded when input into the model.

[0186] Table 2 Experimental environment

[0187]

[0188] A first embodiment of the present invention provides a method for detecting a target grasped by a robot arm, comprising:

[0189] Collect the target object image to obtain the initial data set; pre-process the initial data set by data enhancement to obtain the target data set; divide the target data set into a training set and a validation set, and randomly extract some sample images from the target data set as a test set; use the generative residual convolutional neural network model as the benchmark model to construct an improved generative residual convolutional neural network model as a robot arm grasping target detection network model; input the training set into the robot arm grasping target detection network model for model training, and improve the parameters of the robot arm grasping target detection network model based on the training results to obtain the trained robot arm grasping target detection network model; input the validation set into the trained robot arm grasping target detection network model for model evaluation; input the test set into the trained robot arm grasping target detection network model to obtain the robot arm grasping target detection result. The present invention solves the problem of excessive early calculation of the network model by replacing the encoder of the generative residual convolutional neural network model with an encoder constructed by a hole convolution module, and realizes the lightweight of the model. By building a backbone feature extraction network to replace the backbone extraction network of the generative residual convolutional neural network model, richer feature information is extracted, and the model feature extraction capability is improved. By correcting the original loss calculation method and performing weighted calculation on each part of the total loss, the accuracy of the predicted capture information is improved, and the accuracy and speed of the capture target detection are improved.

[0190] Second implementation method:

[0191] like Figure 8As shown, the second embodiment of the present invention provides a robot arm grasping target detection system, which includes: a data set acquisition module 201, a data set preprocessing module 202, an improved modeling module 203, a weight training module 204, a weight updating module 205, and a detection result generation module 206.

[0192] Specifically, the data set acquisition module 201 is used to acquire the target object image and obtain the initial data set; the data set preprocessing module 202 is used to preprocess the initial data set by data enhancement to obtain the target data set; the improved modeling module 203 is used to construct an improved generative residual convolutional neural network model as a robotic arm grasping target detection network model with the generative residual convolutional neural network model as the benchmark model; the weight training module 204 is used to input the training set into the robotic arm grasping target detection network model for model training, and improve the parameters of the robotic arm grasping target detection network model based on the training results to obtain the trained robotic arm grasping target detection network model; the weight updating module 205 is used to input the verification set into the trained robotic arm grasping target detection network model for model evaluation; the detection result generation module 206 is used to input the test set into the trained robotic arm grasping target detection network model to obtain the robotic arm grasping target detection result.

[0193] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the first embodiment.

[0194] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0195] A third embodiment of the present invention relates to a network side server, such as Fig. 9 As shown, it includes at least one processor 302; and a memory 301 that is communicatively connected to the at least one processor 302; wherein the memory 301 stores instructions that can be executed by the at least one processor 302, and the instructions are executed by the at least one processor 302 so that the at least one processor 302 can execute the above-mentioned data processing method.

[0196] The memory 301 and the processor 302 are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors 302 and the memory 301 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor 302 is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor 302.

[0197] The processor 302 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 301 can be used to store data used by the processor 302 when performing operations.

[0198] The fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a method for detecting a target grasped by a robot arm in the first embodiment.

[0199] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0200] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

[0201] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting a target grasped by a robot arm, characterized in that: include: S1, collect the target material image to obtain the initial data set; S2, preprocess the initial data set using data enhancement to obtain the target data set; S3, using the generative residual convolutional neural network model as the baseline model, an improved generative residual convolutional neural network model is constructed as the robot arm grasping target detection network model; S4, using the improved loss calculation method to calculate each output loss and the total loss, and obtain the difference between the actual feasible crawling information; S5. Use the loss calculation result to update the model weight to obtain the optimal training weight; S6, inputting the test set into the trained robot arm grasping target detection network model to obtain the robot arm grasping target detection result.

2. A method for detecting a target grasped by a robot arm according to claim 1, characterized in that: In S3, an improved generative residual convolutional neural network model is constructed with the generative residual convolutional neural network model as the benchmark model as the robot arm grasping target detection network model, including: Step S31, construct a new encoder module to replace the original encoder, and input the training set image into the encoder for downsampling and preliminary feature extraction; Step S32, building a backbone feature extraction network to replace the original 5-layer residual module, inputting the feature map output by the encoder into the backbone feature extraction network to perform deep feature extraction; Step S33, the decoder upsamples the output feature map of the backbone extraction network, and uses the upsampled output to predict the target's grasping information including the grasping position, the sine value of the grasping angle, the cosine value of the grasping angle, and the grasping width.

3. A method for detecting a target grasped by a robot arm according to claim 2, characterized in that: The encoder consists of three layers of dilated convolutional layers DconvBNact, which process the input graphic parameters step by step and then output them. The dilated convolutional layer DconvBNact includes: a dilated convolution DilatedConv2d with a convolution kernel size of 3×3, a step size of 1 or 2, and dilation factors of [1, 2, 3] respectively, a data batch normalization BN layer and a SiLU activation function. The dilated convolutional layer DconvBNact is used to downsample the input captured target image and perform preliminary feature extraction.

4. A method for detecting a target grasped by a robot arm according to claim 3, characterized in that: The calculation rule of the data batch normalization BN layer is: Among them, x i The i-th dimension sample data of the batch data output by the convolution DilatedConv2d, μ B For x i The average value of For x i The variance of For x i After normalization, ε is a positive number much smaller than 1 to avoid the denominator being zero. i for The linear transformation output of , γ and β are learnable parameters for scaling and translation The calculation rule of the SiLU function is: f(x)=x*σ(x) Among them, f(x) is the output after SiLU activation function processing, x is the feature map input to SiLU activation function, and σ(x) is the sigmoid function.

5. A method for detecting a target grasped by a robot arm according to claim 4, characterized in that: The inputting of the training set images into the encoder for downsampling and preliminary feature extraction comprises: Step S311, the input image size is 224*224*4, and passes through the first layer of dilated convolution layer, which includes: DilatedConv2d with a convolution kernel size of 3*3, a step size of 1, a dilation factor of 1, a batch normalization BN layer, and a SiLU activation function; the dilated convolution layer is used to perform preliminary downsampling on the input target object image, and after downsampling, the feature map size of the output of this layer is 224*224*32. Step S312, through the second layer of dilated convolution layer, which includes: DilatedConv2d with a convolution kernel size of 3*3, a step size of 2, a dilation factor of 2, a batch normalization BN layer and a SiLU activation function; the dilated convolution layer is used to further downsample the input target object image, and after downsampling, the feature map size of the output of this layer is 112*112*64. Step S313, through the third dilated convolution layer, which includes: a DilatedConv2d with a convolution kernel size of 3*3, a step size of 2, and a dilation factor of 3, a batch normalization BN layer, and a SiLU activation function; the dilated convolution layer is used to downsample the input target object image, and after downsampling, the feature map size output by this layer is 56*56*128.

6. A method for detecting a target grasped by a robot arm according to claim 2, characterized in that: The backbone feature extraction network includes two different convolution structures according to different expansion factors. The first convolution structure is used when the expansion factor is 1. The input feature map extracts spatial position information through the CA attention module, then passes through a 1x1 projection convolution layer and batch normalization, and then passes through the Dropout layer, and then performs an additive residual connection with the input to obtain the output; the second convolution structure is used when the expansion factor is not 1. The input first passes through a 3x3 expansion convolution layer, then batch normalization and SiLU activation, then passes through a 1x1 projection convolution layer and batch normalization, and then passes through the Dropout layer, and then performs an additive residual connection with the input to generate the output.

7. A method for detecting a target grasped by a robot arm according to claim 2, characterized in that: The structure of the decoder consists of three layers of transposed convolutional layers ConvT2D with different convolution kernel sizes, and the first and second layers use batch normalization BN layers and ReLU activation functions.

8. A method for detecting a target grasped by a robot arm according to claim 1, characterized in that: In S4, the improved loss calculation method calculates each output loss and the total loss including: Step S41: The predicted value output by the robot arm target detection network model is G c = {G p ,G sin ,G cos ,G w }, the predicted value consists of four parts, namely the grasping position G p , Sine value of the grab angle G sin , grab angle cosine value G cos , Grab width G w ; Step S42, the actual value of the capture obtained from the image annotation is Regression loss estimation of the graspable area’s grasp position, angle, and width: loss p 、loss cos 、loss sin , and loss w , except for the width loss, the other losses i Use the SmoothL1 function to calculate the average value of n training samples; Step S43, for the width loss loss w , make a certain bias in the calculation of the loss, increase the loss value when the width is smaller than the actual width, and adjust the loss calculation based on SmoothL1; Step S44, calculate the total network loss Loss all .

9. A method for detecting a target grasped by a robot arm according to claim 1, comprising: In S5, the weights of the model are updated using the result of the loss calculation to obtain the optimal training weights, including: inputting the verification set images into the robotic arm grasping target detection network model after the weights are updated, detecting the performance of the current model according to the performance indicators on the verification set, and comparing it with the performance of the current optimal model, and recording the training weights of the model with higher performance as the optimal training weights at the current stage.

10. A robot arm grasping target detection system, characterized in that: Applied to the method for detecting a target grasped by a robot arm as described in claim 1, the system comprises: A data set acquisition module is used to acquire images of target objects and obtain an initial data set; The data set preprocessing module is used to preprocess the initial data set by using data enhancement to obtain the target data set; An improved modeling module is used to construct an improved generative residual convolutional neural network model as a robot arm grasping target detection network model based on the generative residual convolutional neural network model as a baseline model; A weight training module is used to input the training set into the robot arm grasping target detection network model for model training, and improve the parameters of the robot arm grasping target detection network model based on the training results to obtain the trained robot arm grasping target detection network model; A weight updating module, used for inputting the verification set into the trained robot arm grasping target detection network model for model evaluation; The detection result generation module is used to input the test set into the trained robot arm grasping target detection network model to obtain the robot arm grasping target detection result.

Images