A deep learning-based mechanical arm grabbing method

Abstract

The application discloses a kind of based on deep learning's mechanical arm grabbing method, obtains through depth camera and obtains the RGB image with depth image of object with grabbing. The depth image is detected, and invalid pixel is obtained and reconstructs these data, so that neural network obtains more accurate grabbing pose. RGB image and depth are aligned, and RGB-D image is obtained, and the center part is cropped in alignment, and RGB-D image is obtained. The RGB-D image with the object with grabbing is input using the improved GGCNN2 network model trained in advance, and the grabbing pose information with the object with grabbing is generated. The grabbing pose under the base coordinate system of mechanical arm, that is, Cartesian coordinate system. Input grabbing information, control mechanical arm to carry out grabbing task. The application solves the problem of poor generalization and learning ability in the original network, so that the improved neural network has better performance in obtaining the grabbing pose with the object with grabbing, and is more practical in the field of high-precision grabbing.

Description

Technical Field

[0001] This invention relates to a method for a deep learning-based robotic arm to grasp unknown objects, belonging to the field of intelligent robots. Background Technology

[0002] A key characteristic of intelligent robots is their ability to perceive and interact with their environment. Among the many functions of robots, grasping is the most basic and crucial. In industrial production, robots perform numerous heavy grasping and placement tasks daily. Home robots, designed to provide convenience for the elderly and disabled, also primarily focus on routine grasping tasks. Therefore, endowing robots with perceptual abilities and enabling them to better perform grasping tasks using perceived information has always been a significant research area in robotics and machine vision.

[0003] Visually-aware robotic grasping systems typically consist of grasp detection, grasp planning, and a control unit. To complete a grasping task, the robot needs to detect the object to be grasped beforehand. In the early stages of grasping, the object is usually placed in a simple and structured scene. The grasping scheme is formulated based on the object's geometry through mechanical analysis, often involving complex static and kinematic constraints requiring complex calculations. With the successful application and continuous development of deep learning in instance segmentation and recognition, it has been widely applied to robot grasp detection. One approach is to combine convolutional neural networks (CNNs) with a camera to identify the structural features of the object to be grasped and evaluate candidate grasping poses. This approach generally provides a system that allows the robot to grasp various shapes, where the CNN is used to train the shape detection model. This method generates grasping points by perceiving the shape of the object being grasped, but it is not effective for grasping irregularly shaped objects.

[0004] Another approach is to directly generate grasping schemes and use learning methods to evaluate grasping quality. Typically, generating a grasping scheme first requires detecting and identifying the object's position, then employing traditional geometric analysis methods to finally formulate a grasping plan. Geometric analysis usually involves significant computation. Therefore, deep learning networks can be used to directly train grasping detection, where the grasping scheme is derived from the image, enabling end-to-end grasping. This approach proposes a model that, given an input image, outputs the robot's optimal grasping pose. This model uses convolutional neural networks to extract features from the scene and then uses convolutional neural networks to predict the grasping configuration for the object of interest. For example, a real-time grasping synthesis method for closed-loop grasping is the Generative Grasping Convolutional Neural Network (GGCNN). This system predicts the grasping quality and pose at each pixel. However, GGCNN and its subsequent upgrade, GGCNN2, perform poorly on grasping test data.

[0005] The performance of both the Cornell and Jacquard datasets and actual robotic arm grasping needs improvement. Summary of the Invention

[0006] To address the performance issues of GGCNN and GGCNN2 in both test sets and actual grasping scenarios, this invention proposes a deep learning network for face-based robotic arm grasping pose estimation by modifying the GGCNN2 network. This makes the GGCNN2 model more practical in the field of high-precision grasping. Furthermore, this invention proposes a robotic arm grasping method based on this network, which is applied to robotic arms equipped with depth cameras.

[0007] The main technical solution adopted in this invention is a robotic arm grasping method based on deep learning, which includes:

[0008] S1. Before the robotic arm starts grasping, adjust the initial state of the robotic arm, that is, the initial working point, so that the depth camera is perpendicular to a fixed height above the XOY plane, and obtain the RGB image and depth image of the object to be grasped through the depth camera.

[0009] S2. Detect invalid pixels in the depth image and reconstruct these data to help the neural network obtain a more accurate grasping pose. Simultaneously, align the RGB image with the depth map to obtain an RGB-D image, and then crop the center portion of the aligned image to obtain a 300×300 pixel RGB-D image.

[0010] S3. The pre-trained improved GGCNN2 network model is used as input to the RGB-D image of the object to be grasped, and the output is three 300×300 pixel grasping information images, namely the pixel-level grasping quality image, grasping angle image, and grasping width image.

[0011] S3. Based on the generated grasping quality map, obtain the pixel with the highest grasping quality, and obtain its corresponding grasping angle and width in the grasping angle map and grasping width map, thereby generating the grasping pose information of the object to be grasped.

[0012] S4. The obtained grasping pose information is transformed into coordinates through hand-eye calibration to obtain the grasping pose in the robot arm base coordinate system, i.e., the Cartesian coordinate system.

[0013] S5. Input the grasping information and control the robotic arm to perform the grasping task.

[0014] The improved GGCNN2 network enhances the generalization and learning capabilities of the existing GGCNN2 network structure by adding an IBN module. Furthermore, it incorporates ECA_ResNet, a powerful channel attention module combined with residual modules in the residual network, to improve model performance. This results in higher accuracy in obtaining the pose information of the object to be grasped, a higher success rate in actual grasping tasks, and greater practical value.

[0015] Optionally, in step S2, the specific operation method of the depth map repair technique proposed in this method is as follows:

[0016] The first step is to obtain a depth map of the object to be crawled, which may contain invalid information;

[0017] The second step is to generate a depth map of the object to be grasped with a 2D gradient;

[0018] The third step is to perform thresholding on the 2D gradient to generate a binary mask;

[0019] The fourth step is to fill the binary mask with water and at the same time generate a mask for all invalid points in the depth image of the object to be captured.

[0020] The fifth step is to dilate the generated mask;

[0021] The sixth step is to provide the generated mask to the OpenCV image inpainting function, which can attempt to infer and reconstruct invalid data from the pixels.

[0022] Optionally, the improved GGCNN2 network model includes:

[0023] Extract the shallow features of the image from the convolutional part, the dilated convolutional part, the ECA_ResNet part, and the final output part;

[0024] The convolutional part consists of four convolutional layers and two max-pooling layers. First, there are two convolutional layers and one max-pooling layer. The two convolutional layers have the same parameters: first a 5x5 convolutional module, then an IBN module, and finally a ReLU function. Next are two more convolutional layers and a max-pooling layer with identical parameters; the number of filters in the convolutional part is 16.

[0025] The dilated convolution part consists of two dilated convolutional layers with identical parameters: first, it passes through a 5x5 dilated convolutional module with dilation=2, then through a ReLU activation function, then through a 5x5 dilated convolutional module with 32 dilation, and finally through the same ReLU activation function. The total number of dilated convolutional filters is 32.

[0026] The ECA_ResNet part consists of six concatenated ECA_ResNet modules. The ECA_ResNet parameters include: first, a 3x3 convolutional module, followed by a BN module, then a ReLU activation function, another 3x3 convolutional module and a BN module, and finally the ECANet. The total number of filters in the ECA_ResNet is 32.

[0027] The output section consists of two convolutional layers: the first convolutional layer consists of a 3x3 convolutional module followed by a ReLU activation function. Next, there are three linearly mapped convolutional layers, each consisting of a 3x3 convolutional module. These three linearly mapped convolutional layers sequentially map the output capture success rate, capture angle, and capture width, respectively.

[0028] Optionally, when training the improved GGCN2 network, the following criteria can be used to measure grasping accuracy:

[0029] (1) The difference between the predicted grasping angle and the actual grasping angle is less than or equal to the specified threshold (30°).

[0030] (2) The union Iou between the predicted crawl rectangle and the actual crawl rectangle is greater than 25%.

[0031] The formula for the ratio of parallels: ;

[0032] in, This represents the rectangle to be predicted and captured. This represents the actual captured rectangle, and the cross ratio is calculated as the ratio of the intersection to the union of two rectangles.

[0033] Compared with existing technologies, this invention not only proposes a method for reconstructing invalid pixels in depth maps, which can reduce the impact of invalid data in depth maps and effectively form grasping information, but also adds IBN and ECA_ResNet modules to the structure of GGCNN2, solving the problem of poor generalization and learning ability in the original network. This makes the improved neural network perform better in obtaining the grasping pose of the object to be grasped, and more practical in the field of high-precision grasping. Attached Figure Description

[0034] Figure 1 The flowchart for implementing this method.

[0035] Figure 2 Invalid information technology flowcharts in the reconstruction depth map of this application.

[0036] Figure 3 Grab point definition diagram.

[0037] Figure 4 A partial visualization of the Jacquard dataset.

[0038] Figure 5 Original network structure diagram of GGCNN2.

[0039] Figure 6 The improved GGCNN2 network structure diagram.

[0040] Figure 7 IBN module structure diagram.

[0041] Figure 8 ECA attention mechanism network diagram.

[0042] Figure 9 ECA_ResNet structure diagram.

[0043] Figure 10 Improved GGCNN2 generation of grasp quality, grasp angle, and grasp width maps. Detailed Implementation

[0044] The method will be described in detail below with reference to the accompanying drawings and embodiments.

[0045] This invention employs a deep learning-based robotic arm grasping method, which utilizes an IBN module and an ECA (Extra-Attention and Residual Network) to improve the grasping generative convolutional neural network (GGCNN2), thereby enhancing the accuracy of the model in generating the optimal grasping pose. Experimental results show that the improved GGCNN2 network achieves an accuracy of 94.3% on the Cornell dataset with only depth map input and 96.2% with RGB-D input, representing a significant improvement over the original network's 80% accuracy. Furthermore, on the Jacquard dataset, the proposed model achieves an accuracy of 91.6% with only depth map input and 93.4% with RGB-D input, also a significant improvement over the original network's 84% ​​accuracy with only depth map input.

[0046] according to Figure 1 The specific implementation method of the present invention is as follows:

[0047] The first stage involves adjusting the robotic arm to its initial state, i.e., adjusting it to its initial pose, and placing the depth camera at a known height above the vertical XOY plane. The depth camera is then used to obtain the RGB image and depth image of the object to be grasped.

[0048] The hand-eye calibration of the depth camera and the robotic arm can be performed by either having the eye on the hand or having the eye outside the hand.

[0049] The second stage begins with preprocessing the depth map. The purpose of this preprocessing is to remove invalid pixels from the depth map and reconstruct these data for more accurate image capture. Finally, the RGB values ​​are aligned with the depth map to obtain an RGB-D image, which is then cropped from the center to obtain a 300x300 pixel RGB-D image. Figure 2 As shown, the steps for preprocessing the depth map are as follows:

[0050] S1. Obtain a depth map that may contain invalid information;

[0051] S2, generate a depth map with a 2D gradient;

[0052] S3. Threshold the gradient to generate a binary mask;

[0053] S4. Fill the mask with water and generate a mask that only contains invalid points in the depth image.

[0054] S5. Expand the generated mask;

[0055] S6. Provide the generated mask to the OpenCV image inpainting function, which can attempt to infer and reconstruct invalid data from the pixels.

[0056] The third stage involves inputting the RGB-D image of the object to be grasped into the pre-trained model, which outputs three 300x300 pixel grasping information images, including a grasping quality image, a grasping angle image, and a grasping width image. The pixel with the highest grasping quality is selected as the grasping point, and the corresponding grasping angle and grasping width are obtained from the other two output images to obtain grasping information.

[0057] The fourth stage involves converting the pose of the point to be grasped in the camera coordinate system to the pose in the robot arm's base coordinate system (Cartesian coordinate system) based on the grasped information and through the transformation of the hand-eye calibration coordinate system.

[0058] The fifth stage involves controlling the robotic arm to perform the grasping action based on the grasping information. The grasping information includes the coordinates in the robotic arm's base coordinate system, the grasping angle, and the grasping width.

[0059] The improved GGCNN2 network described above is based on the GGCNN2 network structure and incorporates the IBN and ECA_ResNet modules to increase the model's generalization and learning capabilities, thereby enhancing its performance and resulting in higher grasping accuracy and greater practical value during robotic arm grasping.

[0060] To better understand the present invention, the following description is provided in conjunction with the accompanying drawings.

[0061] Crawling solution based on GGCNN2

[0062] The gripper of the robotic arm extends in a direction perpendicular to the xy plane, which is planar grasping, using the formula... Indicates the grasping posture, where The position of the gripper at the center of the Cartesian coordinate system Given the direction of rotation of the gripper around the z-axis, the required gripper width is... , This represents the confidence level for the grasping posture. A grasping diagram is shown below. Figure 3 As shown.

[0063] In a depth image with height H and width W In the middle, parameters can be captured using To describe, among which, It retrieves the pixel coordinates of the center in the image. This represents the rotation angle of the gripper along the z-axis in the camera coordinate system. Represents the grip width in image coordinates. This describes the quality of the fetching performed at each point (u,v).

[0064] Grasping posture in robot coordinate system (world coordinate system) Grasping posture can be determined from the depth map plane. It is obtained through the following formula:

[0065] (1)

[0066] in This represents the transformation matrix from the depth map plane coordinate system to the camera coordinate system (obtained from camera intrinsic parameter calibration). The transformation matrix from the camera coordinate system to the robot coordinate system (obtainable through hand-eye calibration).

[0067] Unlike methods that use candidate grasp points, GGCNN2 directly generates grasp poses for each pixel in the depth map, generating both the depth and the grasp pose. Figure 1 A three-channel grasping pose map of sample size is defined in the image as follows:

[0068] (2)

[0069] in and All , respectively corresponding to Figure 9 The image shows the capture quality map, capture angle map, and capture width map. The capture quality map describes the capture quality of each pixel in the depth map; the capture quality value is limited to [0,1], with values ​​closer to 1 indicating higher capture quality. After determining the highest capture quality, the optimal capture point in the image space can be obtained using the following definition: .

[0070] To obtain the complex mapping relationship M from the depth image I to the grasping image G, a convolutional neural network is constructed for approximation, where Mθ represents a neural network with weights θ. This is achieved through a series of inputs... Depth image set and corresponding Grab an image set and define a loss function that minimizes the squared error to train the model:

[0071] (3)

[0072] Graph G is captured through three sets of graphs. , We use Q to estimate the capture scheme for spatial point p (corresponding to pixel s). The values ​​in the figure represent the capture confidence for each corresponding pixel (u,v), which are mainly used to describe the capture quality of that pixel. The values ​​in the image range from 0 to 1. The closer the value is to 1, the better the capture quality of that pixel and the higher the capture success rate. The values ​​in the graph represent the grasping angle for each corresponding pixel (u,v), with the angle ranging from -π / 2 to π / 2. The values ​​in the W graph represent the grasping width of the end effector for each corresponding pixel (u,v).

[0073] Improved GGCNN2 Network Construction and Training

[0074] The improved GGCNN2 network described in this invention uses the Jacquard dataset, a large-scale grasping dataset created through simulation based on CAD models by researchers at the LIRIS laboratory of École Centrale Lyon, France. Because it eliminates the need for manual data collection, the Jacquard dataset is larger than the Cornell dataset, containing 50k images of 11k objects and over 1 million grasping labels. Some example images are shown below. Figure 4 As shown. Furthermore, since the Jacquard dataset has sufficient data for model training, this invention does not employ data augmentation for experiments on this dataset.

[0075] Each input image is scaled to 300x300 pixels before being fed into the network. Simultaneously, corresponding capture labels are encoded for training and learning. Specifically, using... The angle of the gripper is represented in the form of and the width is used to represent the opening and closing distance of the gripper. The center coordinates of the gripping frame are obtained by searching for the position of the maximum gripping mass, where the pixel value of the corresponding area is set to 1, and other pixels are set to 0. To eliminate the potential discontinuities and excessively high values ​​that may occur when using the original angle at ±π / 2 radians, the angle is decomposed into two vector components on the unit circle, producing values ​​in the range [-1, 1]. Since the corresponding grip is symmetrical around ±π / 2 radians, the two components sin(2π / ... ) and cos(2 These provide unique values ​​within the range [-π / 2, π / 2]. During training, a pre-trained model is obtained by training with RGB-D images.

[0076] The complete structure of GGCNN2 is as follows: Figure 5 As shown, since GGCNN2 has limited performance in recognition and grasping, this invention addresses this limitation. Figure 5 The network structure shown is modified as follows: Figure 6 As shown. The improvement process is described below.

[0077] Improved GGCNN2 network

[0078] The main idea behind the improved GGCNN2 network is to add IBN and ECA_ResNet modules to the network structure, thereby improving the accuracy of the original model in generating grasping poses and improving the accuracy of robotic arm grasping.

[0079] The IBN module structure is as follows: Figure 7 As shown, the IBN module is a combination of IN (Instance Normalization) and BN (Batch Normalization).

[0080] In general, the Batch Normalization (BN) module is used after convolutional layers and before non-linear activation functions. Each convolutional layer's input is the output of the previous layer, and each is influenced by the parameters of the previous layer. Therefore, each layer's input has a different data distribution, requiring the trained network model to continuously adapt to this data distribution at each layer. Furthermore, arbitrary initialization of network model parameters leads to significant differences in the magnitude of training data and the superposition of different data at each layer, making network training complex and difficult to converge. Normalizing the data distribution at each layer prevents the network model from spending time adapting to different data distributions at each layer, increases the network learning rate, and accelerates the convergence speed of network training. Additionally, BN ensures that image sample training is done in batches, meaning the network's output no longer depends on a single sample but on the entire batch. However, since each iteration of network training randomly selects a batch from the image samples, the network's learning results will not always follow a single direction. This, to some extent, gives BN a regularization effect in network training, preventing overfitting.

[0081] The formula for BN is described as follows: (4)

[0082] in, and These are the affine parameters learned during training, which make the data conform to the distribution required by the model. and These are the mean and standard deviation calculated for each feature channel of all images in the current training batch, and their definitions are as follows:

[0083] (5)

[0084] (6)

[0085] Instance Normalization (IN) is similar to Batch Normalization (BN) as shown in Equation 3-9. Their principle is also similar, except that IN normalizes each channel of a single input sample. IN can perform a form of style normalization by normalizing feature statistics, namely the mean and variance. In network models using BN, BN can also normalize a batch of samples centered on a single style; however, each sample may still have a different style. If you want to convert all images in a sample to the same style and eliminate differences between images, you can use IN instead of BN.

[0086] The formula for IN is described as follows: (7)

[0087] in and These are the affine parameters learned during training; using different values ​​can generate images with different styles. and The definition is as follows:

[0088] (8)

[0089] (9)

[0090] The parameters in the above formula have the same meaning as those in the BN formula, except that... and This represents the mean and standard deviation obtained for each feature channel of a single instance image.

[0091] ECANet proposes a local cross-channel interaction strategy without dimensionality reduction and an adaptive one-dimensional convolutional kernel size method. It utilizes one-dimensional convolutional layers to obtain more accurate attention information without reducing dimensionality. ECANet is a lightweight attention mechanism that significantly improves model performance with only a few additional parameters. Its network structure is as follows: Figure 8 As shown.

[0092] After one-dimensional convolution, the channel attention learned by the ECANet attention mechanism is represented by W(k). W(k) involves only k×C parameters, which is less than the number of parameters in SENet. Furthermore, it avoids the complete independence of different local cross-channel interaction information. Here, parameter k represents the number of neighbors participating in the prediction of that channel, i.e., the coverage of local cross-channel interaction. Each channel and its k neighbors are considered to capture cross-channel interaction information. In the experiments of this invention, based on the number of model parameters and computational speed, the one-dimensional convolution kernel k is set to 3. The calculation formula for W(k) is as follows:

[0093] (10)

[0094] The formula for calculating the shared weight of all channels in ECANet is as follows:

[0095] (11)

[0096] (12)

[0097] in, Indicates a channel. It is a passage The weight, Indicates and channel k adjacent channels, This represents a one-dimensional convolution operation with kernel k.

[0098] The ECA_ResNet module is formed by combining the residual modules in ECANet and ResNet. Figure 9 As shown. Considering that the accuracy of neural networks increases with the number of layers, but beyond a certain number of layers, gradient vanishing and dimensionality errors occur, leading to a decrease in accuracy. Therefore, using residual layers allows for better achievement of identity mappings through residual connections.

[0099] 1. Experimental Results and Analysis

[0100] The robotic arm grasping simulation experiment algorithm in this application uses the GGCNN2 model improved by residual network. The experimental environment is Ubuntu 22.04 system. The programming environment for the pose generation algorithm and grasping algorithm is Python 3.6. The training process is accelerated by using the GTX4090 graphics card of the laboratory server and multiple improvement tests are carried out.

[0101] In the training and testing of network models, the concept of Intersection over Union (IoU) from the object detection field is used to measure the model's accuracy. IoU is defined as follows:

[0102] (13)

[0103] The ratio of the intersection and union of the crawl boxes generated by the network and the marked crawl boxes is used as the accuracy of the crawl generated by the network in this application.

[0104] Experiments were conducted to improve and optimize the network parameters of the original GG-CNN. By adjusting the network optimizer type, learning rate, regularization parameter, batch size, loss function, activation function, and number of neural network layers, the accuracy of the GG-CNN network was improved. After multiple experiments, the Adam optimizer was finally selected, the batch size was set to 8, the loss function was MSE, and the initial learning rate was set to 0.001. A deep network was constructed by stacking multiple modules.

[0105] The final results and comparisons with other models having similar functionality are shown in Table 1. Experiments were conducted on the Jacquard dataset using both D and RGB-D modal inputs. Some detection results are visualized as follows: Figure 10 As shown, the best result was achieved with 93.4% accuracy when using RGB-D data as input, and 91.6% accuracy when using D as input. Compared to the original GGCNN2 network, the accuracy improved from 84% to 93.4% when using RGB-D as input.

[0106] Table 1. Detection accuracy results on the Jacquard dataset.

[0107]

[0108] The network structure described in this invention incorporates the IBN and ECA_ResNet modules compared to the original GGCNN2 network, thereby increasing the model's generalization and learning capabilities, significantly improving its performance, and making it competitive with other similar and high-performing models.

Claims

1. A robotic arm grasping method based on deep learning, characterized in that, The implementation steps of this method are as follows: S1. Before the robotic arm starts to grasp, adjust the initial state of the robotic arm, that is, the working initial point, so that the depth camera is perpendicular to the XOY plane at a fixed height above it, and obtain the RGB image and depth image of the object to be grasped through the depth camera. S2. Detect invalid pixels in the depth image and reconstruct them; simultaneously align the RGB image with the depth image to obtain an RGB-D image, and crop the center portion of the aligned image to obtain a 300×300 pixel RGB-D image. S3. The pre-trained improved GGCNN2 network model is used as input to the RGB-D image of the object to be grasped, and the output is three 300×300 pixel grasping information images, namely the pixel-level grasping quality image, grasping angle image, and grasping width image. S3. Based on the generated grasping quality map, obtain the pixel with the highest grasping quality, and obtain its corresponding grasping angle and width in the grasping angle map and grasping width map to generate the grasping pose information of the object to be grasped. S4. The obtained grasping pose information is transformed into coordinates through hand-eye calibration to obtain the grasping pose in the robot arm base coordinate system, i.e., the Cartesian coordinate system. S5. Input the grasping information and control the robotic arm to perform the grasping task; The specific steps for repairing the depth map are as follows: The first step is to obtain a depth map of the object to be crawled, which may contain invalid information; The second step is to generate a depth map of the object to be grasped with a 2D gradient; The third step is to perform thresholding on the 2D gradient to generate a binary mask; The fourth step is to fill the binary mask with water and at the same time generate a mask for all invalid points in the depth image of the object to be captured. The fifth step is to dilate the generated mask; The sixth step is to provide the generated mask to the OpenCV image inpainting function, which infers and reconstructs invalid data from the pixels; The improved GGCNN2 network model incorporates the IBN module into the GGCNN2 network structure; it also incorporates the ECA_ResNet, a powerful channel attention module combined with the residual module in the residual network, to improve model performance.

2. The robotic arm grasping method based on deep learning according to claim 1, characterized in that, The improved GGCNN2 network model includes: Extract the shallow features of the image from the convolutional part, the dilated convolutional part, the ECA_ResNet part, and the final output part; The convolutional part consists of four convolutional layers and two max pooling layers. First, there are two convolutional layers and one max pooling layer. The two convolutional layers have the same parameters: first, a 5×5 convolutional module, then an IBN module, and finally a ReLU function. Next, there are two convolutional layers and a max pooling layer with the same parameters. The number of filters in the convolutional part is 16. The dilated convolution part consists of two dilated convolutional layers with the same parameters: first, it passes through a 5×5 dilated convolutional module with dilation=2, then through a ReLU activation function, then through a 5×5 dilated convolutional module with 32 dilation, and finally through a ReLU activation function again. The number of dilated convolutional filters is 32. The ECA_ResNet part consists of 6 ECA_ResNet concatenations; the ECA_ResNet parameters include: first, a convolutional module with a stride of 3×3, followed by a BN module, then a ReLU activation function, followed by another convolutional module with a stride of 3×3 and a BN module, and finally the ECANet; the number of ECA_ResNet filters is 32. The output section includes two convolutional layers: the first convolutional layer consists of a 3x3 convolutional module followed by a ReLU activation function; the next layer consists of three linearly mapped convolutional layers, each consisting of a 3x3 convolutional module. These three linearly mapped convolutional layers sequentially map the output capture success rate, capture angle, and capture width, respectively.

3. The robotic arm grasping method based on deep learning according to claim 1, characterized in that, When training the improved GGCNN2 network model, the following criteria were used to measure the grasping accuracy: (1) The difference between the gripping angle predicted by the robotic arm and the actual gripping angle is less than or equal to a specified threshold of 30°; (2) The intersection ratio (Iou) between the grasping rectangle predicted by the robotic arm and the actual grasping rectangle is greater than 25%. The formula for the ratio of parallels: ; in, This represents the rectangle to be predicted and captured. This represents the actual captured rectangle, and the cross ratio is calculated as the ratio of the intersection to the union of two rectangles.

Images