Collaborative robot grabbing detection method based on lightweight network

Abstract

The invention provides a collaborative robot grabbing detection method based on a lightweight network, relates to the technical field of industrial intelligent grabbing, solves the problems existing in traditional two-dimensional plane grabbing, and optimizes an existing deep learning grabbing algorithm. According to the method, an attention mechanism module is introduced, a captured image is reconstructed, a novel network framework ORANGE (ORANGE) is provided, and a capturing problem is changed into a regression and classification problem. In addition, the invention further provides a directional attention mechanism, and a traditional capture strategy is improved through an enhanced capture map (AGM). According to the method, the grabbing direction of the grabbing problem is divided into a plurality of discrete intervals, and each interval represents a direction angle within a certain range, so that the overlapping problem of the same grabbing point in different directions is reduced. According to the method, through pixel-level processing on the depth image and in combination with the feature extraction capability of the convolutional neural network, the prediction accuracy and stability of the capture point are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligent grasping, and in particular to a collaborative robot grasping detection method based on a lightweight network. Background Art

[0002] Traditional grasping methods are difficult to adapt to complex environments. Robots need to have intelligent grasping capabilities. Grasping technology based on deep learning can better cope with irregular shapes and diverse objects. In the field of robot grasping, grasping detection is a key link in its execution of tasks. Its accuracy and efficiency directly affect the overall performance of the system. Traditional grasping detection methods are often based on rules and traditional computer vision technology. Due to various factors, their accuracy and stability are often unsatisfactory. With the rapid development of deep learning technology, grasping detection methods based on deep learning have made great breakthroughs, but there are still challenges.

[0003] The existing mature GG-CNN grasping network inputs the preprocessed depth image into the network, extracts the features in the depth image through the convolution layer, then uses deconvolution to generate a grasping map, outputs the grasping quality score, grasping angle and gripper width, and finally selects the point with the highest quality score in the grasping map for grasping. The existing technology has the following problems: Many existing solutions for grasping objects use a relatively simple network structure, so they may not be accurate enough when processing complex scenes, and have poor adaptability to the shapes and materials of different objects. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a collaborative robot grasping detection method based on a lightweight network. The present invention aims to solve the problems existing in traditional two-dimensional plane grasping and optimize the existing deep learning grasping algorithm. By introducing the attention mechanism module and reconstructing the grasping graph, a new network framework ORANGE (ORientationAttentive Grasp synthEsis) is proposed to transform the grasping problem into a regression and classification problem. In addition, a directional attention mechanism is proposed, which improves the traditional grasping strategy through an augmented grasping map (Augmented Grasp Map, AGM). This method divides the grasping direction into multiple discrete intervals, each of which represents a direction angle within a certain range, thereby reducing the overlap problem of the same grasping point in different directions. This method significantly improves the prediction accuracy and stability of the grasping point by processing the depth image at the pixel level and combining the feature extraction capability of the convolutional neural network.

[0005] A collaborative robot grasping detection method based on a lightweight network includes the following steps:

[0006] Step 1: Visual perception: Use a depth camera to collect images of the capture and grasping process of the six-degree-of-freedom robotic arm, and pre-process the collected images, where the collected images include depth images and color images;

[0007] The preprocessing is specifically as follows: first, Gaussian filtering is used to eliminate image noise, then the depth image is aligned with the color image, and finally the processed image is input into the ORANGE network framework;

[0008] The ORANGE network framework is based on GGCNN2 and an improved U-Net network architecture. The improved U-Net network architecture is specifically: ordinary convolution in the U-Net network is replaced by depthwise separable convolution, a SE module is added to the decoding layer, and an SPP module is added to the bridging layer;

[0009] Step 2: Grab the predictions;

[0010] Step 2.1: Network algorithm prediction; the ORANGE network framework generates an enhanced grasping map AGM, which separates different angles of the grasping direction and divides the direction of the grasping point into multiple discrete angle partitions as a direction attention mechanism;

[0011] The enhanced grasping map uses the pixel-level information in the depth image to extract features through a convolutional neural network to generate a quality map, an orientation map, and a grasping probability map for predicting grasping points; the enhanced grasping map is defined by the following formula:

[0012] G={Φ,Ω,Q,O,Γ}∈R (4×N)+1×H×W

[0013] Specifically, it includes: angle map Φ, width map Ω, quality map Q, direction map O, and grasp probability map Γ, where N is the number of direction bins, H×W is the spatial resolution of the grasp map, and H and W represent the height and width of the image;

[0014] (1) Angle map Φ: The grasping direction is discretized into N direction bins, each bin represents a fixed angle range; the direction value is expressed as Φ cos =cos2Φ and Φ sin =sin 2Φ function encoding solves the discontinuity problem caused by angular periodicity;

[0015] (2) Width map Ω: combines the grasp width with the direction bins and expands it into a multi-channel representation;

[0016] (3) Quality graph Q: The grasping quality becomes a continuous value with a value range of 0≤q≤1, indicating the confidence of the grasping point;

[0017] (4) Direction map O: a binary tensor used to represent the validity of each pixel in a specific direction. If the pixel is valid in a certain direction bin, the corresponding bin value is 1, otherwise it is 0.

[0018] (5) Grasp probability map Γ: A two-dimensional matrix that represents the possibility of each pixel being a grasp point, that is, the binary label of the grasp, which guides the ORANGE network framework to focus on the grasp area when predicting the grasp point and judge whether the pixel is a valid grasp point.

[0019] The directional attention mechanism obtains the most effective grasping direction by weighting different directions, which specifically includes the following steps:

[0020] Step S1: Input directional features: Assume that there is a directional feature map F∈R in the output of the ORANGE network framework N *H*W , where: N is the number of direction bins, H×W is the spatial resolution of the captured image, H and W represent the height and width of the image; each direction bin contains a direction feature map, which represents the capture features in different directions;

[0021] Step S2: Perform global average pooling GAP: Perform global average pooling on the directional feature map in each direction to obtain a global description of each direction, as follows:

[0022]

[0023] where z n is the global feature of direction bin n, indicating the global information of this direction; F n (i, j) is the value of pixel (i, j) in the directional feature map of the nth directional bin;

[0024] Step S3: Generate direction weights: Use a lightweight fully connected layer to generate the weight ω of the bin in each direction n :

[0025] ω n =σ(W2·ReLU(W1·z n ))

[0026] Where W1 and W2 are the weight matrices of the fully connected layer, σ is the Sigmoid activation function, which is used to normalize the weights to the range of [0, 1], and ω n Indicates the degree of attention to direction bin n;

[0027] Step S4: Weight application: Use direction weight ω n For each direction feature map F n Perform weighted processing:

[0028] F′ n =ω n ·F n

[0029] Step S5: Output the weighted directional features: The obtained weighted directional feature map F′:

[0030]

[0031] The weighted directional feature maps of each direction are merged to form the final directional feature map as the input of the subsequent network;

[0032] During the training process of the ORANGE network framework, a multi-task loss function is designed to train the ORANGE network framework to solve the optimization problem of multiple tasks contained in the output features of the ORANGE network (enhanced grasping map), and a pixel-level attention mechanism is used to improve the prediction quality of grasping points. The specific outputs include quality maps, direction maps, and grasping probability maps.

[0033] The multi-task loss function optimizes multiple grasping tasks simultaneously, combining binary cross entropy BCE and mean square error MSE, as follows:

[0034]

[0035] in, is the cross entropy function, is the mean square error function;

[0036] Step 2.2: Generate grasping configuration: After the ORANGE network framework outputs the enhanced grasping graph, the optimal grasping point and optimal grasping direction are screened out;

[0037] Specifically: the optimal grasping point is obtained by combining the maximum grasping quality and the grasping direction, and the optimal grasping direction is obtained by classification and regression;

[0038] Step 2.3: Analyze the relationship between objects in the environment and obtain the grasping order:

[0039] Specifically, probability maps and orientation maps as well as object occlusion analysis are used to obtain the grasping order;

[0040] Step 3: Execute the grasping: Combine the three-dimensional coordinates (x, y, z) of the object and the grasping direction to output the grasping posture of the robot arm; then transform the predicted grasping point from the camera coordinate system to the robot arm coordinate system, use the hand-eye calibration method to calculate the transformation matrix, map the predicted grasping point to the range of motion of the robot arm, and the robot arm executes the grasping action and moves to the target point to control the gripper to complete the grasping action;

[0041] The beneficial effects of adopting the above technical solution are:

[0042] The present invention provides a collaborative robot grasping detection method based on a lightweight network, which improves the accuracy and robustness of the robot in grasping complex objects, especially when the object's posture and direction change greatly. Traditional grasping methods usually ignore the change in the direction of the object, resulting in inflexible grasping strategies and difficulty in coping with the challenges brought by the rotation or irregular posture of the object. Therefore, the present invention proposes a solution based on a directional attention mechanism and an enhanced grasping graph representation, which enables the robot to dynamically perceive and adjust the grasping direction, thereby selecting the most appropriate grasping strategy under different object postures, which can significantly improve the grasping success rate and adaptability.

[0043] The present invention has the following beneficial effects:

[0044] 1. Improved the accuracy and stability of grab detection.

[0045] 2. Reduced computing costs and improved real-time performance and efficiency.

[0046] 3. Experimental verification shows that the proposed method has high accuracy and robustness in various grasping tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is an overall flow chart of the collaborative robot grasping detection method of the present invention;

[0048] Figure 2 The specific structure diagram of the improved U-Net network module of the present invention;

[0049] Among them, (a)-depth separable convolution, (b)-SE module, (c)-SPP module;

[0050] Figure 3 The overall structure diagram of the improved U-Net network module of the present invention;

[0051] Figure 4 This is a structural diagram of the six-degree-of-freedom robotic arm grasping system of the present invention;

[0052] Figure 5 A capture map generated for the present invention;

[0053] Figure 6 This is an actual grasping diagram of the robot arm of the present invention. DETAILED DESCRIPTION

[0054] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0055] The experimental environment in this embodiment is as follows:

[0056] Operating system: Ubuntu MATE16.04

[0057] CPU: Intel Xeon E5-2620 v4@2.10GHz

[0058] GPU: Titan X

[0059] Python version: 3.9.12

[0060] PyTorch version: 1.10.1+cu111

[0061] Torchvision version: 0.11.2+cu111

[0062] Dataset preparation:

[0063] 1. The Jacquard dataset contains 54,000 depth images and more than 1.1 million grasp annotation data, which can provide annotations for multiple grasp directions and widths;

[0064] 2. The Cornell dataset contains 885 RGB-D images of real objects, including 5110 manually labeled positive samples and 2909 negative samples.

[0065] Use Cornell and Jacquard public datasets, and set the ratio of Jacquard training set to test set to 9:1. Resize the input depth image to 320×320 pixels to increase training speed.

[0066] Training process:

[0067] Pre-training was performed on the Jacquard dataset, and then a “warm-start” strategy was adopted on the Cornell dataset. The model weights trained on the Jacquard dataset were initialized, and then fine-tuning was performed on the Cornell dataset with an initial learning rate of 0.0002, a batch size of 8, and an epoch of 100.

[0068] Experimental results:

[0069] At an IOU threshold of 0.3, the improved network achieves 92.33% grasping accuracy on the Cornell dataset.

[0070] Experiments show that the network performs well in processing multi-scale information and inter-channel dependencies, significantly improving the performance of grasp detection.

[0071] A collaborative robot grasping detection method based on lightweight network, such as Figure 1 As shown, the following steps are included:

[0072] Step 1: Visual perception: Use a depth camera to collect images of the capture and grasping process of the six-degree-of-freedom robotic arm, and pre-process the collected images, where the collected images include depth images and color images;

[0073] The preprocessing is specifically as follows: first, Gaussian filtering is used to eliminate image noise, then the depth image is aligned with the color image to ensure consistency of spatial data, and finally the processed image is input into the ORANGE network framework;

[0074] In this embodiment, a Realman grasping system is built, which specifically includes the Realman company's RM65-6F six-degree-of-freedom robotic arm and the Inter D-435i depth camera. The Inter D-435i depth camera is used to capture image information of the grasping area, where the collected content includes color images (including the texture, color and shape of the object) and depth images (the distance between the object and the camera, constructing three-dimensional environmental information), and the collected image information is transmitted to the computer system.

[0075] The depth image is unified into an image of size 320x320 and input into the ORANGE network framework, which is based on the GGCNN2 and improved U-Net network architecture, such as Figure 3 As shown in Figure 2, during actual crawling, it was found that the U-Net network in the algorithm framework did not perform well in unstructured scenarios, so the U-Net network was improved: Figure 2 As shown in , the improved U-Net network architecture is specifically: replacing the ordinary convolution in the U-Net network with a depth-separable convolution, such as Figure 2 As shown in (a), the depthwise separable convolution can decompose the convolution operation into two parts: depthwise convolution and pointwise convolution, which greatly reduces the amount of calculation and the number of model parameters without significantly reducing the model performance, which is in line with the original intention of lightweight. The SE module is added to the decoding layer, such as Figure 2 As shown in (b), the SE module enables the network to adaptively emphasize important feature channels and suppress unimportant features during the decoding stage by learning the inter-channel weights of features. This helps the network focus on key local information, thereby improving the accuracy of decoding. It can also adaptively adjust the feature responses of different channels according to context information, so that the network can better capture the importance of different semantic information, and only a small number of parameters are added, which has limited impact on computing performance; adding an SPP module in the bridge layer, such as Figure 2As shown in (c), the SPP module can better capture the features of objects at different sizes through multi-scale pooling (such as maximum pooling or mean pooling), and can expand the receptive field of the network, so that the model can consider more global information in the encoding stage and obtain feature representations of different receptive field ranges, thereby enhancing the model's understanding of global and local information.

[0076] These improvements can improve the model's computational efficiency, accuracy, multi-scale processing capabilities, and robustness, making the improved U-Net more suitable for complex image processing tasks, especially in resource-constrained environments, where it can achieve a better balance between computation and accuracy.

[0077] Step 2: Grab the predictions;

[0078] Step 2.1: Network algorithm prediction; the ORANGE network framework generates an augmented grasp map AGM (Augmented GraspMap), which separates different angles of the grasping direction and divides the direction of the grasping point into multiple discrete angle partitions as the orientation-attentive mechanism;

[0079] a. Enhanced capture image

[0080] The enhanced grasping graph represents a way to comprehensively encode the grasping characteristics of an object. Unlike traditional grasping methods, AGM not only records the position information of each grasping point, but also considers the success probability of each grasping point in different directions and postures. This method enables the robot to select the most suitable grasping point under different object postures and directions, thereby improving the grasping success rate.

[0081] The enhanced grasp graph representation method reduces the overlap problem of the same point in multiple directions by dividing the grasp direction angle into multiple discrete intervals, thereby effectively improving the prediction accuracy of the grasp point. The core idea of ​​this method is to divide the continuous angle of the grasp direction into multiple fixed intervals (bins) through discretization. Specifically, the grasp angle φ is divided into N intervals, and the angle range of each interval is 180 / N degrees. This can reduce the prediction instability caused by the overlap of the same grasp point in different directions.

[0082] The enhanced grasping map uses the pixel-level information in the depth image to extract features through a convolutional neural network to generate a quality map, an orientation map, and a probability map for predicting grasping points; the enhanced grasping map is defined by the following formula:

[0083] G={Φ,Ω,Q,O,Γ}∈R (4×N)+1×H×W

[0084] Specifically, it includes: angle map Φ, width map Ω, quality map Q, direction map O, graspability map Γ, N is the number of direction bins, H×W is the spatial resolution of the grasp map, H and W represent the height and width of the image;

[0085] (1) Angle map Φ: The grasping direction is discretized into N direction bins, each bin represents a fixed angle range; the direction value is expressed as Φ cos =cos2Φ and Φ sin =sin 2Φ function encoding solves the discontinuity problem caused by angular periodicity;

[0086] (2) Width map Ω: combines the grasp width with the direction bins and expands it into a multi-channel representation;

[0087] (3) Quality graph Q: The grasping quality becomes a continuous value with a value range of 0≤q≤1, indicating the confidence of the grasping point;

[0088] (4) Direction map O: a binary tensor used to represent the validity of each pixel in a specific direction. If the pixel is valid in a certain direction bin, the corresponding bin value is 1, otherwise it is 0.

[0089] (5) Graspability graph Γ: A two-dimensional matrix representing the possibility of each pixel being a grasping point, i.e., the binary label of the grasp, which guides the ORANGE network framework to focus on the grasping area when predicting grasping points and to determine whether the pixel is a valid grasping point.

[0090] b. Direction Attention Mechanism

[0091] The introduction of a direction-aware attention mechanism enables the robot to dynamically adjust its grasping strategy according to the different directions and postures of the object. Through this mechanism, the robot can automatically select and optimize the grasping direction, avoiding the reliance on fixed directions in traditional methods. The directional attention mechanism is designed to optimize the multi-directional grasping problem in grasping tasks, especially to solve the conflict of directions of multiple grasping boxes (or grasping points). Its core goal is to guide the ORANGE network framework to focus on the most important grasping direction, especially in multi-directional grasping tasks, where the model needs to choose between multiple directions.

[0092] The directional attention mechanism obtains the most effective grasping direction and reduces the interference of invalid directions by weighting different directions. Specifically, it includes the following steps:

[0093] Step S1: Input directional features: Assume that there is a directional feature map F∈R in the output of the ORANGE network framework N *H*W, where: N is the number of direction bins, H ×W is the spatial resolution of the captured image, H and W represent the height and width of the image; each direction bin contains a direction feature map, which represents the captured features in different directions;

[0094] Step S2: Perform global average pooling GAP (Global Average Pooling): Perform global average pooling on the directional feature map in each direction to obtain a global description of each direction, as follows:

[0095]

[0096] where z n is the global feature of direction bin n, indicating the global information of this direction; F n (i, j) is the value of pixel (i, j) in the directional feature map of the nth directional bin;

[0097] Step S3: Generate direction weights: Use a lightweight fully connected layer to generate the weight ω of the bin in each direction n :

[0098] ω n =σ(W2·ReLU(W1·z n ))

[0099] Where W1 and W2 are the weight matrices of the fully connected layer, σ is the Sigmoid activation function, which is used to normalize the weights to the range of [0, 1] to ensure that the weights in each direction are within this range, and ω n Indicates the degree of attention to direction bin n;

[0100] Step S4: Weight application: Use direction weight ω n For each direction feature map F n Perform weighted processing:

[0101] F′ n =ω n ·F n

[0102] Step S5: Output weighted directional features: The obtained weighted directional feature map F ′ :

[0103]

[0104] The weighted directional feature maps of each direction are merged to form the final directional feature map as the input of the subsequent network;

[0105] During the training process of the ORANGE network framework, a multi-task loss function is designed to train the ORANGE network framework to solve the optimization problem of multiple tasks contained in the output features of the ORANGE network (enhanced grasping map), and a pixel-level attention mechanism is used to improve the prediction quality of grasping points. The specific outputs include quality maps, direction maps, and probability maps.

[0106] The multi-task loss function optimizes multiple grasping tasks simultaneously, combining binary cross entropy BCE and mean square error MSE, as follows:

[0107]

[0108] in, is the cross entropy function, is the mean square error function; the binary cross entropy is used for the binary classification problem of grasping quality and direction weight (O and Γ), while the mean square error is used for grasping direction, width, and the product of grasping quality and direction weight.

[0109] Through joint training, this method enables the model to simultaneously consider the grasping characteristics of the object and the actual grasping effect, thereby improving the performance and robustness of the robot grasping task.

[0110] Step 2.2: Generate grasping configuration: After the ORANGE network framework outputs the enhanced grasping graph, the optimal grasping point and optimal grasping direction are screened out;

[0111] Specifically: the optimal grasping point is obtained by combining the maximum grasping quality and the grasping direction, and the optimal grasping direction is obtained by classification and regression;

[0112] Step 2.3: Analyze the relationship between objects in the environment and obtain the grasping order:

[0113] For some unstructured scenes, especially stacked or complexly distributed objects, it is necessary to use probability maps, direction maps and object occlusion analysis to obtain the grasping order; to ensure that obstacles or objects are removed first during actual grasping to improve the grasping success rate.

[0114] Step 3: Execute the grasping: Combine the three-dimensional coordinates (x, y, z) of the object and the grasping direction to output the grasping posture of the robot arm; then transform the predicted grasping point from the camera coordinate system to the robot arm coordinate system, use the hand-eye calibration method to calculate the transformation matrix, map the predicted grasping point to the range of motion of the robot arm, and the robot arm executes the grasping action and moves to the target point to control the gripper to complete the grasping action;

[0115] Crawl experiment:

[0116] The Realman grasping system was used to conduct grasping experiments. By generating grasping boxes and calculating IOU values, the grasping accuracy and stability of the improved network were verified.

[0117] Experimental verification

[0118] After using the RealSense D435i depth camera for data collection, the collected images are preprocessed and cropped to a uniform pixel size of 320x320 for better input into the network. The images are further processed by data enhancement and outlier removal.

[0119] The preprocessed depth image is input into the improved composite neural network to generate a capture map, such as Figure 5 As shown. According to the reconstructed grasping graph, the quality and stability of different grasps are predicted, and finally the point with the highest grasping quality is determined as the grasping point. In the experiment, the RM65-6F six-degree-of-freedom robot arm of Realman Company is used. Figure 4 As shown in the figure, the robot uses the open ROS interface and the RealSense D435i depth camera to conduct grasping experiments on the Cornell and Jacquard public data sets, and generates an optimized grasping strategy based on the prediction results to determine the grasping position and posture, as shown in the figure. Figure 6 shown.

[0120] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A collaborative robot grasping detection method based on a lightweight network, characterized in that: The following steps are involved: Step 1: Visual perception: Use a depth camera to collect images of the capture and grasping process of the six-degree-of-freedom robotic arm, and pre-process the collected images, where the collected images include depth images and color images; Step 2: Grab the predictions; Step 2.1: Network algorithm prediction; the ORANGE network framework generates an enhanced grasping map AGM, which separates different angles of the grasping direction and divides the direction of the grasping point into multiple discrete angle partitions as a direction attention mechanism; Step 2.2: Generate grasping configuration: After the ORANGE network framework outputs the enhanced grasping graph, the optimal grasping point and optimal grasping direction are screened out; Step 2.3: Analyze the relationship between objects in the environment and obtain the grasping order: Specifically, probability maps and orientation maps as well as object occlusion analysis are used to obtain the grasping order; Step 3: Execute grasping: Combine the three-dimensional coordinates (x, y, z) of the object and the grasping direction to output the grasping posture of the robot arm; then transform the predicted grasping point from the camera coordinate system to the robot arm coordinate system, use the hand-eye calibration method to calculate the transformation matrix, map the predicted grasping point to the range of motion of the robot arm, and the robot arm executes the grasping action and moves to the target point to control the gripper to complete the grasping action.

2. According to the lightweight network-based collaborative robot grasping detection method of claim 1, it is characterized in that: The preprocessing is specifically as follows: first, Gaussian filtering is used to eliminate image noise, then the depth image is aligned with the color image, and finally the processed image is input into the ORANGE network framework.

3. According to the lightweight network-based collaborative robot grasping detection method of claim 1, it is characterized in that: The ORANGE network framework is based on GGCNN2 and an improved U-Net network architecture. The improved U-Net network architecture is specifically: ordinary convolution in the U-Net network is replaced by depthwise separable convolution, a SE module is added to the decoding layer, and an SPP module is added to the bridging layer.

4. The lightweight network-based collaborative robot grasping detection method according to claim 1 is characterized in that: The enhanced grasping map described in step 2.1 uses the pixel-level information in the depth image to extract features through a convolutional neural network to generate a quality map, an orientation map, and a grasping probability map for predicting grasping points; the enhanced grasping map is defined by the following formula: G={Φ,Ω,Q,O,Γ}∈R (4×N)+1×H×W Specifically, it includes: angle map Φ, width map Ω, quality map Q, direction map O, and grasp probability map Γ, where N is the number of direction bins, H×W is the spatial resolution of the grasp map, and H and W represent the height and width of the image; (1) Angle map Φ: The grasping direction is discretized into N direction bins, each bin represents a fixed angle range; the direction value is expressed as Φ cos = sin 2Φ and Φ sin =sin 2Φ function encoding solves the discontinuity problem caused by angular periodicity; (2) Width map Ω: combines the grasp width with the direction bins and expands it into a multi-channel representation; (3) Quality graph Q: The grasping quality becomes a continuous value with a value range of 0≤q≤1, indicating the confidence of the grasping point; (4) Direction map O: a binary tensor used to represent the validity of each pixel in a specific direction. If the pixel is valid in a certain direction bin, the corresponding bin value is 1, otherwise it is 0. (5) Grasp probability map Γ: A two-dimensional matrix that represents the possibility of each pixel being a grasp point, that is, the binary label of the grasp, which guides the ORANGE network framework to focus on the grasp area when predicting the grasp point and judge whether the pixel is a valid grasp point.

5. The lightweight network-based collaborative robot grasping detection method according to claim 1, characterized in that: The directional attention mechanism described in step 2.1 obtains the most effective grasping direction by weighting different directions, which specifically includes the following steps: Step S1: Input directional features: Assume that there is a directional feature map F∈R in the output of the ORANGE network framework N*H*W , where: N is the number of direction bins, H×W is the spatial resolution of the captured image, H and W represent the height and width of the image; each direction bin contains a direction feature map, which represents the capture features in different directions; Step S2: Perform global average pooling GAP: Perform global average pooling on the directional feature map in each direction to obtain a global description of each direction, as follows: where z n is the global feature of direction binn, indicating the global information of this direction; F n (i, j) is the value of pixel (i, j) in the directional feature map of the nth directional bin; Step S3: Generate direction weights: Use a lightweight fully connected layer to generate the weight ω of the bin in each direction n : ω n =σ(W2 ReLU(W1 z n )) Where W1 and W2 are the weight matrices of the fully connected layer, σ is the Sigmoid activation function, which is used to normalize the weights to the range of [0, 1], and ω n Indicates the degree of attention to direction bin n; Step S4: Weight application: Use direction weight ω n For each direction feature map F n Perform weighted processing: F′ n =ω n ·F n Step S5: Output the weighted directional features: The obtained weighted directional feature map F′: The weighted directional feature maps in each direction are merged to form the final directional feature map as the input of the subsequent network.

6. The lightweight network-based collaborative robot grasping detection method according to claim 1, characterized in that: During the training process of the ORANGE network framework, a multi-task loss function is designed to train the ORANGE network framework to solve the optimization problem of multiple tasks contained in the output features of the ORANGE network, and a pixel-level attention mechanism is used to improve the prediction quality of the grasping points. The specific outputs include quality maps, direction maps, and grasping probability maps.

7. A lightweight network-based collaborative robot grasping detection method according to claim 6, characterized in that: The multi-task loss function optimizes multiple grasping tasks simultaneously, combining binary cross entropy BCE and mean square error MSE, as follows: in, is the cross entropy function, is the mean square error function.

8. The lightweight network-based collaborative robot grasping detection method according to claim 1, characterized in that: The step 2.2 is specifically as follows: obtaining the optimal grasping point by combining the maximum grasping quality and the grasping direction, and obtaining the optimal grasping direction by classification and regression.

Images