A lightweight encoding / decoding capture pose detection method, system, terminal, and medium

Abstract

This invention relates to the field of robot vision grasping and detection technology, and discloses a lightweight encoding and decoding grasping pose detection method, system, terminal, and medium. The proposed encoding and decoding grasping pose detection network GDLNet is based on an encoding and decoding network architecture. It utilizes depthwise separable convolution and shuffling units to construct a target feature depth separation and fusion extraction block, reducing the number of parameters in the encoding network and enhancing the network's ability to extract features from the grasping region. It uses bilinear interpolation and depthwise separable convolution to establish a depth separation feature reconstruction block, which effectively reduces the number of parameters in the decoding network while restoring lost information in high-level features. Addressing the problem of inconsistency between the pixel representation of the graspable region and the overall appearance of the target object, this invention not only reduces the number of parameters in the encoding and decoding network and enhances the network's ability to extract features from the grasping region, but also effectively enhances the features of the graspable target region and suppresses redundant features of non-grasping pixels, exhibiting superior robustness.

Description

Technical Field

[0001] This invention relates to the field of robot vision grasping and detection technology, specifically a lightweight encoding and decoding grasping pose detection method, system, terminal, and medium. Background Technology

[0002] In recent years, robots have experienced rapid development in fields such as intelligent manufacturing, medical assistance, social services, and space exploration. Robotic arm grasping is the most widely used and challenging key technology in robot applications. In structured environments with fixed target positions and simple scenarios, industrial robots can already replace humans in performing precise and repetitive grasping tasks. However, fields such as intelligent manufacturing and social services face numerous open environments and unstructured scenarios with changing targets. These scenarios require industrial robots to calculate an optimal grasping pose to accurately and quickly grasp objects from an image or depth information containing unknown objects. This poses a challenge to the accurate and rapid perception of targets by industrial robots. Therefore, rapid and accurate grasping perception is of significant research importance and practical application value for the development of robots from standardized to intelligent systems.

[0003] Grasping detection methods in unstructured environments are mainly divided into perceptual generative methods and reasoning-based decision-making methods. Perceptual generative methods utilize mathematical or physical models of an object's geometry, physical properties, and kinematic formulas to calculate fixed grasping parameters. However, because this method relies on a known 3D model of the object, it struggles to effectively execute grasping strategies for unknown objects in new grasping scenarios. Currently, reasoning-based decision-making methods train grasping models using visual information from known objects and use these models to detect the grasping pose of unknown objects. Since it does not require extracting a 3D model of the object, it is applicable to unknown objects and therefore has wider applications. However, early algorithms were only suitable for relatively stable teaching environments. In real-world scenarios, objects of varying shapes exhibit significant differences, making it difficult for robotic arms to meet the demands of accurate and rapid intelligent grasping in unstructured environments.

[0004] Visual perception-based object grasping detection requires feature analysis of the acquired images to quickly determine the graspable region and establish a robotic arm grasping pose model. Therefore, contact-point-based grasping representation necessitates the establishment of a pixel-level accurate representation model of the graspable region. However, the inconsistency between the graspable representation pixel region and the object, and the susceptibility to accuracy loss due to the influence of non-grasping region pixels, affect the accurate global perception capability of pixel-level grasping detection. Current research has found that conventional encoding / decoding structures only learn contextual information in the spatial dimension, neglecting the contextual information between the target object and the graspable region. These discrepancies often lead the network to learn more non-grasping region features in the image during the encoder stage, making it difficult to accurately extract graspable region features. Based on this, to effectively extract graspable region features, it is necessary to investigate a lightweight encoding / decoding robotic arm pose grasping detection method based on a context aggregation strategy. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention aims to provide a lightweight encoding and decoding grasping pose detection method, system, terminal and medium to solve the technical problem that ignores the contextual information between the target object and the grasping area, which causes the network to learn more non-grasping area features of the object in the image during the encoder stage, making it difficult to accurately extract the graspable area features.

[0006] This invention is achieved through the following technical solution:

[0007] A lightweight encoding / decoding method for capturing pose detection includes the following steps:

[0008] Obtain images containing object grasping poses and construct a grasping pose detection dataset;

[0009] The pre-built encoding / decoding grasping detection network is trained using the acquired grasping pose detection dataset to obtain the trained encoding / decoding grasping pose detection network.

[0010] Using a trained encoder-decoder grasping pose detection network, the grasping pose of objects in the grasping scene to be extracted is detected.

[0011] Preferably, the encoder-decoder grasping pose detection network includes an encoder and a decoder, wherein the encoder includes a depth separation convolution block, four depth separation fusion extraction blocks and a grasping region context aggregation block; and the decoder includes five depth separation reconstruction blocks.

[0012] Furthermore, the encoder's depth-separating convolutional block comprises two convolutional layers and a depth-separable convolution, with each convolutional layer connected to a batch normalization layer and a linear rectifier.

[0013] Furthermore, the encoder's depth separation fusion extraction block includes two stages. The first stage includes a depth-separable convolutional layer and a point convolutional layer. The second stage includes a depth-separable convolution, the output of which is sequentially connected to a batch normalization layer and a linear rectifier.

[0014] Furthermore, the encoder's grasping region context aggregation block includes a depth-separable convolutional layer, a convolutional layer, and a Softmax layer; the output of the convolutional layer is connected to a point convolutional layer, a batch normalization layer, and a linear rectifier layer.

[0015] Furthermore, each decoder's depth separation reconstruction block includes a bilinear interpolation layer, and each bilinear interpolation layer is connected to a depth-separable convolution, a batch normalization layer, and a linear rectified layer.

[0016] Preferably, the pre-constructed encoding / decoding grasping detection network is trained using the acquired grasping pose detection dataset to obtain the trained encoding / decoding grasping pose detection network. The specific method is as follows:

[0017] Using the acquired grasping pose detection dataset, the pre-built encoding / decoding grasping detection network is iteratively optimized and trained by backpropagation combined with the loss function of the grasping region, resulting in the trained encoding / decoding grasping pose detection network.

[0018] A lightweight encoding / decoding capture pose detection system, including

[0019] The data acquisition module is used to acquire images containing object grasping poses and construct a grasping pose detection dataset.

[0020] The network training module is used to train the pre-built encoding / decoding grasping detection network using the acquired grasping pose detection dataset, so as to obtain the trained encoding / decoding grasping pose detection network.

[0021] The encoding / decoding grasping and detection module is used to perform grasping pose detection on objects in the grasping scene to be extracted using a trained encoding / decoding grasping pose detection network.

[0022] A mobile terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the lightweight encoding / decoding capture pose detection method described above.

[0023] A computer-readable storage medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the lightweight encoding / decoding capture pose detection method described above.

[0024] Compared with the prior art, the present invention has the following beneficial technical effects:

[0025] This invention provides a lightweight encoding / decoding grasping pose detection method. The proposed encoding / decoding grasping pose detection network GDLNet first uses an encoding / decoding network architecture as a foundation, employing depthwise separable convolutions and shuffling units to construct a target feature depthwise separation and fusion extraction block, reducing the number of parameters in the encoding network and enhancing the network's ability to extract features from the grasping region. Second, it uses bilinear interpolation and depthwise separable convolutions to establish a depthwise separation feature reconstruction block, effectively reducing the number of parameters in the decoding network while recovering lost information in high-level features. Finally, addressing the inconsistency between the pixel representation of the graspable region and the overall picture of the target object, a target grasping region context aggregation strategy is proposed based on cross-entropy-assisted loss and self-attention mechanism. This strategy guides the network to enhance the features of the graspable target region and suppress redundant features of non-grasping pixels. This invention not only reduces the number of parameters in the encoding / decoding network and enhances the network's ability to extract features from the grasping region, but also effectively enhances the features of the graspable target region and suppresses redundant features of non-grasping pixels, exhibiting superior robustness. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the lightweight encoding / decoding capture pose detection method of the present invention.

[0027] Figure 2 This is a schematic diagram of the lightweight encoding / decoding capture pose detection network structure in this invention;

[0028] Figure 3 This is a structural diagram of the deep separation, fusion, and extraction module in this invention;

[0029] Figure 4 This is a structural diagram of the deep separation and reconstruction block in this invention;

[0030] Figure 5 This is a structural diagram of the context capture aggregation block in this invention;

[0031] Figure 6 This is a diagram showing the encoding and decoding detection effect in this invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0033] The present invention will now be described in further detail with reference to the accompanying drawings:

[0034] The purpose of this invention is to provide a lightweight encoding and decoding method, system, terminal, and medium for capturing pose detection, in order to solve the technical problem that neglecting the contextual information between the target object and the capturing area leads to the network learning more non-capturing area features of the object in the image during the encoder stage, making it difficult to accurately extract the features of the captureable area.

[0035] Specifically, according to Figure 1 As shown, this lightweight encoding / decoding method for capturing pose detection includes the following steps:

[0036] Step 1: Obtain images containing object grasping poses and construct a grasping pose detection dataset;

[0037] Step 2: Use the acquired grasping pose detection dataset to train the pre-built encoding / decoding grasping detection network to obtain the trained encoding / decoding grasping pose detection network.

[0038] Step 3: Use the trained encoder-decoder grasping pose detection network to perform grasping pose detection on objects in the grasping scene to be extracted.

[0039] Specifically, according to Figure 2 As shown, the encoder-decoder grasping pose detection network includes an encoder and a decoder. The encoder includes a depth separation convolution block, four depth separation fusion extraction blocks, and a grasping region context aggregation block; the decoder includes five depth separation reconstruction blocks.

[0040] The encoder's depth-separating convolutional block includes two convolutional layers and a depth-separable convolution, with each convolutional layer connected to a batch normalization layer and a linear rectifier.

[0041] Among them, according to Figure 3 As shown, the encoder's depth separation fusion extraction block includes two stages. The first stage includes a depth-separable convolutional layer and a point convolutional layer. The second stage includes a depth-separable convolution, the output of which is sequentially connected to a batch normalization layer and a linear rectifier.

[0042] Among them, according to Figure 5 As shown, the encoder's capture region context aggregation block includes a depth-separable convolutional layer, a convolutional layer, and a Softmax layer; the output of the convolutional layer is connected to a point convolutional layer, a batch normalization layer, and a linear rectifier layer.

[0043] Among them, according to Figure 4 As shown, each decoder's depth separation reconstruction block includes a bilinear interpolation layer, and each bilinear interpolation layer is connected to a depth-separable convolution, a batch normalization layer, and a linear rectified layer.

[0044] Specifically, the pre-built encoding / decoding grasping detection network is trained using the acquired grasping pose detection dataset to obtain the trained encoding / decoding grasping pose detection network. The specific method is as follows:

[0045] Using the acquired grasping pose detection dataset, the pre-built encoding / decoding grasping detection network is iteratively optimized and trained by backpropagation combined with the loss function of the grasping region, resulting in the trained encoding / decoding grasping pose detection network.

[0046] Example

[0047] This embodiment provides a lightweight encoding / decoding capture pose detection method based on a context aggregation strategy, including the following steps:

[0048] Step 1: Construct a lightweight encoding / decoding capture pose detection network based on a context aggregation strategy. The structure of the lightweight encoding / decoding capture pose detection network based on the context aggregation strategy is as follows: Figure 2 As shown.

[0049] The proposed encoding / decoding crawling detection method is based on an encoding / decoding structure network. It constructs a target crawling region context aggregation block (GRCA) based on a target context strategy, and then proposes a lightweight and efficient lightweight encoding / decoding crawling pose detection network (GDLNet). The proposed GDLNet network mainly consists of three parts: a deep separation fusion extraction block (DSFEB), a deep separation reconstruction block (DSRB), and a context aggregation block (GRCA).

[0050] Among them, the deep separation and fusion extraction block significantly reduces the computational complexity and number of parameters without reducing the network's feature extraction capability, extracts more discriminative and robust features, and inputs the extracted features into the context aggregation block to extract context information;

[0051] The context aggregation block guides the network to focus on the contextual information of the graspable region by using cross-entropy auxiliary loss of the graspable region ground truth and deep features, along with a self-attention mechanism. This enhances the discriminability of the graspable region features and addresses the issue of inconsistency between the graspable pixel region and the overall object. Specifically, it considers the correlation between all graspable object regions and each pixel to enhance the grasping representation of each pixel, guiding the network to selectively learn graspable regions, suppress interference from non-grasping regions, and improve the accuracy of grasping detection. Finally, the graspable region context aggregation strategy helps improve the network's ability to discriminate objects with inconsistencies between the graspable pixel region and the overall object.

[0052] The depthwise separable reconstruction block fills unknown pixels with bilinear interpolation. Depthwise separable convolution can effectively reduce the number of network parameters while recovering information lost in high-level features. It improves the accuracy of pixel feature capture and reconstruction while reducing the overall computational cost and parameter count of the network, and reconstructs the original resolution of the image.

[0053] The present invention relates to (1) a target grasping region context aggregation block (GRCA) based on the target context strategy, and designs a lightweight and efficient lightweight encoder-decoder grasping pose detection network (GDLNet) based on the target context strategy to enhance the discriminability of graspable region features and solve the problem that the graspable pixel region is not consistent with the whole picture of the object; (2) a depth separation fusion extraction block based on shuffling units and depth-separable convolution; and (3) a depth separation reconstruction block based on depth-separable convolution.

[0054] Step 2: Construct a grasping pose detection dataset. This invention uses two grasping pose detection datasets, the Cornell grasping dataset and the Jacquard grasping dataset, to conduct experiments.

[0055] Then, the collected grasping pose detection dataset was augmented by cropping, random rotation and random scaling. The dataset was divided into two sub-datasets according to image segmentation and object segmentation to verify the model’s discrimination and generalization ability.

[0056] Step 3, construct the encoder part of GDLNet, the main steps are as follows:

[0057] The GDLNet encoder is mainly composed of stacked Deep Separable Fusion Extraction Blocks (DSFEBs), which consist of multiple Deeply Separable Convolutional and Shuffling Units. The network feature extraction stage is formed by stacking four single-branch DFEBs, fusing the 4-channel features. As input, the output is a 7x7 deep feature block. The depthwise separable fusion extraction block consists of a fusion unit and a depthwise separable convolution. The fusion unit first processes the input features... The feature extraction process splits the data into two branches with the same input and output channels. Then, for one branch, a combination of a 1×1 convolution with the same input and output channels and a 3×3 depthwise separable convolution is applied to reduce memory consumption. After feature extraction, a channel shuffling operation is used to fuse the feature information from the two branches. This channel shuffling feature recombination improves the information interaction between different channels, enhancing the ability to extract features from the target region. The shuffled features are then fed into a 3×3 depthwise separable convolution with a stride of 2, reducing the input image feature resolution while minimizing additional computational overhead. Finally, the output features are... .

[0058] The extraction blocks based on fusion units and depthwise separable convolutions are described as follows:

[0059] (1)

[0060] in, This represents the extracted output features of the DSFEB block; This indicates that the inference block captures input features; Indicates channel mixing and 3×3 depthwise separable convolution; This represents the features captured after 1×1 convolution and depthwise separable convolution.

[0061] The context aggregation layer is placed after the deep features of the backbone network. By introducing a context aggregation strategy based on the grasping region, the network is guided to enhance the features of the graspable region and suppress redundant features of non-grasping pixels, outputting a 7x7 grasping region aggregated feature. The proposed region context extraction strategy leverages deep features. 1×1 convolution is used to generate coarse-grained classification features of size 2×7×7. coarse-grained classification features We directly upsample to the original image size using interpolation, and then use the cross-entropy function to optimize the features. The auxiliary loss is calculated with the labels; then the coarse-grained classification features are used. 512×7×7 features generated by 3×3 depthwise separable convolution Multiply to obtain a 2×512 grasping region representation. After the above operations are completed, the captured region will be represented. With features Each pixel is weighted using a self-attention mechanism to obtain a 512×7×7 size grasping region attention context information; finally, the grasping region attention context information is combined with features. The summation updates the representation of each pixel.

[0062] The aggregation strategy based on the crawled region context is described as follows:

[0063] (2)

[0064] (3)

[0065] in, Indicates deep features coarse-grained classification features Representing deep features Through depthwise separable convolution and batch normalization, This represents the crawlable region. , , Indicates Transition functions for convolution, batch normalization, and activation functions. This indicates the GRCA capture output features.

[0066] The feature reconstruction layer uses multiple depthwise separable reconstruction blocks (DSRBs), which consist of bilinear interpolation, depthwise separable convolution, batch normalization, and ReLU activation functions stacked together, to replace the unmaxpooling commonly used in decoders, and outputs features. Input features Output after sampling by 5 upsampling modules Features are used to fill unknown pixels using bilinear interpolation.

[0067] The upsampling block based on interpolation algorithm and depthwise separable convolution is described as follows:

[0068] (4)

[0069] in, This indicates that the upsampling block captures the input features; This represents the reconstruction function consisting of a 3×3 depth separable convolutional layer, a batch normalized layer (BN), and a nonlinear activation function (ReLU); This represents the capture and reconstruction features of the upsampling module.

[0070] Step 4, train GDLNet, the specific steps include:

[0071] The established grasping pose detection training dataset is input into the network, and the loss is calculated by forward propagation. The partial derivatives of the objective function with respect to the features are solved. The gradient is obtained by backpropagation algorithm and the parameters are updated.

[0072] This invention evaluates the true set of crawled tags by using a crawling area network loss function. With the predicted set of crawlable regions The differences.

[0073] The crawling region network loss function consists of a crawling region context aggregation auxiliary loss function and crawling position, angle and width regression loss functions.

[0074] The auxiliary loss function for capturing region context aggregation is defined as follows:

[0075] (5)

[0076] in, For the first The ground truth of the capture region for each sample. For the corresponding number Each sample predicts the probability of a crawlable region, where the probability of each pixel in that region represents the crawlable probability at that location in each real input image of size 224×224 pixels. This represents the total number of targets captured.

[0077] The regression loss function for capture position, angle, and width is defined as follows:

[0078] (6)

[0079] in, It is the sum of the captured targets. and These are the actual tags crawled. The corresponding ground truth region and the predicted region features output by the network.

[0080] The total loss function of the GDLNet network is defined as:

[0081] (7)

[0082] in, To aggregate auxiliary loss for capturing regional context; Location confidence ,angle ,width loss.

[0083] Figure 6 For example, here is a screenshot of the encoding / decoding detection results. Figure 6As shown in the eighth line of truth, the graspable areas of these objects often have regional inconsistency with the overall shape of the object, and are usually concentrated on the edges of the object. These edges are the areas that the end effector of the robotic arm can grasp. Figure 6 (a) Figure 6 (b) Figure 6 (g) The disc-shaped object has a similar color to the background, making it difficult to accurately identify the features of the grasping area. The interference of the non-grasping area on the graspable area in the grasping detection will also seriously affect the grasping detection performance. Figure 6 (c) Figure 6 (d) Figure 6 (e) shows that not all pixels on an object are graspable regions; grasping on these objects is more easily affected by their central region. Many objects, although significantly different from the background, have graspable and non-graspable regions with the same color, which can lead to grasping difficulties. Figure 6 (f) Figure 6 As shown in (h);

[0084] like Figure 6 As shown in the figure, the GDLNet proposed in this invention significantly improves the accuracy of grasping and detecting difficult samples, and also improves the overall success rate of grasping and detecting target objects. Therefore, the proposed network not only adapts well to the feature discrimination ability of graspable regions and objects with inconsistent features, such as... Figure 6 (a) Figure 6 (b) Figure 6 As shown in (g), the GDLNet network proposed in this invention effectively distinguishes between high-confidence grasped regions and non-grasped regions by enhancing the backbone feature extraction capability and the target grasping region context aggregation strategy, thus suppressing the occurrence of non-grasped regions; Figure 6 (c) Figure 6 (d) Figure 6 As shown in (e), the GDLNet network proposed in this invention has a good suppression effect on non-grabbing regions. In the figure, this is demonstrated by its ability to effectively locate the grabbing region within the graspable area of ​​objects such as the top of a round hat, rather than the entire hat object. Therefore, this network can capture more graspable regions, improving the accuracy of grabbing detection. Figure 6 (f) Figure 6As shown in (h), the proposed GDLNet network not only accurately distinguishes between the background and the object, but also significantly suppresses the interference in the center region caused by the color consistency between the graspable and non-grasping regions, further improving the grasping detection accuracy. In summary, the comparative experimental results demonstrate that the proposed GDLNet network can better address the problem of grasping region discrimination errors when there is inconsistency between the graspable and object regions by utilizing the grasping region context strategy. Furthermore, it has a wider adaptive range within the graspable region, resulting in stronger robustness and applicability.

[0085] In summary, this invention provides a lightweight encoding / decoding grasping pose detection method. The proposed encoding / decoding grasping pose detection network GDLNet first uses an encoding / decoding network architecture as its foundation, employing depthwise separable convolutions and shuffling units to construct a target feature depthwise separation and fusion extraction block, reducing the number of parameters in the encoding network and enhancing the network's ability to extract features from the grasping region. Second, it utilizes bilinear interpolation and depthwise separable convolutions to establish a depthwise separation feature reconstruction block, effectively reducing the number of parameters in the decoding network while recovering lost information from high-level features. Finally, addressing the inconsistency between the pixel representation of the graspable region and the overall picture of the target object, a target grasping region context aggregation strategy is proposed based on cross-entropy-assisted loss and a self-attention mechanism. This strategy guides the network to enhance the features of the graspable target region and suppress redundant features of non-grasping pixels. This invention not only reduces the number of parameters in the encoding / decoding network and enhances the network's ability to extract features from the grasping region, but also effectively enhances the features of the graspable target region and suppresses redundant features of non-grasping pixels, exhibiting superior robustness.

[0086] The present invention also provides a lightweight encoding and decoding grasping pose detection system, including a data acquisition module, a network training module and an encoding and decoding grasping detection module;

[0087] The data acquisition module is used to acquire images containing object grasping poses and construct a grasping pose detection dataset.

[0088] The network training module is used to train the pre-built encoding / decoding grasping detection network using the acquired grasping pose detection dataset, so as to obtain the trained encoding / decoding grasping pose detection network.

[0089] The encoding / decoding grasping and detection module is used to perform grasping pose detection on objects in the grasping scene to be extracted using a trained encoding / decoding grasping pose detection network.

[0090] The present invention also provides a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, such as a lightweight encoding / decoding capture pose detection program.

[0091] When the processor executes the computer program, it implements the steps of the above-described lightweight encoding and decoding grasping pose detection method, such as: acquiring an image containing object grasping poses and constructing a grasping pose detection dataset.

[0092] The pre-built encoding / decoding grasping detection network is trained using the acquired grasping pose detection dataset to obtain the trained encoding / decoding grasping pose detection network.

[0093] Using a trained encoder-decoder grasping pose detection network, the grasping pose of objects in the grasping scene to be extracted is detected.

[0094] Alternatively, the processor may execute the computer program to implement the functions of each module in the aforementioned system.

[0095] For example: the data acquisition module is used to acquire images containing object grasping poses and construct a grasping pose detection dataset;

[0096] The network training module is used to train the pre-built encoding / decoding grasping detection network using the acquired grasping pose detection dataset, so as to obtain the trained encoding / decoding grasping pose detection network.

[0097] The encoding / decoding grasping and detection module is used to perform grasping pose detection on objects in the grasping scene to be extracted using a trained encoding / decoding grasping pose detection network.

[0098] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the mobile device. For example, the computer program can be divided into a data acquisition module, a network training module, and an encoding / decoding capture and detection module, with the specific functions of each module as follows:

[0099] The data acquisition module is used to acquire images containing object grasping poses and construct a grasping pose detection dataset.

[0100] The network training module is used to train the pre-built encoding / decoding grasping detection network using the acquired grasping pose detection dataset, so as to obtain the trained encoding / decoding grasping pose detection network.

[0101] The encoding / decoding grasping and detection module is used to perform grasping pose detection on objects in the grasping scene to be extracted using a trained encoding / decoding grasping pose detection network.

[0102] The mobile terminal can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The mobile terminal may include, but is not limited to, a processor and memory.

[0103] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the mobile terminal, connecting various parts of the mobile terminal via various interfaces and lines.

[0104] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the lightweight encoding / decoding capture pose detection method.

[0105] If the modules / units integrated in the mobile terminal are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0106] Based on this understanding, all or part of the processes in the above-described method can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-described real-time monitoring method for hydropower LCU controller variables. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form.

[0107] The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A lightweight encoding / decoding method for capturing pose detection, characterized in that, Includes the following steps: Obtain images containing object grasping poses and construct a grasping pose detection dataset; The pre-built encoding / decoding grasping detection network is trained using the acquired grasping pose detection dataset to obtain the trained encoding / decoding grasping pose detection network. Using a trained encoder-decoder grasping pose detection network, the grasping pose of objects in the grasping scene to be extracted is detected; The encoding / decoding grasping pose detection network includes an encoder and a decoder, wherein the encoder includes a depth separation convolution block, four depth separation fusion extraction blocks and a grasping region context aggregation block; the decoder includes five depth separation reconstruction blocks; The pre-built encoding / decoding grasping detection network is trained using the acquired grasping pose detection dataset to obtain the trained encoding / decoding grasping pose detection network. The specific method is as follows: Using the acquired grasping pose detection dataset, the pre-built encoding / decoding grasping detection network is iteratively optimized and trained by backpropagation combined with the loss function of the grasping region, resulting in the trained encoding / decoding grasping pose detection network.

2. The lightweight encoding / decoding capture pose detection method according to claim 1, characterized in that, The encoder's depth-separated convolutional block includes two convolutional layers and a depth-separable convolution, with each convolutional layer connected to a batch normalization layer and a linear rectifier.

3. The lightweight encoding / decoding capture pose detection method according to claim 1, characterized in that, The encoder's depth separation fusion extraction block includes two stages. The first stage includes a depth-separable convolutional layer and a point convolutional layer. The second stage includes a depth-separable convolution, the output of which is sequentially connected to a batch normalization layer and a linear rectifier.

4. The lightweight encoding / decoding grasping pose detection method according to claim 1, characterized in that, The encoder's capture region context aggregation block includes a depth-separable convolutional layer, a convolutional layer, and a Softmax layer; the output of the convolutional layer is connected to a point convolutional layer, a batch normalization layer, and a linear rectifier layer.

5. The lightweight encoding / decoding capture pose detection method according to claim 1, characterized in that, Each decoder's depth separation and reconstruction block includes a bilinear interpolation layer, and each bilinear interpolation layer is connected to a depth-separable convolution, a batch normalization layer, and a linear rectified layer.

6. A lightweight encoding / decoding grasping pose detection system, used to implement the lightweight encoding / decoding grasping pose detection method according to any one of claims 1-5, characterized in that, include The data acquisition module is used to acquire images containing object grasping poses and construct a grasping pose detection dataset. The network training module is used to train the pre-built encoding / decoding grasping detection network using the acquired grasping pose detection dataset, so as to obtain the trained encoding / decoding grasping pose detection network. The encoding / decoding grasping and detection module is used to perform grasping pose detection on objects in the grasping scene to be extracted using a trained encoding / decoding grasping pose detection network.

7. A mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the lightweight encoding / decoding capture pose detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the lightweight encoding / decoding capture pose detection method as described in any one of claims 1-5.

Images