Mechanical arm grabbing method and system, electronic equipment and medium

Through the combination of GGCNN and SAC deep reinforcement learning, a closed-loop control system is formed, which solves the problem of separation of target recognition and decision-making control in robotic arm grabbing, improves the crawling success rate and timeliness of task execution, and is suitable for complex environments and not Regularly shaped objects.

CN119036465AInactive Publication Date: 2024-11-29SHANGHAI UNIV OF ENG SCI
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411437752.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-15
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing robotic arm grasping technology, the separation of target recognition and decision-making control leads to insufficient timeliness and coherence in task execution, especially when dealing with complex environments and irregularly shaped objects, the acquisition success rate is low, and the calculation amount increases greatly. Overall model training Complexity.

Method used

Combined with GGCNN to predict the position and posture of objects, and introduced deep reinforcement learning SAC method for strategy generation and real-time adjustment, forming a closed-loop control system to ensure the effectiveness and accuracy of the crawling strategy.

Benefits of technology

It improves the timeliness and coherence of robotic arm gripping, enhances the success rate of grabbing complex environments and irregularly shaped objects, reduces errors, and ensures efficient and accurate task execution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119036465A_ABST
    Figure CN119036465A_ABST
Patent Text Reader

Abstract

The invention relates to a mechanical arm grabbing method and system, electronic equipment and a medium. The method comprises the steps that a depth image of a grabbed object is obtained and preprocessed; the preprocessed image is input into the trained GGCNN to be processed, and object grabbing information is output; the object grabbing information is input into an SAC optimization block, the SAC optimization block carries out strategy generation and strategy updating, and a grabbing strategy is output; an execution state obtained after the mechanical arm receives the grabbing strategy and executes the grabbing strategy is obtained, real-time adjustment is conducted through the GGCNN and the SAC optimization block according to the execution state, and the grabbing task is ended; the mechanical arm grabs the target object through the grabbing method. Compared with the prior art, target recognition and grabbing strategy control are effectively combined, the grabbing task can be executed more accurately in a complex environment, and task execution timeliness and coherence are guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a robotic arm grasping method and system, electronic equipment and medium. Background Art

[0002] In recent years, robots have been increasingly used in various fields and scenarios. With the cooperation of robots, many jobs and tasks that are difficult to complete by human labor can be solved, and human work can be simpler and more efficient. For example, a grasping robot equipped with a robotic arm and a grasping function can replace humans to grasp workpieces and other tasks. The robot is tireless and can lift heavy objects. It has many better capabilities than human hands. Therefore, it is of great significance to study the grasping technology of robotic arms.

[0003] China's patent application "CN115861780A" discloses a robot arm detection and grasping method based on YOLO-GGCNN. This method uses the YOLOv4 model as the target grasping part and combines the GGCNN network to output the optimal robot arm grasping solution. Although it avoids the influence of other non-grasping objects in the grasping scene on the grasping detection, the introduction of yolov4 increases the calculation amount of the detection process and cannot guarantee the timeliness. The application scenario is very limited, which increases the complexity of the overall model training; China's patent application "CN117400254A" discloses a robot arm path planning method, device and medium based on SAC deep reinforcement learning. The soft actor-critic SAC deep reinforcement learning model is used to plan the path of the robot arm. Although it improves the accuracy of the robot arm grasping to a certain extent, for irregular objects, if the grasping point is not properly selected, the grasping failure cannot be avoided. In addition, both of the above methods separate target recognition and decision control, which will cause the execution of tasks to be disconnected during the actual execution of tasks by the robot arm, and the timeliness and consistency of task execution cannot be guaranteed. Summary of the invention

[0004] The purpose of the present invention is to provide a robot arm grasping method and system, electronic equipment and medium in order to overcome the defects of the above-mentioned prior art.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to a first aspect of the present invention, a robot arm grasping method is provided, the method comprising:

[0007] Obtain the depth image of the grasped object and perform preprocessing;

[0008] The preprocessed image is input into the trained GGCNN for processing, and the object grasping information including the object's position, posture and grasping score is output;

[0009] Inputting the object grasping information into the SAC optimization block, the SAC optimization block performs strategy generation and strategy update, and outputs the grasping strategy;

[0010] The execution status obtained after the robot arm receives and executes the grasping strategy is obtained, and the GGCNN and SAC optimization blocks are used to make real-time adjustments according to the execution status to end the grasping task.

[0011] As a preferred technical solution, the pretreatment comprises the following steps:

[0012] The depth image is cropped and scaled to the specified specifications to obtain an intermediate image;

[0013] Fix invalid depth values ​​in intermediate images to get preprocessed images.

[0014] As a preferred technical solution, the process of GGCNN processing specifically includes:

[0015] Get the capture quality of each pixel in the preprocessed image;

[0016] Calculate the grasp of each pixel using the pixel coordinate center point of the preprocessed image, the rotation angle in the camera coordinate system, the grasp width, and the pixel grasp quality;

[0017] Construct a grab image function based on the grab of each pixel;

[0018] Use the grab image function to output object grab information.

[0019] As a preferred technical solution, the training process of GGCNN is as follows:

[0020] Obtain a data set, and randomly extract three parts from the data set as a training set, a test set, and a validation set, respectively; the data set includes RGB-D images of objects of different shapes and structures and successful grasping position marks based on grasping attempts performed in a simulated environment;

[0021] Use the training set to train the model, use the validation set to adjust the model parameters, and finally use the test set to evaluate the trained model;

[0022] The model with the best performance is selected as the final GGCNN.

[0023] As a preferred technical solution, the training process of the SAC optimization block is: constructing an experience replay pool; using the reincarnation experience replay pool to train the SAC optimization block;

[0024] The experience replay pool includes the current execution state, robot arm action, reward and updated execution state; the SAC optimization block includes a policy network, two Q networks and two target Q networks;

[0025] Among them, the loss function of the Q network is:

[0026]

[0027] Among them, L Q (ω) represents the loss function, ω j Indicates the Q network type, i.e. s t Indicates the execution status, a t Indicates the robot arm movement, r t It means that the reward is obtained through the preset reward function and the reward function expression is: t represents the time step, R represents the experience replay pool, π θ represents the policy network value, Q ω (s t ,a t ) means in state s t Take action a t The predicted value of Represents the current reward r t and the next state s t+1 and action a t+1 The discounted reward; α represents the regularization coefficient; γ represents the discount factor;

[0028] The loss function of the regularization coefficient α is:

[0029]

[0030] Among them, L(α) represents the regularization coefficient loss function, Represents the state s in the experience replay pool R t And the action a generated based on the strategy π t+1 Perform expected value calculation, ζlog π (a t |s t ) indicates that the calculation is in state s t Next select action a t The logarithmic probability, H 0 represents the preset target entropy value, and ζ represents the preset coefficient.

[0031] As a preferred technical solution, the specific steps of the real-time adjustment include:

[0032] The SAC optimization block receives the object grasping information and generates the first strategy;

[0033] Obtain the execution status of the robot arm after receiving and executing the first strategy. If the execution status is successful, end the grasping task. If the execution status is failed, use the SAC optimization block to update the strategy and generate a new execution strategy.

[0034] Update the execution status of the robot after executing the new execution strategy;

[0035] If the SAC optimization block performs strategy updates to the preset maximum number of times and the robot fails to execute the last new execution strategy, GGCNN is used to reprocess and output the object grasping information.

[0036] As a preferred technical solution, the expression of the generation strategy is:

[0037]

[0038] Among them, α represents the regularization coefficient, s represents the execution state, a represents the robot action, π represents the strategy at the previous moment, and π * represents the optimal strategy for output, t represents the time step, r represents the reward, and H represents the entropy value corresponding to the strategy.

[0039] According to a second aspect of the present invention, a robotic arm grasping system is provided, the robotic arm grasping system comprising:

[0040] The depth image acquisition module is used to obtain the depth image of the object to be grasped;

[0041] The image processing module is used to receive and preprocess the depth image and generate object grasping information through GGCNN;

[0042] The SAC optimization block is used to receive object grasping information and generate grasping strategies;

[0043] The robot control module is used to execute the grasping strategy and output the execution status;

[0044] The feedback module is used to receive and judge the execution status, and feed back the judgment result to the image processing module or the SAC optimization block to adjust the grasping strategy in real time.

[0045] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0046] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described when executed by a processor.

[0047] Compared with the prior art, the present invention provides a robotic arm grasping method, which uses GGCNN as target recognition and introduces a deep reinforcement learning SAC method as decision control, effectively solving the separation problem of target recognition and decision control in the prior art, and can ensure the timeliness and consistency of task execution; and the entire grasping process is realized by two sets of closed-loop control, which can provide more accurate prediction of grasping quality and grasping position for objects in complex environments and / or irregular shapes, thereby improving the overall grasping success rate and reducing erroneous operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of the robot arm grasping method of the present invention;

[0049] Figure 2 The present invention is a SAC optimized block training terminal capture process. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, ordinary technicians in this field can obtain the above-mentioned technical solutions without creative work.

[0051] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0052] Embodiment 1:

[0053] This embodiment provides a robotic arm grasping method, which utilizes image depth information and uses a Generative Grasping Convolutional Neural Network (GGCNN) to predict the position of the grasped object, and can evaluate the grasping quality and posture of each pixel to predict the optimal grasping posture, and then use a deep reinforcement learning algorithm SAC (Soft Actor-Critic) method to train the robotic arm to autonomously perform grasping operations.

[0054] The flow chart of this method is as follows Figure 1 The specific steps include:

[0055] S1. Obtain the depth image of the grasped object and perform preprocessing:

[0056] S11, using a depth camera to obtain a depth image of the captured object;

[0057] S12, crop the depth image into a square and scale it to 300*300 pixels to obtain an intermediate image suitable for GGCNN input;

[0058] S13. Use OpenCV to repair the invalid depth value of the intermediate image to obtain a preprocessed image.

[0059] S2. Output object grasping information:

[0060] S21, obtaining the grasping quality of each pixel in the preprocessed image, where the grasping quality is generated during the GGCNN training process;

[0061] S22, calculating the grasping of each pixel point by using the pixel coordinate center point of the preprocessed image, the rotation angle in the camera coordinate system, the grasping width, and the pixel grasping quality;

[0062] S23, based on the capture of each pixel, a capture image function is constructed through a convolutional neural network approximation, and the capture image function expression is:

[0063] G=(Φ,W,Q)∈O 3*H′*W ,

[0064] Where G represents the image capture function, Φ represents the camera rotation angle and satisfies W represents the coordinate capture width in the depth image, Q represents the pixel capture quality set, and O 3*H′*W represents the depth image, H′ represents the grasping height;

[0065] S24. Outputting object grasping information including the position, posture and grasping score of the object using the grasping image function.

[0066] S3, input the object grasping information into the SAC optimization block, which generates and updates the strategy and outputs the grasping strategy;

[0067] S4. Obtain the execution status obtained after the robot arm receives and executes the grasping strategy. Use the GGCNN and SAC optimization blocks to make real-time adjustments according to the execution status to end the grasping task. For detailed process, see Figure 2 , the specific steps include:

[0068] S41, SAC optimization block receives object grasping information and generates a first strategy. The expression of the generated strategy is:

[0069]

[0070] Among them, α represents the regularization coefficient, s represents the execution state, a represents the robot action, π represents the strategy at the previous moment, and π * represents the optimal strategy for output, t represents the time step, r represents the reward, and H represents the entropy value corresponding to the strategy.

[0071] S42, obtaining the execution status of the robot arm after receiving and executing the first strategy, if the execution status is successful, ending the grasping task, if the execution status is failed, using the SAC optimization block to update the strategy, for generating a new execution strategy, wherein the successful execution status refers to the scenario where the object to be grasped is lifted by the robot arm and leaves the workbench;

[0072] S43, updating the execution status of the robot after executing the new execution strategy. If the SAC optimization block updates the strategy a maximum number of times and the robot fails to execute the last new execution strategy, GGCNN is used to reprocess and output the object grasping information.

[0073] Among them, the training process of the above GGCNN is:

[0074] A1. Obtain a data set and randomly extract three parts from the data set as training set, test set and validation set respectively;

[0075] The dataset includes: RGB-D images of objects of different shapes and structures and successful grasping position marks based on grasping attempts performed in a simulated environment; in order to allow the depth camera to have different views of the object, that is, to create multiple different postures for each object, each view corresponds to four image files and a label text file. The image contains an RGB image, a depth image, an image that uses an off-the-shelf stereo vision algorithm to represent the depth of the noise, a binary mask image that separates the object and the background, and a label file containing an array representing the grasp;

[0076] A2. Use the training set to train the model, use the validation set to adjust the model parameters, and finally use the test set to evaluate the trained model;

[0077] A3. Select the model with the best performance as the final GGCNN.

[0078] The training process of the above SAC optimization block is:

[0079] B1. Construct an experience replay pool R and a SAC optimization block. The experience replay pool includes the current execution state, robot arm action, reward, and updated execution state. The SAC optimization block includes a policy network, two Q networks, and two target Q networks.

[0080] B2. Calculate the loss function of the Q network based on the reincarnation experience pool, and its expression is:

[0081]

[0082] Among them, L Q (ω) represents the loss function, ω j Indicates the Q network type, i.e. s t Indicates the execution status, a t Indicates the robot arm movement, r t It means that the reward is obtained through the preset reward function and the reward function expression is: t represents the time step, R represents the experience replay pool, π θ represents the policy network value, Q ω (s t ,a t ) means in state s t Take action a t The predicted value of Represents the current reward r t and the next state s t+1 and action a t+1 The discounted reward; γ represents the discount factor;

[0083] α represents the regularization coefficient, and its loss function expression is:

[0084]

[0085] Among them, L(α) represents the regularization coefficient loss function, Represents the state s in the experience replay pool R t And the action a generated based on the strategy π t+1 Perform expected value calculation, ζlog π (a t |s t ) indicates that the calculation is in state s t Next select action a t The logarithmic probability, H 0 represents the preset target entropy value, and ζ represents the preset coefficient.

[0086] B3, continuously adjust the SAC optimization block parameters based on the loss function and the experience replay pool;

[0087] B4. Evaluate the performance of the SAC optimization blocks after parameter adjustment and select the best performing SAC optimization block as the final SAC optimization block.

[0088] Embodiment 2:

[0089] This embodiment also provides a robotic arm grasping system, the robotic arm grasping system comprising:

[0090] Depth image acquisition module: used to obtain the depth image of the object to be grasped;

[0091] Image processing module: used to receive and preprocess depth images and generate object grasping information through GGCNN;

[0092] SAC optimization block: used to receive object grasping information and generate grasping strategies;

[0093] Robotic arm control module: used to execute the grasping strategy and output the execution status;

[0094] Feedback module: used to receive and judge the execution status, and feed back the judgment result to the image processing module or SAC optimization block to adjust the grasping strategy in real time.

[0095] Those skilled in the art can clearly understand that for the convenience and brevity of description, reference can be made to the corresponding process in the aforementioned method embodiment, which will not be described in detail here.

[0096] Embodiment 3:

[0097] The present invention also provides an electronic device, which includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or computer program instructions loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0098] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0099] The processing unit performs the various methods and processes described above, such as methods S1 to S4, A1 to A3, and B1 to B4. For example, in some embodiments, methods S1 to S4, A1 to A3, and B1 to B4 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S4, A1 to A3, and B1 to B4 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S4, A1 to A3, and B1 to B4 by any other appropriate means (e.g., by means of firmware).

[0100] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0101] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A robotic arm grasping method, characterized in that: The method includes: Obtain the depth image of the grasped object and perform preprocessing; The preprocessed image is input into the trained GGCNN for processing, and the object grasping information including the object's position, posture and grasping score is output; Inputting the object grasping information into the SAC optimization block, the SAC optimization block performs strategy generation and strategy update, and outputs the grasping strategy; The execution status obtained after the robot arm receives and executes the grasping strategy is obtained, and the GGCNN and SAC optimization blocks are used to make real-time adjustments according to the execution status to end the grasping task.

2. A robot arm grasping method according to claim 1, characterized in that: The pretreatment comprises the following steps: The depth image is cropped and scaled to the specified specifications to obtain an intermediate image; Fix invalid depth values ​​in intermediate images to get preprocessed images.

3. A robot arm grasping method according to claim 1, characterized in that: The processing process of the GGCNN specifically includes: Get the capture quality of each pixel in the preprocessed image; Calculate the grasp of each pixel using the pixel coordinate center point of the preprocessed image, the rotation angle in the camera coordinate system, the grasp width, and the pixel grasp quality; Construct a grab image function based on the grab of each pixel; Use the grab image function to output object grab information.

4. A robot arm grasping method according to claim 1, characterized in that: The training process of GGCNN is as follows: Obtain a data set, and randomly extract three parts from the data set as a training set, a test set, and a validation set, respectively; the data set includes RGB-D images of objects of different shapes and structures and successful grasping position marks based on grasping attempts performed in a simulated environment; Use the training set to train the model, use the validation set to adjust the model parameters, and finally use the test set to evaluate the trained model; The model with the best performance is selected as the final GGCNN.

5. A robot arm grasping method according to claim 1, characterized in that: The training process of the SAC optimization block is as follows: constructing an experience replay pool; using the reincarnation experience replay pool to train the SAC optimization block; The experience replay pool includes the current execution state, robot arm action, reward and updated execution state; the SAC optimization block includes a policy network, two Q networks and two target Q networks; Among them, the loss function of the Q network is: Among them, L Q (ω j ) represents the loss function, ω j Indicates the Q network type, i.e. s t Indicates the execution status, a t Indicates the robot arm movement, r t It means that the reward is obtained through the preset reward function and the reward function expression is: t represents the time step, R represents the experience replay pool, π θ represents the policy network value, Q ω (s t ,a t ) means in state s t Take action a t The predicted value of Represents the current reward r t and the next state s t+1 and action a t+1 The discounted reward; α represents the regularization coefficient; γ represents the discount factor; The loss function of the regularization coefficient α is: Among them, L(α) represents the regularization coefficient loss function, Represents the state s in the experience replay pool R t And the action a generated based on the strategy π t+1 Perform expected value calculation, ζlog π (a t |s t ) indicates that the calculation is in state s t Next select action a t The logarithmic probability of , H0 represents the preset target entropy value, and ζ represents the preset coefficient.

6. A robot arm grasping method according to claim 1, characterized in that: The specific steps of the real-time adjustment include: The SAC optimization block receives the object grasping information and generates the first strategy; Obtain the execution status of the robot arm after receiving and executing the first strategy. If the execution status is successful, end the grasping task. If the execution status is failed, use the SAC optimization block to update the strategy and generate a new execution strategy. Update the execution status of the robot after executing the new execution strategy; If the SAC optimization block performs strategy updates to the preset maximum number of times and the robot fails to execute the last new execution strategy, GGCNN is used to reprocess and output the object grasping information.

7. A robot arm grasping method according to claim 6, characterized in that: The expression for generating the strategy is: Among them, α represents the regularization coefficient, s represents the execution state, a represents the robot action, π represents the strategy at the previous moment, and π * represents the optimal strategy for output, t represents the time step, r represents the reward, and H represents the entropy value corresponding to the strategy.

8. A robotic arm grasping system, characterized in that: The robotic arm grasping system comprises: The depth image acquisition module is used to obtain the depth image of the object to be grasped; The image processing module is used to receive and preprocess the depth image and generate object grasping information through GGCNN; The SAC optimization block is used to receive object grasping information and generate grasping strategies; The robot control module is used to execute the grasping strategy and output the execution status; The feedback module is used to receive and judge the execution status, and feed back the judgment result to the image processing module or the SAC optimization block to adjust the grasping strategy in real time.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Body robot high-speed grabbing control method based on image processing

    CN121733590A

  • Image Processing-Based High-Speed ​​Grasping Control Method for Embossed Robots

    CN121733590B