Target grabbing method and device of robot, robot and medium
By generating target affordance maps and operation affordance maps, combined with imitation learning and reinforcement learning models, the problems of poor generalization ability of robot object grasping and long training time in the existing technology are solved, and efficient and flexible universal object grasping is achieved.
Patent Information
- Application Number
- CN202411549526.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the existing robot object grasping technology, imitation learning has limitations in generalization ability of generalized objects, while reinforcement learning has a long training time, resulting in inefficiency in practical applications.
By obtaining images containing the target to be captured, a target availability map and an operation availability map are generated, and a simulation learning and reinforcement learning model is used to plan motion trajectories and grab actions to achieve flexible capture of general objects by the robot.
It improves the applicability and generalization ability of the robot to general objects, shortens the training time, and improves the efficiency in practical applications.
Smart Images
Figure CN119526386A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of robotics technology, and in particular to a method, device, robot and medium for grasping a target by a robot. Background Art
[0002] In the related technologies, the robot's object grasping technology mostly adopts imitation learning and reinforcement learning. Among them, imitation learning is a supervised learning method, which learns motion strategies by collecting teaching data; its advantage is that its training time is short, but it has certain limitations in the generalization ability of general objects. After training, it is difficult to transfer to other objects, which reduces the scope of application of imitation learning. Reinforcement learning has better generalization. As long as there are relevant objects in the training environment, reinforcement learning can achieve grasping of different objects by designing a suitable reward function as a guide. However, the design of the reward function is quite complicated, and there are many factors to consider. In addition, the training time of reinforcement learning is usually long, which limits its efficiency in practical applications to a certain extent. Summary of the invention
[0003] The embodiments of the present application provide a robot target grasping method, device, robot and medium, so as to enable the robot to grasp common objects, thereby improving applicability and generalization ability.
[0004] The specific technical solutions provided by the embodiments of this application are as follows:
[0005] In a first aspect, an embodiment of the present application provides a target grasping method of a robot, comprising:
[0006] Acquire a first image containing a target to be captured, and generate a target availability map of a region containing the target based on the first image;
[0007] Based on the target affordance graph, planning a first motion trajectory of the robot approaching the target, and moving according to the first motion trajectory;
[0008] After moving to the vicinity of the target, acquiring a second image containing the target, and generating an operational affordance map of a grasping area containing the target based on the second image;
[0009] Based on the operation affordance graph, a second motion trajectory of the robot for grasping the target is planned, and grasping the target is completed according to the second motion trajectory.
[0010] A target grasping method for a robot provided in an embodiment of the present application generates a target availability map of the area where the target is located based on a first image, so that the robot can move to the area where the target in the target availability map is located, rather than moving to the specific target to be grasped in the first image, thereby improving the wide applicability of the method and the generalization ability, so that the robot can complete the movement approaching the area where the target included in the target availability map is located in a variety of scenarios; further, after the robot moves to the vicinity of the target, based on the acquired second image containing the target, an operation availability map of the grasping area containing the target is generated, and based on the operation availability map, the specific trajectory for grasping the target is planned, thereby further improving the generalization ability of the method, so that the robot can complete flexible grasping of different objects.
[0011] In a possible implementation manner, generating, based on the first image, a target availability map of the area where the target is located, includes:
[0012] Based on the description information of the target included in the first grabbing instruction, perform target recognition on the first image to identify the area of the target in the first image; or,
[0013] Inputting the first grabbing instruction and the first image into a visual language model, and using the description information of the target included in the first grabbing instruction to guide the visual language model to perform target recognition in the first image, and recognize the region of the target in the first image;
[0014] In the first image, the area is filled with a first color to obtain the target availability map.
[0015] In the above method, the specific type and shape of the grasping target are weakened, so that the robot only moves towards the area filled with the first color when planning the trajectory, rather than towards a specific object. This allows the robot to cope with motion trajectory planning in a variety of scenarios, improving its applicability and generalization ability.
[0016] In a possible implementation, planning a first motion trajectory of the robot approaching the target based on the target affordance graph, and moving according to the first motion trajectory includes:
[0017] Inputting the target affordance graph and the current posture of the robot into an imitation learning model, wherein the imitation learning model is obtained by performing at least one round of training on the imitation learning model to be trained based on a plurality of teaching data, each teaching data comprising a target affordance graph based on a sample target and a motion trajectory of the posture planning of the robot;
[0018] Using the imitation learning model to plan the posture of the robot in the process of moving toward the target based on the target affordance graph and the current posture of the robot, so as to obtain the first motion trajectory;
[0019] generating, according to the first motion trajectory, motion instructions for each joint of the robot to move;
[0020] According to the joint motion instructions, the joint machines of the robot are driven to work so as to move according to the first motion trajectory.
[0021] In the above method, the imitation learning model can be used to directly plan trajectories based on images, so that the robot can quickly adapt and plan effective movement trajectories when facing diverse backgrounds.
[0022] In a possible implementation, generating, based on the second image, an operation affordance map of a grab area including the target includes:
[0023] Performing target recognition on the second image, and determining a capture area of the target in the second image based on the recognized appearance information of the target in the second image; or,
[0024] Inputting the second grabbing instruction and the second image into a visual language model, and using the grabbing position information of the target included in the second grabbing instruction, guiding the visual language model to mark a grabbing area of the target in the second image;
[0025] In the second image, the grab area is filled with a second color to obtain an operation affordance map of the target.
[0026] In the above method, the specific outline of the target's grasping area is weakened, so that when the robot plans the grasping action, it only plans for the grasping area in the operation availability graph filled with the second color, so that the robot can still maintain an efficient grasping effect when facing new objects and scenes.
[0027] In a possible implementation, planning a second motion trajectory for the robot to grasp the target based on the operation affordance graph, and completing grasping the target according to the second motion trajectory includes:
[0028] Inputting the operation affordance graph and the current state of the robotic arm into a reinforcement learning model, wherein the reinforcement learning model is obtained by training the reinforcement learning model to be trained for at least one round based on multiple samples and reward and penalty criteria, and each sample includes the operation affordance graph of the sample target and the initial state of the robotic arm;
[0029] Using the reinforcement learning model, the following operations are performed for each pending state of the robotic arm until the target is grasped, wherein the first pending state is the current state, and the second motion trajectory includes multiple sub-motion trajectories:
[0030] Using the reinforcement learning model, according to the operation affordance graph and the state to be processed, from various grasping strategies learned by the reinforcement learning model through training, a grasping strategy matching the state to be processed and the operation affordance graph is screened out, wherein each grasping strategy includes a motion trajectory of moving from one state to another state in the process of grasping a target;
[0031] According to the motion trajectory included in the screened grasping strategy, the robot arm is controlled to move, and when the robot arm completes the motion trajectory, the intermediate state of the robot arm is determined, and the state to be processed is updated with the intermediate state.
[0032] In the above method, the reinforcement learning model is mainly used to process the grasping action based on the operation affordance graph. By training the reinforcement learning model, more general and flexible grasping skills can be learned. With the strong generalization ability of the reinforcement learning model, the robot can still maintain efficient grasping effect when facing new objects and scenes.
[0033] In a possible implementation, the reward and punishment criteria include part or all of the following:
[0034] The rewarding robot arm is rewarded when the distance of the gripper of the robot arm close to the grasping area in the operation affordance graph of the sample target is less than a first threshold;
[0035] The rewarding robot arm is rewarded for picking up the sample target within a time period shorter than a second threshold;
[0036] Rewarding the sample target that is picked up when the distance from the ground is greater than a third threshold;
[0037] The reward is that the distance difference between the center of the sample target picked up and the center of the gripper of the robot arm is less than a fourth threshold;
[0038] The shutdown parameter value and joint limit value of the penalty robot arm are less than the fifth threshold;
[0039] The joint movement amplitude of the robot arm is penalized to be greater than the sixth threshold.
[0040] In the above method, the specific grasping action planning is completed with the help of the reinforcement learning model, which simplifies the design difficulty of the reward and punishment standards (i.e., the reward function), shortens the model training cycle, and improves the efficiency in practical applications.
[0041] In a second aspect, an embodiment of the present application provides a target grasping device of a robot, comprising:
[0042] A first acquisition module, configured to acquire a first image containing a target to be captured, and generate a target availability map of a region containing the target based on the first image;
[0043] A first planning module, configured to plan a first motion trajectory of the robot approaching the target based on the target affordance graph, and move the robot along the first motion trajectory;
[0044] A second acquisition module, configured to acquire a second image containing the target after moving to the vicinity of the target, and generate an operation affordance map of a grasping area containing the target based on the second image;
[0045] The second planning module is used to plan a second motion trajectory for the robot to grasp the target based on the operation affordance graph, and complete the grasping of the target according to the second motion trajectory.
[0046] In a possible implementation manner, the first acquisition module is specifically configured to:
[0047] Based on the description information of the target included in the first grabbing instruction, perform target recognition on the first image to identify the area of the target in the first image; or,
[0048] Inputting the first grabbing instruction and the first image into a visual language model, and using the description information of the target included in the first grabbing instruction to guide the visual language model to perform target recognition in the first image, and recognize the region of the target in the first image;
[0049] In the first image, the area is filled with a first color to obtain the target availability map.
[0050] In a possible implementation manner, the first planning module is specifically used to:
[0051] Inputting the target affordance graph and the current posture of the robot into an imitation learning model, wherein the imitation learning model is obtained by performing at least one round of training on the imitation learning model to be trained based on a plurality of teaching data, each teaching data comprising a target affordance graph based on a sample target and a motion trajectory of the posture planning of the robot;
[0052] Using the imitation learning model to plan the posture of the robot in the process of moving toward the target based on the target affordance graph and the current posture of the robot, so as to obtain the first motion trajectory;
[0053] generating, according to the first motion trajectory, motion instructions for each joint of the robot to move;
[0054] According to the joint motion instructions, the joint machines of the robot are driven to work so as to move according to the first motion trajectory.
[0055] In a possible implementation manner, the second acquisition module is specifically configured to:
[0056] Performing target recognition on the second image, and determining a capture area of the target in the second image based on the recognized appearance information of the target in the second image; or,
[0057] Inputting the second grabbing instruction and the second image into a visual language model, and using the grabbing position information of the target included in the second grabbing instruction, guiding the visual language model to mark a grabbing area of the target in the second image;
[0058] In the second image, the grab area is filled with a second color to obtain an operation affordance map of the target.
[0059] In a possible implementation manner, the second planning module is specifically configured to:
[0060] Inputting the operation affordance graph and the current state of the robotic arm into a reinforcement learning model, wherein the reinforcement learning model is obtained by training the reinforcement learning model to be trained for at least one round based on multiple samples and reward and penalty criteria, and each sample includes the operation affordance graph of the sample target and the initial state of the robotic arm;
[0061] Using the reinforcement learning model, the following operations are performed for each pending state of the robotic arm until the target is grasped, wherein the first pending state is the current state, and the second motion trajectory includes multiple sub-motion trajectories:
[0062] Using the reinforcement learning model, according to the operation affordance graph and the state to be processed, from various grasping strategies learned by the reinforcement learning model through training, a grasping strategy matching the state to be processed and the operation affordance graph is screened out, wherein each grasping strategy includes a motion trajectory of moving from one state to another state in the process of grasping a target;
[0063] According to the motion trajectory included in the screened grasping strategy, the robot arm is controlled to move, and when the robot arm completes the motion trajectory, the intermediate state of the robot arm is determined, and the state to be processed is updated with the intermediate state.
[0064] In a possible implementation, the reward and punishment criteria include part or all of the following:
[0065] The rewarding robot arm is rewarded when the distance of the gripper of the robot arm close to the grasping area in the operation affordance graph of the sample target is less than a first threshold;
[0066] The rewarding robot arm is rewarded for picking up the sample target within a time period shorter than a second threshold;
[0067] The distance of the sample target picked up by the reward clamp is greater than the third threshold;
[0068] Rewarding that the distance difference between the center of the sample target picked up and the center of the gripper of the robot arm is less than a fourth threshold;
[0069] The shutdown parameter value and joint limit value of the penalty robot arm are less than the fifth threshold;
[0070] The joint movement amplitude of the robot arm is punished to be greater than the sixth threshold.
[0071] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0072] A camera, deployed on the robot, for acquiring images;
[0073] Memory, used to store computer programs or instructions;
[0074] A processor is used to execute the computer program or instructions in the memory so that any method in the first aspect described above is executed.
[0075] In a fourth aspect, an embodiment of the present application provides a computer-readable non-volatile storage medium. When instructions in the storage medium are executed by a processor, the processor is enabled to execute any one of the methods described in the first aspect above.
[0076] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, enables the computer to execute the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] Figure 1 A schematic diagram of a process flow of a target grasping method of a robot in an embodiment of the present application;
[0078] Figure 2 A flowchart of a method for determining a target affordance graph in an embodiment of the present application;
[0079] Figure 3 A schematic diagram of the effect of a target availability graph in an embodiment of the present application;
[0080] Figure 4 A flowchart of a method for planning to approach a target by means of an imitation learning model in an embodiment of the present application;
[0081] Figure 5 A schematic diagram of a processing method of an imitation learning model in an embodiment of the present application;
[0082] Figure 6 A flowchart of a method for determining an operation affordance graph in an embodiment of the present application is shown;
[0083] Fig. 7A This is a schematic diagram of an operation affordance diagram effect in an embodiment of the present application;
[0084] Figure 7B It is a schematic diagram of another operation affordance diagram effect in an embodiment of the present application;
[0085] Figure 8 A schematic diagram of a process of planning a target grasping action with the help of a reinforcement learning model in an embodiment of the present application;
[0086] Fig. 9 A schematic diagram of the logical architecture of a target grasping device of a robot in an embodiment of the present application;
[0087] Fig.10 Schematic diagram of the physical structure of the robot in the embodiment of the present application. DETAILED DESCRIPTION
[0088] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0089] Below, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.
[0090] (1) The term "affordance" in the embodiments of the present application refers to how a part of an environment or an object can be used. Compared to the images directly perceived by the robot through the camera, the affordance graph simulates the process of human perception of complex scenes. By focusing on specific areas, it helps to design efficient robot perception and control systems.
[0091] (2) The terms "first", "second", "third", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in sequences other than those illustrated or described herein.
[0092] The following is a brief introduction to the design concept of the embodiments of the present application.
[0093] The present application relates to the field of robotics technology, and mainly to a method, device, robot and medium for grasping a target of a robot.
[0094] In current robotics technology, the training methods for object grasping are mainly based on imitation learning and reinforcement learning, which have good practical application effects.
[0095] Imitation learning is a supervised learning method that learns motion strategies by collecting teaching data. Its advantage is that its training time is short, but it has certain limitations in its generalization ability for general objects. The main reason is that imitation learning is a cloning of teaching behavior. When the teaching data only contains a certain object, it is difficult to transfer to other objects after training. For example, if the grasping target of the training is an apple, then when the grasping target changes to a pear, which is a similar but not exactly the same object in shape, it may not achieve the desired effect, which seriously limits the scope of application of imitation learning in practical applications.
[0096] Reinforcement learning has good generalization. Usually, as long as there are relevant objects in the training environment, reinforcement learning can achieve the grasping of different objects by designing a suitable reward function as a guide. However, its disadvantages are: first, the design process of the reward function is quite complicated and many factors need to be considered; second, the training time of reinforcement learning is usually long, which limits its efficiency in practical applications to a certain extent.
[0097] In the related art, there is also a way to fuse the two algorithms to give full play to the advantages of each algorithm, that is, first use imitation learning to implement the initial grasping strategy, and then fine-tune and optimize this initial strategy through reinforcement learning. In this way, since this initial strategy is implemented through imitation learning, there may still be problems in the generalization ability of general object grasping. Furthermore, although reinforcement learning is used to fine-tune the initial strategy, this does not mean that the difficulty of reward function design will be reduced. In other words, this method is just a simple combination of imitation learning and reinforcement learning. Both algorithms complete the grasping of the target. The difference is that reinforcement learning is based on the results of imitation learning. Therefore, the method of fusing imitation learning and reinforcement learning in the related art still has poor generalization ability and poor applicability for general object grasping.
[0098] In view of this, in order to solve the problems of poor generalization ability and poor applicability of object grasping of robots in related technologies, an embodiment of the present application provides a target grasping method of a robot. In the embodiment of the present application, a first image containing a target to be grasped is acquired, and based on the first image, a target availability map of the area where the target is located is generated; based on the target availability map, a first motion trajectory of the robot approaching the target is planned, and the robot moves according to the first motion trajectory; after moving to the vicinity of the target, a second image containing the target is acquired, and based on the second image, an operation availability map of the grasping area containing the target is generated; based on the operation availability map, a second motion trajectory of the robot to grasp the target is planned, and the grasping of the target is completed according to the second motion trajectory.
[0099] Thus, in an embodiment of the present application, by generating a target availability map of the area where the target is located based on the first image, the robot can move to the area where the target is located in the target availability map, rather than moving to the specific target to be grasped in the first image, thereby improving the wide applicability of the method and the generalization capability, so that the robot can complete the movement approaching the area where the target is located contained in the target availability map in a variety of scenarios; further, after the robot moves to the vicinity of the target, based on the acquired second image containing the target, an operation availability map of the grasping area containing the target is generated, and based on the operation availability map, the specific trajectory for grasping the target is planned, thereby further improving the generalization capability of the method, so that the robot can complete flexible grasping of different objects.
[0100] The target grasping method of a robot provided in an embodiment of the present application is applicable to robots, for example, robots, fixed-base robots, multi-legged robots, bionic humans, bipedal robots, humanoid / type robots, mobile robots, etc.
[0101] The preferred implementation modes of the present application are further described in detail below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.
[0102] See also Figure 1 As shown, the embodiment of the present application provides a method for grasping a target by a robot, and the specific process of the method may include but is not limited to the following:
[0103] Step 100: Acquire a first image containing a target to be captured, and based on the first image, generate a target availability map of the area where the target is located.
[0104] In some embodiments of the present application, when executing step 100 to obtain a first image including a captured target, the first image can be acquired by a camera deployed on the head of the robot; the first image can also be acquired by a camera deployed on the robotic arm of the robot; or the image acquired by the head camera and the image acquired by the camera on the robotic arm can be fused to obtain the first image. The present application does not limit the method for acquiring the first image.
[0105] In some embodiments of this application, refer to Figure 2 As shown, when generating the target availability map of the area where the target is located based on the first image in step 100, it can be specifically implemented by executing the following method:
[0106] Step 200: Based on the description information of the target included in the first grabbing instruction, perform target recognition on the first image to identify the area of the target in the first image; or, input the first grabbing instruction and the first image into a visual language model, use the description information of the target included in the first grabbing instruction to guide the visual language model to perform target recognition in the first image, and identify the area of the target in the first image.
[0107] In some embodiments of the present application, the first grabbing instruction may be a voice instruction from the user, such as, help me pick up the water cup on the table.
[0108] In other embodiments of the present application, the first grasping instruction may be a preset scenario instruction. For example, in an industrial scenario, if the grasping target is fixed, a first grasping instruction for a certain target may be issued to the robot in advance.
[0109] Optionally, the first crawling instruction includes description information of the target to be crawled, which is used to clarify context information of the crawling target.
[0110] In some embodiments of the present application, when executing step 200, a traditional target detection algorithm can be used to perform target recognition on the first image to obtain the exact location of the target, that is, the area in the first image, such as calling the target detection algorithm, identifying the target from the first image based on the description information of the target included in the first capture instruction, and determining the area of the target in the first image.
[0111] In some other embodiments of the present application, when executing step 200, the target may be recognized in the first image with the help of a visual language model, such as a multimodal large model, so as to obtain the region of the target in the first image.
[0112] Step 210: In the first image, fill the above-mentioned area with the first color to obtain the aforementioned target availability map.
[0113] In some embodiments of the present application, after executing step 200 to obtain the area of the target in the first image, step 210 is executed to fill the above area with the first color in the first image, thereby obtaining a target availability map of the target.
[0114] For example, Figure 3 As shown in , assuming that the first image obtained contains a tape measure, a small square, a large square and a medicine bottle, and the grasping target is a small square, the target affordance map of the small square is obtained, such as Figure 3 In (1), the area of the small square in the first image is filled with the first color; assuming that the grasping target is a medicine bottle, the target affordance map of the medicine bottle is obtained, such as Figure 3 In (2), the area of the medicine bottle in the first image is filled with the first color.
[0115] In this way, the specific type and shape of the grasping target are weakened, so that the robot only moves towards the area filled with the first color when planning the trajectory, rather than moving towards a certain object. This allows the robot to cope with motion trajectory planning in a variety of scenarios, improving its applicability and generalization ability.
[0116] Step 110: Based on the target availability graph, plan a first motion trajectory of the robot approaching the target, and move according to the first motion trajectory.
[0117] In some embodiments of this application, refer to Figure 4 As shown, when executing step 110, the following steps may be specifically performed:
[0118] Step 400: inputting the target affordance graph and the current posture of the robot into an imitation learning model, wherein the imitation learning model is obtained by performing at least one round of training on the imitation learning model to be trained based on a plurality of teaching data, each teaching data including a target affordance graph based on a sample target and a motion trajectory of the posture planning of the robot;
[0119] Step 410: using the imitation learning model to plan the posture of the robot in the process of moving toward the target based on the target affordance graph and the current posture of the robot, and obtain a first motion trajectory;
[0120] Step 420: generating motion instructions for each joint of the robot to move according to the first motion trajectory;
[0121] Step 430: According to the joint motion instructions, the joint machines of the robot are driven to work so as to move according to the first motion trajectory.
[0122] In some embodiments of the present application, the imitation learning model is trained in advance. First, a large amount of teaching data is collected. The teaching data uses the target availability graph of the grasping target previously generated to guide trajectory planning, including the target availability graph of the grasping target and the joint state changes of the robot movement. Optionally, in an embodiment of the present application, the teaching trajectory of the imitation learning model focuses on the trajectory from the starting position to the area near the target in the target availability graph. Then, the imitation learning model to be trained is trained based on a large amount of teaching data to obtain a trained imitation learning model. Based on a large number of teaching trajectories, the imitation learning model can learn that the grasping task is to move from the starting position of the robot to the vicinity of the object represented by the first color in the target availability graph.
[0123] After training, the imitation learning model will directly plan the robot's motion trajectory close to the grasping target based on the input target affordance graph and the robot's current position, such as the robot's joint status. Then, by calling the joint control interface, the robot can complete the task of moving to the vicinity of the target.
[0124] In an embodiment of the present application, when training the imitation learning model, in the target availability diagram of the grasping target, the grasping target is represented by the first color, such as the first color is a highlight color, which weakens the dependence on the appearance of a specific object, so that the imitation learning model can only focus on the highlighted target without understanding which specific object the grasping target is, thereby achieving generalization of the motion trajectory.
[0125] like Figure 5 As shown, the input of the imitation learning model is the target affordance map of the grasping target and the current position of the robot, and the output is the joint motion instruction, that is, the aforementioned steps 400 to 420 can be implemented by the imitation learning model. Then, step 430 is executed to drive the joint machine of the robot to work according to each joint motion instruction, so that the robot can move according to the first motion trajectory, thereby approaching the target in the first image.
[0126] Step 120: After moving to the vicinity of the target, obtain a second image containing the target, and based on the second image, generate an operational availability map of the grasping area containing the target.
[0127] In some embodiments of the present application, when obtaining the second image containing the target in step 120, it can be obtained through the camera on the robot's robotic arm or through the robot's head camera. Since the robot is close to the target, the target occupying a larger area in the second image is recorded as the target to be captured.
[0128] In some embodiments of the present application, when generating an operation affordance map of a grab area including a target based on the second image in step 120, refer to Figure 6As shown, this can be achieved by executing the following steps:
[0129] Step 600: Perform target recognition on the second image, and determine the grabbing area of the target in the second image based on the appearance information of the target in the recognized second image; or, input the second grabbing instruction and the second image into the visual language model, and use the grabbing position information of the target included in the second grabbing instruction to guide the visual language model to mark the grabbing area of the target in the second image.
[0130] In some embodiments of the present application, when executing step 600, the target is recognized in the second image, and then, an image generation algorithm, such as Generative Adversarial Networks (GAN), a stable diffusion model, etc., can be used to determine the grasping area of the target in the second image based on the appearance information of the target in the recognized second image, wherein the image generation algorithm can be trained in advance to achieve the above functions. During the training process, the input of the image generation algorithm is an image containing the grasping target, and the output is an operation affordance map containing the corresponding grasping position, so that while maintaining high efficiency, the accuracy and flexibility of grasping are also ensured.
[0131] In some embodiments of the present application, referring to the description of step 200, when executing step 600, a visual language model, such as a multimodal large model, can be used to have a strong common sense understanding ability. Based on the second image, the grasping position information of the target included in the second grasping instruction can be used to guide the visual language model to mark the grasping area of the target in the second image.
[0132] It should be noted that there are the following differences between the operation affordance graph and the target affordance graph: the target affordance graph for grasping the target represents the entire target and has an independent outline, while the operation affordance graph indicates which areas of the target can be grasped, and these areas do not have fixed outlines.
[0133] Step 610: In the second image, fill the capture area with a second color to obtain an operational availability map of the target.
[0134] In some embodiments of the present application, after executing step 600 to obtain the grab area of the target, step 610 is executed to fill the grab area with a second color in the second image, thereby obtaining an operational availability map of the target.
[0135] For example, assuming that the obtained second image contains a square, the operation affordance graph of the square can be obtained by any of the above methods, such as Fig. 7AAs shown, the grab area of the block is filled with the second color; and assuming that the second image obtained contains a medicine bottle, the operation affordance diagram of the medicine can be obtained by any of the above methods, such as Figure 7B As shown, the grabbing area of the medicine is filled with the second color.
[0136] In this way, the specific outline of the target's grasping area is weakened, so that when the robot plans the grasping action, it only plans for the grasping area in the operation availability graph filled with the second color, so that the robot can still maintain efficient grasping effects when facing new objects and scenes.
[0137] Step 130: Based on the operation affordance graph, plan a second motion trajectory for the robot to grasp the target, and complete the grasping of the target according to the second motion trajectory.
[0138] In some embodiments of the present application, when executing step 130, refer to Figure 8 As shown, this can be achieved by executing the following steps:
[0139] Step 1301: Input the operation availability graph and the current state of the robotic arm into the reinforcement learning model, wherein the reinforcement learning model is obtained by training the reinforcement learning model to be trained for at least one round based on multiple samples and reward and punishment criteria, and each sample includes the operation availability graph of the sample target and the initial state of the robotic arm.
[0140] The reinforcement learning model is trained in a simulation environment. First, an interactive environment is designed, which may include a variety of objects that need to be grasped, and each object is marked with its grasping position. The input of the reinforcement learning model is the operation affordance graph of the grasping target, and its reward and penalty criteria (i.e., reward function) for grasping the object part include some or all of the following contents:
[0141] 1. Reward the robot arm if the distance of the gripper of the robot arm close to the grasping area in the operation affordance graph of the sample target is less than the first threshold;
[0142] 2. Reward the robot arm for picking up the sample target within a time period less than the second threshold;
[0143] 3. Reward the sample target that is picked up when its distance from the ground is greater than the third threshold;
[0144] 4. The reward is given when the distance difference between the center of the sample target picked up and the center of the gripper of the robot arm is less than a fourth threshold;
[0145] 5. The shutdown parameter value and joint limit value of the penalty robot arm are less than the fifth threshold;
[0146] 6. Punish the robot arm’s joint movement range that is greater than the sixth threshold.
[0147] When training the reinforcement learning model, any initial grasping starting position is considered, so that the reinforcement learning model can grasp the target at any starting position. By training the reinforcement learning model, the reinforcement learning model can optimize the grasping strategy according to the reward maximization route based on the designed reward and penalty standard, that is, the reward function, in the simulation environment, and after multiple interactions with the environment, learn the operation affordance graph based on the grasping target, complete the strategy of grasping any target, and solidify the optimized grasping strategy in the model, so that in the model reasoning stage, the robot arm is guided to perform motion planning based on the solidified grasping strategies to grasp the target. The processing flow of the model reasoning stage is detailed in the subsequent step 1302.
[0148] Step 1302: using the reinforcement learning model, perform the following operations for each pending state of the robot arm until the target is grasped, wherein the first pending state is the current state, and the second motion trajectory includes multiple motion trajectories:
[0149] Step 13021: using the reinforcement learning model to select a grasping strategy that matches the state to be processed and the operation affordance graph from various grasping strategies learned by the reinforcement learning model through training according to the operation affordance graph and the state to be processed, wherein each grasping strategy includes a motion trajectory of moving from one state to another state in the process of grasping the target;
[0150] Step 13022: Control the robot arm to move according to the motion trajectory included in the filtered grasping strategy. When the robot arm completes the motion trajectory, determine the intermediate state of the robot arm and use the intermediate state to update the state to be processed.
[0151] In some embodiments of the present application, in the model reasoning stage, the model is based on the current state of the robotic arm and the operation availability graph. During the entire process of grasping the target, the robotic arm moves according to a preset control cycle. According to the operation availability graph and the current state of the robotic arm (i.e., the first state to be processed), a grasping strategy that is similar to the scenario represented by the operation availability graph and matches the current state is screened out from the various grasping strategies trained and solidified in the model; then, the robotic arm is controlled to move according to the motion trajectory included in the screened grasping strategy.
[0152] In some embodiments, if the control accuracy of the robotic arm is high, at this time, by moving according to the motion trajectory included in the screened grasping strategy, it can reach another state in the motion trajectory. However, it is found in actual applications that there are often certain deviations due to various factors. Therefore, it is necessary to obtain images of the environment through cameras and various sensors deployed on the robot. By recognizing the image, the environmental information can be obtained, and then the real-time state of the robotic arm, such as the aforementioned intermediate state, is determined based on the environmental information, and the intermediate state is used to update the pending state. Subsequently, the motion trajectory is determined, moved, and updated for each pending state in the above manner until the target is grasped.
[0153] In summary, in some embodiments of the present application, the imitation learning model and the reinforcement learning model are used in series to jointly complete the target grasping task, wherein the imitation learning model can directly perform trajectory planning based on the image, without relying on the complex reinforcement learning training environment, so that it can quickly adapt to and plan effective movement trajectories when facing diverse backgrounds, which perfectly makes up for the defects that the objects and scenes in the real world are ever-changing and difficult to cover one by one in the training environment; the reinforcement learning model is mainly used to process the grasping action based on the operation affordance graph, and by training the reinforcement learning model, more general and flexible grasping skills can be learned, so that when facing new objects and scenes, efficient grasping effects can still be maintained. Therefore, in the embodiments of the present application, the imitation learning model and the reinforcement learning model are combined, and the advantages of the imitation learning model training time period and the strong generalization ability of the reinforcement learning model are combined, thereby realizing the grasping of general objects, so that the robot's grasping has both rapid adaptability and excellent generalization performance.
[0154] Based on the same inventive concept, refer to Fig. 9 As shown, in an embodiment of the present application, a target grasping device of a robot is provided, comprising:
[0155] A first acquisition module 910 is used to acquire a first image containing a target to be captured, and generate a target availability map of an area containing the target based on the first image;
[0156] A first planning module 920, configured to plan a first motion trajectory of the robot approaching the target based on the target affordance graph, and move according to the first motion trajectory;
[0157] A second acquisition module 930 is used to acquire a second image containing the target after moving to the vicinity of the target, and generate an operation affordance map of the grasping area containing the target based on the second image;
[0158] The second planning module 940 is used to plan a second motion trajectory for the robot to grasp the target based on the operation affordance graph, and complete the grasping of the target according to the second motion trajectory.
[0159] In a possible implementation, the first acquisition module 910 is specifically configured to:
[0160] Based on the description information of the target included in the first grabbing instruction, perform target recognition on the first image to identify the area of the target in the first image; or,
[0161] Inputting the first grabbing instruction and the first image into a visual language model, and using the description information of the target included in the first grabbing instruction to guide the visual language model to perform target recognition in the first image, and recognize the region of the target in the first image;
[0162] In the first image, the area is filled with a first color to obtain the target availability map.
[0163] In a possible implementation, the first planning module 920 is specifically configured to:
[0164] Inputting the target affordance graph and the current posture of the robot into an imitation learning model, wherein the imitation learning model is obtained by performing at least one round of training on the imitation learning model to be trained based on a plurality of teaching data, each teaching data comprising a target affordance graph based on a sample target and a motion trajectory of the posture planning of the robot;
[0165] Using the imitation learning model to plan the posture of the robot in the process of moving toward the target based on the target affordance graph and the current posture of the robot, so as to obtain the first motion trajectory;
[0166] generating, according to the first motion trajectory, motion instructions for each joint of the robot to move;
[0167] According to the joint motion instructions, the joint machines of the robot are driven to work so as to move according to the first motion trajectory.
[0168] In a possible implementation, the second acquisition module 930 is specifically configured to:
[0169] Performing target recognition on the second image, and determining a capture area of the target in the second image based on the recognized appearance information of the target in the second image; or,
[0170] Inputting the second grabbing instruction and the second image into a visual language model, and using the grabbing position information of the target included in the second grabbing instruction, guiding the visual language model to mark a grabbing area of the target in the second image;
[0171] In the second image, the grab area is filled with a second color to obtain an operation affordance map of the target.
[0172] In a possible implementation, the second planning module 940 is specifically configured to:
[0173] Inputting the operation affordance graph and the current state of the robotic arm into a reinforcement learning model, wherein the reinforcement learning model is obtained by training the reinforcement learning model to be trained for at least one round based on multiple samples and reward and penalty criteria, and each sample includes the operation affordance graph of the sample target and the initial state of the robotic arm;
[0174] Using the reinforcement learning model, the following operations are performed for each pending state of the robotic arm until the target is grasped, wherein the first pending state is the current state, and the second motion trajectory includes multiple sub-motion trajectories:
[0175] Using the reinforcement learning model, according to the operation affordance graph and the state to be processed, from various grasping strategies learned by the reinforcement learning model through training, a grasping strategy matching the state to be processed and the operation affordance graph is screened out, wherein each grasping strategy includes a motion trajectory of moving from one state to another state in the process of grasping a target;
[0176] According to the motion trajectory included in the screened grasping strategy, the robot arm is controlled to move, and when the robot arm completes the motion trajectory, the intermediate state of the robot arm is determined, and the state to be processed is updated with the intermediate state.
[0177] In a possible implementation, the reward and punishment criteria include part or all of the following:
[0178] The rewarding robot arm is rewarded when the distance of the gripper of the robot arm close to the grasping area in the operation affordance graph of the sample target is less than a first threshold;
[0179] The rewarding robot arm is rewarded for picking up the sample target within a time period shorter than a second threshold;
[0180] The distance of the sample target picked up by the reward clamp is greater than the third threshold;
[0181] Rewarding that the distance difference between the center of the sample target picked up and the center of the gripper of the robot arm is less than a fourth threshold;
[0182] The shutdown parameter value and joint limit value of the penalty robot arm are less than the fifth threshold;
[0183] The joint movement amplitude of the robot arm is punished to be greater than the sixth threshold.
[0184] Based on the same inventive concept, refer to Fig.10 As shown, an embodiment of the present application provides a robot, comprising:
[0185] A camera 101 is deployed on the robot and is used to acquire images;
[0186] Memory 102, used to store computer programs or instructions;
[0187] The processor 103 is used to execute the computer program or instruction in the memory 102 so that any one of the methods in the above embodiments is executed.
[0188] The processor 103 may include one or more central processing units (CPUs) or digital processing units, etc., for executing computer programs or instructions in the memory 102, so that any of the methods in the above embodiments is executed;
[0189] It should be noted that the specific connection medium between the memory 102 and the processor 103 is not limited in the embodiment of the present application. Fig.10 In the embodiment, the memory 102 and the processor 103 are connected via a bus 104, and the connection between other components is only for illustrative purposes and is not intended to be limiting. The bus 104 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0190] Based on the same inventive concept, the embodiment of the present application provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor, the processor is enabled to execute any one of the methods in the above embodiments. Since the principle of solving the problem by the above computer-readable storage medium is similar to a target grasping method of a robot, the implementation of the above computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.
[0191] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0192] The present application is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.
[0193] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0194] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0195] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A robot target grasping method, characterized in that: include: Acquire a first image containing a target to be captured, and generate a target availability map of a region containing the target based on the first image; Based on the target affordance graph, planning a first motion trajectory of the robot approaching the target, and moving according to the first motion trajectory; After moving to the vicinity of the target, acquiring a second image containing the target, and generating an operational affordance map of a grasping area containing the target based on the second image; Based on the operation affordance graph, a second motion trajectory of the robot for grasping the target is planned, and grasping the target is completed according to the second motion trajectory.
2. The method according to claim 1, characterized in that The step of generating, based on the first image, a target availability map of an area including the target, comprises: Based on the description information of the target included in the first grabbing instruction, perform target recognition on the first image to identify the area of the target in the first image; or, Inputting the first grabbing instruction and the first image into a visual language model, and using the description information of the target included in the first grabbing instruction to guide the visual language model to perform target recognition in the first image, and recognize the region of the target in the first image; In the first image, the area is filled with a first color to obtain the target availability map.
3. The method according to claim 1, characterized in that The step of planning a first motion trajectory of the robot approaching the target based on the target affordance graph and moving according to the first motion trajectory includes: Inputting the target affordance graph and the current posture of the robot into an imitation learning model, wherein the imitation learning model is obtained by performing at least one round of training on the imitation learning model to be trained based on a plurality of teaching data, each teaching data comprising a target affordance graph based on a sample target and a motion trajectory of the posture planning of the robot; Using the imitation learning model to plan the posture of the robot in the process of moving toward the target based on the target affordance graph and the current posture of the robot, so as to obtain the first motion trajectory; generating, according to the first motion trajectory, motion instructions for each joint of the robot to move; According to the joint motion instructions, the joint machines of the robot are driven to work so as to move according to the first motion trajectory.
4. The method according to claim 1, characterized in that The step of generating an operation affordance map of a grasping area including the target based on the second image comprises: Performing target recognition on the second image, and determining a capture area of the target in the second image based on the recognized appearance information of the target in the second image; or, Inputting the second grabbing instruction and the second image into a visual language model, and using the grabbing position information of the target included in the second grabbing instruction, guiding the visual language model to mark a grabbing area of the target in the second image; In the second image, the grab area is filled with a second color to obtain an operation affordance map of the target.
5. The method according to any one of claims 1 to 4, characterized in that: The planning, based on the operation affordance graph, of a second motion trajectory for the robot to grasp the target, and completing the grasping of the target according to the second motion trajectory, includes: Inputting the operation affordance graph and the current state of the robotic arm into a reinforcement learning model, wherein the reinforcement learning model is obtained by training the reinforcement learning model to be trained for at least one round based on multiple samples and reward and penalty criteria, and each sample includes the operation affordance graph of the sample target and the initial state of the robotic arm; Using the reinforcement learning model, the following operations are performed for each pending state of the robotic arm until the target is grasped, wherein the first pending state is the current state, and the second motion trajectory includes multiple sub-motion trajectories: Using the reinforcement learning model, according to the operation affordance graph and the state to be processed, from various grasping strategies learned by the reinforcement learning model through training, a grasping strategy matching the state to be processed and the operation affordance graph is screened out, wherein each grasping strategy includes a motion trajectory of moving from one state to another state in the process of grasping a target; According to the motion trajectory included in the screened grasping strategy, the robot arm is controlled to move, and when the robot arm completes the motion trajectory, the intermediate state of the robot arm is determined, and the state to be processed is updated with the intermediate state.
6. The method according to claim 5, characterized in that The reward and punishment standards include some or all of the following: The rewarding robot arm is rewarded when the distance of the gripper of the robot arm close to the grasping area in the operation affordance graph of the sample target is less than a first threshold; The rewarding robot arm is rewarded for picking up the sample target within a time period shorter than a second threshold; The distance of the sample target picked up by the reward clamp is greater than the third threshold; Rewarding that the distance difference between the center of the sample target picked up and the center of the gripper of the robot arm is less than a fourth threshold; The shutdown parameter value and joint limit value of the penalty robot arm are less than the fifth threshold; The joint movement amplitude of the robot arm is penalized to be greater than the sixth threshold.
7. A robot target grasping device, characterized in that: include: A first acquisition module, configured to acquire a first image containing a target to be captured, and generate a target availability map of a region containing the target based on the first image; A first planning module, configured to plan a first motion trajectory of the robot approaching the target based on the target affordance graph, and move the robot along the first motion trajectory; A second acquisition module, configured to acquire a second image containing the target after moving to the vicinity of the target, and generate an operation affordance map of a grasping area containing the target based on the second image; The second planning module is used to plan a second motion trajectory for the robot to grasp the target based on the operation affordance graph, and complete the grasping of the target according to the second motion trajectory.
8. The device according to claim 7, characterized in that The first acquisition module is specifically used for: Based on the description information of the target included in the first grabbing instruction, perform target recognition on the first image to identify the area of the target in the first image; or, Inputting the first grabbing instruction and the first image into a visual language model, and using the description information of the target included in the first grabbing instruction to guide the visual language model to perform target recognition in the first image, and recognize the region of the target in the first image; In the first image, the area is filled with a first color to obtain the target availability map.
9. A robot, characterized in that: include: A camera, deployed on the robot, for acquiring images; Memory, used to store computer programs or instructions; A processor is used to execute the computer program or instructions in the memory so that the method according to any one of claims 1 to 6 is executed.
10. A computer-readable non-volatile storage medium, characterized in that: When the instructions in the storage medium are executed by a processor, the processor is enabled to execute the method according to any one of claims 1 to 6.