A Flexible Object Manipulation Method and Device Based on Anticipatory Operation Representation

Abstract

The present invention discloses a flexible object operation method and device based on anticipatory operation representation. The method includes: based on the depth image of the current state of the flexible object, using the grasping affordance model to predict the grasping affordance value of each pixel point on the entire depth image and screening out the current grasping point; based on the depth image of the current state and the current grasping point, using the placing affordance model to predict the placing affordance value of each pixel point on the entire depth image and screening out the current placing point; based on the current grasping point and the current placing point, controlling the robotic arm to perform the current grasping and placing operations on the flexible object, and obtaining the state of the flexible object after the current grasping and placing operations as the current state for the next grasping and placing operations; repeating the above steps until the operation task is completed. The method of the present invention is more natural in representation, reduces the learning complexity by injecting prior knowledge, and improves the success rate of flexible object operation.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and particularly to a flexible object operation method and device based on anticipatory operation representation. Background Art

[0002] With the continuous progress of artificial intelligence technology, flexible object operation (such as the operation of objects like clothes and ropes) is an indispensable ability for future robots. However, there is still much room for improvement in the research on flexible object operation in the current field of robotics. This is mainly due to the following two difficulties in flexible object operation tasks: First, the states of flexible objects are almost infinite, difficult to represent, and have complex dynamic models. Second, different from rigid bodies and articulated objects, flexible object operation tasks usually require multiple steps of operation. For example, a piece of clothing in a wrinkled state needs multiple steps of operation to be fully unfolded. The first difficulty makes it very difficult to select each step of operation, and random grasping and placing strategies are almost impossible to perform effective operations. The second difficulty further introduces the complexity of long-term planning, making greedy strategies prone to falling into local optimal solutions. Although the operations of greedy strategies can achieve good effects in one step, increasing the coverage area of the fabric, they also cause the corners of the fabric to be covered by itself, making it difficult to smoothly complete the entire task in subsequent operations.

[0003] There have also been many explorations in the field of robotics for flexible object operation, but there is still a huge room for improvement in the research on complex and long-term operation tasks. Many previous works have used reinforcement learning to cope with the complexity and long-term nature of flexible object operation. However, due to the characteristics of reinforcement learning, this method has always faced problems such as high learning complexity and low exploration efficiency. Reinforcement learning models often need to collect actions and corresponding results in various situations. Due to the characteristics of flexible objects, the exploration space for this collection will be very large, and it contains a lot of data with little information. For example, the actions and feedback obtained by random strategies actually do not reflect much operation information. At the same time, a large amount of unstructured data also brings high learning complexity, and the multi-step calculation of rewards in reinforcement learning also makes training more unstable. Therefore, the success rate of previous methods in facing very challenging tasks such as unfolding fabric and operating ropes is very low. Summary of the Invention

[0004] The purpose of the present invention is to provide a flexible object operation method and device based on anticipatory operation representation to solve the problem that the success rate of existing robots in performing flexible object operation tasks is very low.

[0005] The first aspect of the present invention provides a flexible object operation method based on anticipatory operation representation, including: S1, based on the depth image included in the current state of the flexible object, using the grasping affordance model to predict the grasping affordance value of each pixel point on the entire depth image, and screening out the pixel point corresponding to the largest value among the grasping affordance values as the current grasping point; S2, based on the depth image included in the current state and the current grasping point, using the placing affordance model to predict the placing affordance value of each pixel point on the entire depth image; screening out the pixel point corresponding to the largest value among the placing affordance values as the current placing point; S3, based on the current grasping point and the current placing point, controlling the robotic arm to perform the current grasping and placing operations on the flexible object, and obtaining the state of the flexible object after the current grasping and placing operations as the current state for the next grasping and placing operations; S4, repeating steps S1-S3 until the flexible object operation task is completed.

[0006] Further, the step of using the grasping affordance model to predict the grasping affordance value of each pixel point on the entire depth image based on the depth image included in the current state of the flexible object includes: using the depth image included in the given state of the flexible object as the input of the grasping affordance model, and the output of the grasping affordance model is a set of grasping affordance values of each pixel point on the entire depth image. The higher or lower the grasping affordance value corresponding to each pixel point represents the value of each pixel point as a grasping point for completing the flexible object operation task.

[0007] Further, the step of using the placing affordance model to predict the placing affordance value of each pixel point on the entire depth image based on the depth image included in the current state and the current grasping point includes: using the depth image included in the given current state of the flexible object and the pixel position of the current grasping point as the input of the placing affordance model, and the output of the placing affordance model is a set of placing affordance values of each pixel point in the entire depth image. The higher or lower the placing affordance value corresponding to each pixel point represents the value of placing the current grasping point at this pixel point for completing the task.

[0008] Further, the grasping affordance model and the placing affordance model adopt inverse learning during training, that is, learning from the target state of the flexible object operation task to the initial state.

[0009] Further, learning from the target state of the flexible object operation task to the initial state includes: decomposing the operation task into multiple stages. After one-step grasping and placing operations in the first stage of the multiple stages, the target state of the operation task is achieved. After one-step grasping and placing operations in the second stage, the first stage is achieved, and so on, until the degree of chaos of the state of the flexible object in the last stage reaches the initial state of the task.

[0010] Furthermore, both the grasping affordance model and the placing affordance model adopt a residual neural network as the encoder.

[0011] Furthermore, the number of layers of the residual neural network is 43 layers.

[0012] The second aspect of the present invention provides a flexible object manipulation device based on anticipatory operation representation, including:

[0013] A current grasping point determination module, configured to, based on the depth-containing image of the current state of the flexible object, use the grasping affordance model to predict the grasping affordance value of each pixel point on the entire depth-containing image, and screen out the pixel point corresponding to the largest value among the grasping affordance values as the current grasping point;

[0014] A current placing point determination module, configured to, based on the depth-containing image of the current state and the current grasping point, use the placing affordance model to predict the placing affordance value of each pixel point on the entire depth-containing image; screen out the pixel point corresponding to the largest value among the placing affordance values as the current placing point;

[0015] A grasping and placing operation module, configured to, based on the current grasping point and the current placing point, control the robotic arm to perform the current grasping and placing operation on the flexible object, and obtain the state of the flexible object after the current grasping and placing operation as the current state for the next grasping and placing operation;

[0016] A repeated execution module, which repeatedly executes the operations of the current grasping point determination module, the current placing point determination module, and the grasping and placing operation module until the flexible object manipulation task is completed.

[0017] The third aspect of the present invention provides an electronic device, including a processor and a memory, where the memory stores multiple instructions, and the processor is configured to read the instructions and execute the flexible object manipulation method based on anticipatory operation representation according to the first aspect of the present invention.

[0018] The fourth aspect of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores multiple instructions, and the multiple instructions can be read and executed by a processor to perform the flexible object manipulation method based on anticipatory operation representation as described in the first aspect of the present invention.

[0019] The beneficial effects of the present invention are:

[0020] 1. The flexible object manipulation method based on anticipatory operation representation provided by the present invention greatly improves the success rate of long-range flexible object manipulation. Compared with the existing strong learning methods and dynamic model methods, the representation of the present invention is more natural, and the learning complexity is reduced by injecting prior knowledge, so a more mature strategy can be obtained;

[0021] 2. The flexible object operation method based on anticipatory operation representation provided by the present invention improves the utilization efficiency of data. The present invention uses reverse staged learning, and only needs to learn the knowledge of one stage in each step, without collecting an entire operation trajectory, which improves the information content of the samples and reduces the time required for data collection.

[0022] 3. The flexible object operation method based on anticipatory operation representation provided by the present invention has the potential to be applied to new tasks. Compared with reinforcement learning that requires formulating rewards, the method provided by the present invention can be transferred to other tasks, such as plasticine shaping and folding clothes, without manually formulating guiding rewards. Moreover, the generalization of reinforcement learning has always been a major problem. In contrast, the visual representation of the present invention is more likely to generalize. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only one embodiment of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a schematic flowchart of the flexible object operation method based on anticipatory operation representation according to the present invention;

[0025] Figure 2 It is a schematic structural diagram of the flexible object operation device based on anticipatory operation representation according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The technical solutions in the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.

[0027] The technical solutions of the present application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0028] Embodiment 1

[0029] As Figure 1 shown, this embodiment provides a flexible object operation method based on anticipatory operation representation, including the following steps:

[0030] S1. Based on the depth image of the current state of the flexible object, use the grasping affordance model to predict the grasping affordance values of each pixel point on the entire depth image, and select the pixel point corresponding to the maximum value among the grasping affordance values as the current grasping point;

[0031] Specifically, take the depth image O ∈ R that characterizes the given state of the flexible object W×H×C as the input of the grasping affordance model. The depth image includes both the light intensity information of the ordinary image and its depth information. W and H respectively represent the width and height of the depth image, and C represents the number of channels of the image, C = 4; the output of the grasping affordance model is the set A O，Pick ∈ R W×H of the grasping affordance values of each pixel point on the entire depth image. The level of the grasping affordance value corresponding to each pixel point represents the value of each pixel point as a grasping point for completing the flexible object operation task. Select the pixel point corresponding to the maximum value among the grasping affordance values as the current grasping point.

[0032] S2. Based on the depth image of the current state and the current grasping point, use the placing affordance model to predict the placing affordance values of each pixel point on the entire depth image; select the pixel point corresponding to the maximum value among the placing affordance values as the current placing point;

[0033] Specifically, take the depth image O ∈ R that characterizes the given current state of the flexible object W×H×C and the current grasping point as the input of the placing affordance model. W and H respectively represent the width and height of the depth image, and C represents the number of channels of the image, C = 4; the output of the placing affordance model is the set of the placing affordance values of each pixel point in the entire depth image. The level of the placing affordance value corresponding to each pixel point represents the value of placing the current grasping point at this pixel point for completing the task; select the pixel point corresponding to the maximum value among the placing affordance values as the current placing point.

[0034] S3. Based on the current grasping point and the current placing point, control the robotic arm to perform the current grasping and placing operations on the flexible object, and obtain the state of the flexible object after the current grasping and placing operations as the current state for the next grasping and placing operations;

[0035] S4. Repeat steps S1 - S3 until the flexible object operation task is completed.

[0036] It should be noted that the grasping affordance model and the placing affordance model adopted in the present invention use reverse learning during training, that is, learning from the target state of the flexible object operation task to the initial state. When the grasping affordance model and the placing affordance model are trained using the flexible object operation task, the entire operation task is first decomposed into multiple stages. The first stage among the multiple stages achieves the target state of the entire task after one-step grasping and placing operations. The second stage achieves the first stage after one-step grasping and placing operations, and so on, until the degree of state chaos of the flexible object in the last stage reaches the initial state of the task. For example, in the cloth unfolding task, the unfolding area of the cloth reaches 90% of the fully flattened area. At the same time, the grasping and placing operation data corresponding to each stage and the depth image containing the state of the flexible object after the operation are obtained, and the grasping affordance model and the placing affordance model are trained using the grasping and placing operation data corresponding to each stage and the depth image containing the state of the flexible object after the operation.

[0037] The specific design and training of the placing affordance model and the grasping affordance model are as follows:

[0038] I. Placing Affordance Model

[0039] The placing affordance model is used to predict the placing affordance of a flexible object. Based on the depth image containing the current state of the given flexible object and the current grasping point, it predicts the value of all placing points on the entire two-dimensional plane for completing the task.

[0040] The placing affordance model uses a 43-layer residual neural network as the encoder to extract the local features and global features of each pixel point. Among them, the encoder presents a hourglass shape. First, 4 times of max pooling are performed, and then 4 times of upsampling are performed to obtain the local features corresponding to each pixel. The dimension of this local feature is 256, and the feature of the middle bottleneck is used as the global feature with a dimension of 512. Then, the local feature of the grasping point p pick =(i, j) and the local feature of the placing point p place =(u, v), as well as the global feature are concatenated together, and the placing affordance value of this point is obtained through a three-layer multi-layer perceptron. Among them, (i, j) and (u, v) represent the position coordinates of any pixel point on the depth image.

[0041] The training data of the placing affordance model includes the depth image O∈R of the state s of the flexible object before the operation W ×H×C , the current grasping point p pick =(i, j) and the current placing point p place=(u, v) and the state s' of the flexible object after the operation. During the training in the first stage, the placement affordance model is made to predict and fit the evaluation result after this placement operation, that is, at the current placement point p place = the predicted value of (u, v) where R(s') represents the quantitative evaluation of the state s'. For example, in the fabric unfolding task, it is the covered area, and in the rope operation, it is the distance from the target state. In the training after the first stage, not only the quantitative evaluation R(s') returned by the simulator is relied on, but also the value estimation of the state s' of the flexible object after the operation is added, so as to consider the influence of the current action on subsequent operations, that is, the placement point p place = the predicted value of (u, v) where α and β respectively represent the weights of the current quantitative evaluation and the future value estimation, and V(s') represents the value estimation of the state s' of the flexible object, which will be introduced in the grasping affordance model here.

[0042] II. Grasping Affordance Model

[0043] The grasping affordance model realizes the prediction of the grasping affordance of the flexible object through a neural network, and for the observation of a given object state, predicts the value of all grasping points on the entire two-dimensional plane for completing the task.

[0044] The grasping affordance model uses a 43-layer residual neural network as the encoder to extract the local features of each point. It should be noted that the encoder presents an hourglass shape, first performs 4 times of max-pooling, and then performs 4 times of upsampling to obtain the local features corresponding to each pixel. The dimension of this local feature is 256. The features of the grasping point pass through a three-layer multi-layer perceptron to obtain the grasping affordance of this point.

[0045] The training data of the grasping affordance model includes the depth image O ∈ R representing the state S of the flexible object before the operation W×H×C and the current grasping point P pick =(i, j). Through training, the grasping affordance model is made to predict the best result that can be obtained after grasping this grasping point P pick =(i, j). Here, the best result refers to the predicted value of the placement point p place =(u, v) where is the grasping affordance module for the grasping point P pick =(i, j) placed at the placement point p place= Predictive value of affordance for (u, v); Since a grasped point can always be placed at the placement point with the maximum value, the grasping affordance of this grasped point should be based on the maximum value of the placement affordance of this point. Additionally, to estimate the value of a state, the best state that a state can reach in the future should be estimated. This application uses the maximum value of the grasping affordance of a state to estimate the value of this state, that is, value estimation, where value estimation

[0046] In summary, when the grasping affordance model and the placement affordance model provided by the present invention are trained, inverse learning is adopted. Inverse learning means reversing the execution process of the task and learning from the target state of the task to the initial state. First, learn the operation strategy of the last step, and then learn the previous step until the initial state. This learning strategy greatly reduces the scope that needs to be explored and at the same time transmits value estimation, efficiently and stably obtaining the required affordance representation.

[0047] Embodiment 2

[0048] As Figure 2 shown, this embodiment provides a flexible object manipulation device based on anticipatory operation representation, including:

[0049] A current grasping point determination module, configured to use the grasping affordance model to predict the grasping affordance values of each pixel point on the entire depth image based on the depth image included in the current state of the flexible object, and screen out the pixel point corresponding to the maximum value among the grasping affordance values as the current grasping point;

[0050] A current placement point determination module, configured to use the placement affordance model to predict the placement affordance values of each pixel point on the entire depth image based on the depth image included in the current state and the current grasping point; screen out the pixel point corresponding to the maximum value among the placement affordance values as the current placement point;

[0051] A grasping and placement operation module, configured to control the robotic arm to perform the current grasping and placement operations on the flexible object based on the current grasping point and the current placement point, and obtain the state of the flexible object after the current grasping and placement operations as the current state for the next grasping and placement operations;

[0052] A repeated execution module, which repeatedly executes the operations of the current grasping point determination module, the current placement point determination module, and the grasping and placement operation module until the flexible object manipulation task is completed.

[0053] This device can implement a flexible object manipulation method based on anticipatory operation representation provided in the above Embodiment 1. For the specific operation method, reference can be made to the description in Embodiment 1, which will not be elaborated here.

[0054] The present invention provides an electronic device, comprising a processor and a memory. The memory stores a plurality of instructions, and the processor is configured to read the instructions and execute any one of the methods in the foregoing Embodiment 1.

[0055] The present invention also provides a computer-readable storage medium storing a plurality of instructions, which can be read and executed by a processor to execute any one of the methods in the foregoing Embodiment 1.

[0056] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the present invention. Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A flexible object manipulation method based on anticipatory operation representation, characterized in that, Including the following steps: S1. Based on the depth image of the current state of the flexible object, use the grasping affordance model to predict the grasping affordance values of each pixel point on the entire depth image, and screen out the pixel point corresponding to the maximum value among the grasping affordance values as the current grasping point; S2. Based on the depth image of the current state and the current grasping point, use the placing affordance model to predict the placing affordance values of each pixel point on the entire depth image; screen out the pixel point corresponding to the maximum value among the placing affordance values as the current placing point; S3. Based on the current grasping point and the current placing point, control the robotic arm to perform the current grasping and placing operations on the flexible object, and obtain the state of the flexible object after the current grasping and placing operations as the current state for the next grasping and placing operations; S4. Repeat steps S1 - S3 until the flexible object operation task is completed; The training data of the placement affordance model includes the state of the flexible object before operation including depth images , the current grasping point and the current placement point as well as the state of the flexible object after operation ; The training data of the grasping affordance model includes depth images representing the state of the flexible object before the operation and the current grasping point ; ; Among them, and respectively represent the width and height of the depth image, represents the number of channels of the depth image, and represent the position coordinates of any pixel point on the depth image; When training, the grasping affordance model and the placing affordance model adopt inverse learning, that is, learning from the target state of the flexible object operation task to the initial state; Learning from the target state of the flexible object operation task to the initial state includes: decomposing the operation task into multiple stages. After one-step grasping and placing operations in the first stage of the multiple stages, the target state of the operation task is achieved. After one-step grasping and placing operations in the second stage, the first stage is achieved, and so on, until the degree of chaos of the state of the flexible object in the last stage reaches the initial state of the task.

2. The flexible object operation method based on anticipatory operation representation according to claim 1, wherein The step of using the grasping affordance model to predict the grasping affordance values of each pixel point on the entire depth image based on the depth image of the current state of the flexible object includes: using the depth image representing the given state of the flexible object as the input of the grasping affordance model. The output of the grasping affordance model is a set of grasping affordance values of each pixel point on the entire depth image. The level of the grasping affordance value corresponding to each pixel point represents the value of each pixel point as a grasping point for completing the flexible object operation task.

3. The flexible object operation method based on anticipatory operation representation according to claim 1, wherein, The step of using the placing affordance model to predict the placing affordance values of each pixel point on the entire depth image based on the depth image of the current state and the current grasping point includes: using the depth image representing the given current state of the flexible object and the pixel position of the current grasping point as the input of the placing affordance model. The output of the placing affordance model is a set of placing affordance values of each pixel point in the entire depth image. The level of the placing affordance value corresponding to each pixel point represents the value of placing the current grasping point at this pixel point for completing the task.

4. The flexible object operation method based on anticipatory operation representation according to claim 1, wherein Both the grasping affordance model and the placing affordance model adopt a residual neural network as the encoder.

5. The method for operating a flexible object based on anticipatory operation representation according to claim 4, wherein, The number of layers of the residual neural network is 43 layers.

6. A flexible object manipulation device based on anticipatory operation representation, characterized in that, Including: A current grasping point determination module, configured to use the grasping affordance model to predict the grasping affordance values of each pixel point on the entire depth image based on the depth image of the current state of the flexible object, and screen out the pixel point corresponding to the maximum value among the grasping affordance values as the current grasping point; The current placement point determination module is used to, based on the depth image in the current state and the current grasping point, utilize the placement affordance model to predict the placement affordance value of each pixel point on the entire depth image; and screen out the pixel point corresponding to the maximum value among the placement affordance values as the current placement point. The grasping and placement operation module is used to, based on the current grasping point and the current placement point, control the robotic arm to perform the current grasping and placement operation on the flexible object, and obtain the state of the flexible object after the current grasping and placement operation as the current state for the next grasping and placement operation. The repeated execution module repeatedly executes the operations of the current grasping point determination module, the current placement point determination module, and the grasping and placement operation module until the flexible object operation task is completed. The training data of the placement affordance model includes the state of the flexible object before the operation including the depth image , the current grasping point and the current placement point as well as the state of the flexible object after the operation ; The training data of the grasping affordance model includes depth images representing the state of the flexible object before the operation and the current grasping point ; ; Among them, and respectively represent the width and height of the depth image, represents the number of channels of the depth image, and represent the position coordinates of any pixel point on the depth image; When training, the grasping affordance model and the placement affordance model adopt inverse learning, that is, learning from the target state of the flexible object operation task to the initial state. Learning from the target state of the flexible object operation task to the initial state includes: decomposing the operation task into multiple stages. In the first stage of the multiple stages, the target state of the operation task is achieved after one-step grasping and placement operations. In the second stage, the first stage is achieved after one-step grasping and placement operations, and so on, until the degree of chaos of the state of the flexible object in the last stage reaches the initial state of the task.

7. An electronic device, characterized in that, It includes a processor and a memory. The memory stores multiple instructions. The processor is used to read the instructions and execute the flexible object operation method based on anticipatory operation representation according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores multiple instructions. The multiple instructions can be read and executed by the processor to perform the flexible object operation method based on anticipatory operation representation according to any one of claims 1-5.

Images