Method for representing the state of a deformable object and its inverse dynamic model

Abstract

The application discloses a state representation method of a deformable object, comprising the following steps: S1: acquiring an RGB image of the deformable object; S2: acquiring the contour of the deformable object from the RGB image; S3: obtaining the centroid of the contour by using the acquired contour, establishing a polar coordinate system with the centroid as a center point, and dividing a plurality of sector regions; and S4: calculating the average distance of the contour points in each sector region to the centroid as a one-dimensional feature in a multi-dimensional feature of the state of the deformable object. The state representation method of the deformable object makes the inverse dynamics model pay more attention to the contour of the deformable object, so that the robot can better complete the task of operating the deformable object; and the inverse dynamics model combines the convolutional neural network and the attention mechanism in parallel, can capture local and overall features in the feature map at the same time, and thus improves the prediction performance of the inverse dynamics model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot control, in particular to a state representation method of deformable objects and its inverse dynamics model. BACKGROUND

[0002] Deformable objects have very important applications in various fields of robot operation, including operation on human tissues in robot surgery, various tasks in home interaction such as making dumplings, rolling dough, making sushi, etc., and assembly of deformable objects in industry. Although it is very important, the research on robot manipulation of deformable objects is relatively less compared to rigid objects, mainly because of a common challenge in this field: the interaction between robot and deformable object is highly complex, usually nonlinear, and difficult to model, so the modeling of deformable objects is a problem to be solved.

[0003] Deformable objects are mainly divided into three categories: one-dimensional deformable objects represented by ropes, two-dimensional deformable objects represented by cloth, and three-dimensional deformable objects represented by dough and plasticine. The modeling method of deformable objects is mainly divided into two categories: one is a physics-based method, which simulates the forward dynamics of deformable objects, and the other is a data-based method, which can simulate both forward dynamics and inverse dynamics.

[0004] The data-based method is mainly divided into the following two categories: 1. Learning forward dynamics model, this method mainly trains an end-to-end neural network, the input of the network is the current state of the deformable object and the operation implemented on the deformable object, the output of the network is the next state of the deformable object, the training data is collected from a physical simulator or reality, the core difficulty of this method is the representation of the state of the deformable object and the design of the action space. For three-dimensional deformable objects with elasticity and plasticity, it is very popular at present to use GNN (graph neural network) to simulate the interaction between particles of deformable objects, GNN itself provides a lot of prior information and has an inherent advantage in simulating particle interaction, but this method is still in the exploratory stage, and the tasks that can be completed by using GNN are still very limited. 2. Learning inverse dynamics model, this method is similar to method 1, the difference is that the input of the network is the state of the previous and next two time steps, for example, the state of the previous and next two time steps of the deformable object represented by the RGB image, the output of the network is the intermediate execution action, and the training data is also collected from a physical simulator or reality. At present, this method has been applied to the operation of one-dimensional and two-dimensional deformable objects, but there are few applications for three-dimensional deformable objects. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art in the background that the state representation method and the deformable object model cannot be applied to three-dimensional deformable objects, so that the robot cannot operate the three-dimensional deformable objects.

[0006] To achieve the above object, the present application adopts the following technical solutions:

[0007] A state representation method of a deformable object, comprising the following steps:

[0008] S1: obtaining an RGB image of a deformable object;

[0009] S2: obtaining the contour of the deformable object from the RGB image;

[0010] S3: obtaining the centroid of the contour using the obtained contour, establishing a polar coordinate system with the centroid as the center point, and dividing a plurality of sector regions;

[0011] S4: calculating the average distance of the contour points in each sector region to the centroid as one-dimensional features in the multi-dimensional features of the state of the deformable object.

[0012] In some embodiments, the step S2 is specifically: converting the RGB image from the RGB space to the HSV space, obtaining the region of interest of the deformable object by setting the threshold values of H, S and V, converting it into a binary image, then performing morphological operation with a 5*5 convolution kernel, and finally using an edge finding algorithm to obtain the contour of the deformable object.

[0013] In some embodiments, the entire two-dimensional space is divided into 72 sector regions in step S3, each region is 5°, and in step S4, after the average distance is calculated, the average distance is discretized at an interval of 0.25 cm as the 72-dimensional features of the state of the deformable object.

[0014] In some embodiments, the shape of the deformable object satisfies the following condition: any ray starting from the centroid of the contour has only one intersection point with the contour.

[0015] The present application also provides an inverse dynamics model for the state representation method of a deformable object, comprising an encoding layer, a self-attention part, a CNN part and an output part, wherein:

[0016] The encoding layer is used to encode the states of the deformable object at different time points into a three-channel feature map;

[0017] The self-attention part and the CNN part are connected in parallel, and the input of both is the feature map obtained by the encoding layer, and the output of both is classification information;

[0018] The output part is used for combining the outputs of the self-attention part and the CNN part to obtain a classification result.

[0019] In some embodiments, the encoding layer completes the encoding of the input by converting the input state vector into a one-hot encoded matrix and then right multiplying a weight vector; wherein the input state vector is the state vector at the current time, the state vector at the next time, and the state vector difference between the previous and next times.

[0020] In some embodiments, the self-attention part is composed of 6 self-attention modules connected in series, and each self-attention module is composed of a multi-head attention layer, a fully connected layer, and a residual connection.

[0021] In some embodiments, the CNN part adopts the structure of RestNet, including an input layer, 4 intermediate convolutional blocks, and a fully connected layer, wherein the input layer includes a convolutional layer, a regularization, an activation function layer, a pooling layer, and a Dropout layer connected in sequence; each of the 4 intermediate convolutional blocks is composed of n sub-convolutional layers, and each sub-convolutional layer is composed of three convolutional layers.

[0022] In some embodiments, the 4 intermediate convolutional blocks are composed of a first convolutional block, a second convolutional block, a third convolutional block, and a fourth convolutional block connected in sequence, wherein the first convolutional block is composed of 3 sub-convolutional layers, the second convolutional block is composed of 4 sub-convolutional layers, the third convolutional block is composed of 6 sub-convolutional layers, and the fourth convolutional block is composed of 3 sub-convolutional layers.

[0023] In some embodiments, the output part is used for adding the outputs of the self-attention part and the CNN part, and then passing through a SoftMax layer to represent the classification probability of each category.

[0024] The state representation method of the deformable object in the embodiment 1 of the present application is relatively efficient, the inverse dynamics model pays attention to the contour of the deformable object, so that the operation task of the robot on the deformable object can be better completed, the inverse dynamics model combines the convolutional neural network and the attention mechanism, can capture the sequence in space between different sectors, and can capture the texture features on the state feature map, that is, capture the local and overall features in the feature map, thereby improving the prediction performance of the inverse dynamics model. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The state representation method flowchart of the deformable object in the embodiment 1 of the present application;

[0026] Figure 2A rolling task diagram in the embodiment 1 of the present application;

[0027] Figure 3 A motion space diagram in the embodiment 1 of the present application;

[0028] Figure 4 A dough state representation diagram in the embodiment 1 of the present application;

[0029] Figure 5 A network structure diagram of a reverse dynamics model in the embodiment 2 of the present application;

[0030] Figure 6 An operation task diagram in the PlasticineLab in the embodiment 2 of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application are described in detail below. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present application and its applications, and the embodiments in the present application and the features in the embodiments can be combined with each other without conflict

[0032] The terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0033] The change of a three-dimensional deformable object is very complex and highly nonlinear, and directly using an RGB image or an RGBD image to represent it will pose a great challenge to the subsequent network model training. Therefore, the specific embodiment of the present application provides a state representation method for a deformable object, as shown in Figure 1 S1: obtaining an RGB image of a deformable object; S2: obtaining a contour of the deformable object from the RGB image; S3: obtaining a centroid of the contour, establishing a polar coordinate system with the centroid as a center point, and dividing a plurality of sector regions; and S4: calculating an average distance of contour points in each sector region to the centroid as a one-dimensional feature in a multi-dimensional feature of the deformable object.

[0034] The present application is aimed at a three-dimensional deformable object with elasticity and plasticity. As long as the target is to control the shape of the deformable object, and the shape of the deformable object meets the condition that any ray from the centroid of the contour has only one intersection point with the contour, the state representation method of the present application can be used. The above state representation method is a polar coordinate state representation method, which can efficiently represent the change of the object shape.

[0035] In combination with the state representation method proposed above, the specific embodiment of the present application further designs a network structure that is easy to train and has good prediction performance to represent the inverse dynamics model of the three-dimensional deformable object, which includes an encoding layer, a self-attention part, a CNN part and an output part, wherein: the encoding layer is used to encode the states of the deformable object at different time points into a three-channel feature map; the self-attention part and the CNN part are connected in parallel, and the feature map obtained by the encoding layer is inputted into both of them, and the classification information is outputted; the output part is used to combine the outputs of the self-attention part and the CNN part to obtain the classification result. The above inverse dynamics model adds an encoding layer commonly used in the field of NLP (Natural Language Processing) at the input end of the network, and the main part of the framework adopts the parallel structure of RestNet (Residual Network) and self-attention part, so that the inverse dynamics model can better capture the overall and local features of the deformable object, improve the expression ability, and obtain a relatively ideal prediction result.

[0036] The field of deformable object operation is still in the exploratory stage, especially the three-dimensional deformable object with elasticity and plasticity. Due to the complexity of the problem, at present, some specific operation tasks are mainly explored. In the following embodiment 1 and embodiment 2, the deformable object is taken as dough, and the task is rolling. A state representation method of three-dimensional deformable object and an inverse dynamics model for the state representation method are proposed. The operation task of the following embodiment is: using a roller, a piece of dough is rolled from the initial shape to a specified shape (such as a fixed proportion of a rectangle, a circle).

[0037] Embodiment 1

[0038] As shown in Figure 2 , the task of this embodiment is to roll the dough from state S0 to state S n . A person as a demonstrator provides a demonstration sequence to achieve the operation purpose, and the robot predicts the action a0…a n-1 that can complete the task through the demonstration sequence. The action space is shown in Figure 3 , wherein O is the center of the dough contour, the center position of the roller is represented by (Pl, Pθ), and θ represents the rotation angle of the roller. Each rolling operation is fixed at 1 cm, so the action is represented by (Pl, Pθ, θ), wherein each action is discretized, and the prediction problem of the action is converted into a multi-classification problem.

[0039] In view of the complexity of the three-dimensional deformable object, the state representation method proposed in this embodiment is a two-dimensional state representation method, which can achieve a good balance between complexity and reliability for the rolling task. This method mainly includes two parts: 1. Extracting the dough contour; 2. Establishing aFigure 4 The state of the dough is represented by a fan-shaped region. Specifically, the state of the dough is represented by:

[0040] S1: Obtain an RGB image of the dough: For example, a camera mounted vertically above can be used to obtain a dough RGB image with a resolution of 512x512;

[0041] S2: Obtain the contour of the dough from the RGB image: Specifically, the RGB image is converted from the RGB space to the HSV (H refers to hue, S refers to saturation, and V refers to value) space, the dough region of interest is obtained by setting the threshold values of H, S, and V, converted into a binary image, then morphological operations are performed with a 5x5 convolution kernel, and finally the contour of the dough is obtained using an edge finding algorithm;

[0042] S3: Obtain the centroid of the contour using the obtained contour, establish a polar coordinate system with the centroid as the center point, and divide the polar coordinate system into multiple fan-shaped regions: Specifically, first obtain the centroid O of the contour using the obtained contour, establish a polar coordinate system with the centroid as the center, and divide the entire two-dimensional space into 72 fan-shaped regions, each with an angle of 5°;

[0043] S4: Calculate the average distance of the contour points in each fan-shaped region to the centroid as one-dimensional features in the multi-dimensional features of the state of the dough (in this example, one-dimensional features in the 72-dimensional features): Specifically, the average distance of the contour points in each fan-shaped region to the centroid is calculated as l0…l 71 Then, the average distance is discretized at intervals of 0.25 cm as the 72-dimensional features of the state of the dough (each average distance as one-dimensional feature in the 72-dimensional features of the state of the dough).

[0044] In other embodiments, the deformable object is not limited to dough, as long as the shape of the object meets the condition (any ray from the centroid of the contour has only one intersection point with the contour).

[0045] Embodiment 2

[0046] This embodiment provides an inverse dynamics model for the state representation method of embodiment 1. The 72-dimensional features obtained in embodiment 1 are not directly used as the input of the network, but are first passed through an encoding layer, which is used to Figure 5 This embodiment adds an encoding layer commonly used in the field of NLP (natural language processing) to the input end of the network, and the main part of the framework uses a parallel structure of RestNet (residual network) and self-attention modules, so that the inverse dynamics model can better capture the overall and local features of the deformable object, improve the expression ability, and obtain a more ideal prediction result.

[0047] Specifically, the inverse dynamics model in the embodiment includes an encoding layer, a self-attention part, a CNN part, and an output part, the self-attention part and the CNN part are connected in parallel, the input state vector in the entire inverse dynamics model is a state vector at a current time, a state vector at a next time, and a state vector difference value between the state vector at the current time and the state vector at the next time, and the output is an execution action of state transition, and the inverse dynamics model is as shown in Figure 5

[0048] I. The encoding layer is used to encode the states of the deformable object at the previous and next times into a three-channel feature map. Specifically, the encoding layer first converts the input state vector (i.e., the "72-dimensional feature" in Embodiment 1) into a one-hot encoded matrix, and then right multiplies a weight vector to complete the encoding of the input. In the training process, the weight is updated by using the gradient descent method (the optimizer used is Adam), to find the encoding mode that minimizes the loss function, wherein the input state vector is the state vector at the current time, the state vector at the next time, and the state vector difference value between the state vector at the current time and the state vector at the next time. By adding the encoding layer, the contour information can be encoded, and the focus of the inverse dynamics model can be entirely on the contour. Compared with directly using an image as the input, a similar effect is achieved, and the training time is shorter.

[0049] Three encoding layers with the above structure are used in the embodiment, and the details are as follows.

[0050] 1. S t (1,72) and S t+1 (1,72) are the state vectors at the previous and next times obtained in Embodiment 1, which are 1*72 (i.e., the "72-dimensional feature" obtained in Embodiment 1), and the elements thereof are integers from 0 to 200. The two state vectors pass through two same encoding layers to obtain two 72*72 feature maps as the first two channels of the encoding layer input.

[0051] 2. Calculate the state vector difference value between the state vectors at the previous and next times (i.e., the state vector at the current time minus the state vector at the next time): D t = S t+1 (1,72)-S t (1,72), and then input the state vector difference value D t ( Figure 5 to the encoding layer as S t+1 -S t ​elements of the state vector are transformed into integers from 0 to 200, and through another encoding layer (which has the same structure as the encoding layer in (1), but with different weight vectors), a 72*72 feature map representing the difference of the state vector is obtained as the third channel of the encoding layer input. Preferably, the channels of the encoding layer input are different when training and predicting different actions (for example, when predicting θ, only the above three channels are input, but when training P l , the information of P θ and θ needs to be combined, so two channels are added to represent them respectively, and the specific training and prediction process will be explained in detail below), so when the information of P θ and θ is converted into an input image channel, a 1*72 dimensional vector is also obtained through an encoding layer, and then a 72*72 feature map is obtained by copying.

[0052] II. The self-attention part and the CNN part are connected in parallel, and the input of both is the feature map obtained by the encoding layer, and the output of both is classification information:

[0053] 1. Self-attention part: Each sector has a spatial sequence relationship, so an attention mechanism is introduced to capture the sequence relationship between each sector. The self-attention part is composed of 6 self-attention modules connected in series, each of which is composed of a multi-head attention layer (the number of heads in this embodiment is 4), a fully connected layer, and a residual connection ( Figure 5 "add&" in the residual connection). The input of the self-attention part is the linear combination of the three channels of the output of the encoding layer, which is also a 72*72 matrix, and in this example, an additional label dimension is added, so that the 73*72 input of the 72*72 feature is increased. The label dimension will serve as an interface for the output of the self-attention part. In other examples, the label dimension can not be added, but the output of the self-attention part can be reduced in dimension.

[0054] 2. CNN part: The CNN part is mainly used to capture local and global features in the feature map, and mainly adopts the structure of RestNet, and its parameters are as follows:

[0055] (1) Input layer: The input layer includes convolutional layers, regularization, activation function layers, pooling layers, and Dropout layers connected in turn, wherein the convolutional layer: convolution kernel 5*5, step 1, channel 64; regularization: BatchNormalization (batch normalization); activation function layer: ReLU; pooling layer: convolution kernel 3*3, step 2; Dropout layer: with a probability of p (0<p<1) to discard neurons in the network, used to prevent overfitting. The input of the entire input layer is (b*3*72*72), and the output is (b*64*36*36), where b refers to Batch_size;

[0056] (2) The middle four convolutional blocks: the middle four convolutional blocks are composed of a first convolutional block, a second convolutional block, a third convolutional block and a fourth convolutional block connected in turn, each of which is composed of n sub-convolutional layers (in this embodiment, the structure of RestNet is adopted, specifically: the first convolutional block is composed of 3 (n = 3) sub-convolutional layers, the second convolutional block is composed of 4 (n = 4) sub-convolutional layers, the third convolutional block is composed of 23 (n = 6) sub-convolutional layers, and the fourth convolutional block is composed of 3 (n = 3) sub-convolutional layers), and each sub-convolutional layer in the convolutional block is composed of three convolutional layers, and the convolutional kernels of the three convolutional layers corresponding to the data input order are 1x1, 3x3, 1x1, respectively, and the channels are n1, n2, n3, respectively. In this example, the first convolutional block: n = 3, n1 = n2 = 64, n3 = 256, (i.e. the first convolutional block is composed of 3 sub-convolutional layers, each of which is composed of three convolutional layers, wherein the convolutional kernel of the first convolutional layer is 1x1, the channel is 64, the convolutional kernel of the second convolutional layer is 3x3, the channel is 64, and the convolutional kernel of the third convolutional layer is 1x1, the channel is 256); the second convolutional block: n = 4, n1 = n2 = 128, n3 = 512, (i.e. the second convolutional block is composed of 4 sub-convolutional layers, each of which is composed of three convolutional layers, wherein the convolutional kernel of the first convolutional layer is 1x1, the channel is 128, the convolutional kernel of the second convolutional layer is 3x3, the channel is 128, and the convolutional kernel of the third convolutional layer is 1x1, the channel is 512); the third convolutional block: n = 6, n1 = n2 = 256, n3 = 1024, (i.e. the third convolutional block is composed of 6 sub-convolutional layers, each of which is composed of three convolutional layers, wherein the convolutional kernel of the first convolutional layer is 1x1, the channel is 256, the convolutional kernel of the second convolutional layer is 3x3, the channel is 256, and the convolutional kernel of the third convolutional layer is 1x1, the channel is 1024); the fourth convolutional block: n = 3, n1 = n2 = 512, n3 = 2048, (i.e. the fourth convolutional block is composed of 3 sub-convolutional layers, each of which is composed of three convolutional layers, wherein the convolutional kernel of the first convolutional layer is 1x1, the channel is 512, the convolutional kernel of the second convolutional layer is 3x3, the channel is 512, and the convolutional kernel of the third convolutional layer is 1x1, the channel is 2048).

[0057] (3) The output of the CNN part is a fully connected layer with an input of 2048 dimensions and an output of 72 dimensions.

[0058] In this example, the RestNet structure is adopted, which can improve the training efficiency and expression ability.

[0059] III. The output part of the network (i.e. Figure 5The "Information Fusion & Classification" section combines the outputs of the self-attention and CNN parts to obtain the classification result. Specifically, the output adds the outputs of the self-attention and CNN parts, and then uses a SoftMax layer to represent the classification probability of each class. In this example, the first output is the rotation angle θ of the roller, which is then encoded and added to the prediction of the rolling center position P. l During the process, P was obtained l Then, its encoding is combined with the encoding of θ and added to the prediction P. θ During the process. The reason for including P l ,P θ The prediction of θ is divided into three processes because if they are performed simultaneously, the dimension of the classification categories increases exponentially.

[0060] The entire inverse dynamics model is based on a data-driven approach. First, a dough-rolling task environment is set up in Plasticine-Lab. Then, action and state transition sequences are randomly collected, and the collected data is used to train the inverse dynamics model.

[0061] Network training: targeting P l ,P θ For each of the three corresponding inverse dynamical models, θ and θ, the training data input is a state vector, and the output is θ; for P... l During training, θ and the state vector from the training data are used as input, but during prediction, the θ predicted in the previous step is used; for P θ Similarly, during training, θ and the state vector from the training data are used as input, but during prediction, the θ predicted in the previous step is used. The specific workflow is as follows:

[0062] First, data collection is performed, such as... Figure 6 As shown, a task scenario was built in Plasticine-Lab, and a large amount of data was collected. Each data point was in the form of (state-action-state). After the model was built, it was pre-trained offline to obtain an inverse dynamics model. Subsequently, based on the UR5 robot platform, a real task scenario was built, and a small amount of data was collected. Each data point was still in the form of (state-action-state). Then, transfer learning was performed to obtain an inverse dynamics model for the real scenario.

[0063] After obtaining the inverse dynamics model, the robot is first taught by a human to provide a series of state sequences that can complete the predetermined task (i.e., the state sequence that achieves the operational objective). Then, the robot executes the actions predicted by the inverse dynamics model based on the two states before and after, and iterates step by step to finally complete the operational task.

[0064] In one specific example, the goal is to roll a 10cm*10cm*10cm dough into a 20cm diameter circle, the steps are as follows:

[0065] 1. Data collection in the physics engine, training of the inverse dynamics model, the goal is to pre-train the inverse dynamics model with a large amount of low-cost data in the physics engine.

[0066] 2. Collect real data and perform transfer learning, the goal is to adapt the inverse dynamics model to real scenarios.

[0067] 3. A skilled person demonstrates, gives a series of state sequences that can complete the given task (i.e. a skilled person gives a state sequence that can achieve the operation purpose), for example, the given task is to roll a 10cm*10cm*10cm dough into a 20cm diameter circle, then roll it once, record the intermediate state, record each state, and get the state sequence of the intermediate dough.

[0068] 4. According to the state sequence provided in the last step, use the inverse dynamics model to obtain the actions to complete the task, and let the robot execute these actions until the task is completed.

[0069] Application scenario of the embodiment of the present application: the present application can give the robot the ability to operate deformable objects, and can be applied to household service robots, for entertainment robots, it can operate deformable objects such as clay and interact with people, and for catering robots, it can be applied to the operation of deformable objects such as dough.

[0070] The above further describes the present application in conjunction with specific / preferred embodiments, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, they can make several substitutions or variations to the described embodiments, and these substitutions or variations shall be deemed to fall within the protection scope of the present application. In the description of the present application, the description of the terms "an embodiment", "some embodiments", "a preferred embodiment", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are contained in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In the case of no mutual contradiction, those skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples. Although the embodiments of the present application and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method of representing a state of a deformable object, characterized by, The method comprises the following steps: S1: obtaining an RGB image of a deformable object; S2: obtaining a contour of the deformable object from the RGB image; S3: obtaining a centroid of the contour, establishing a polar coordinate system with the centroid as the center point, and dividing a plurality of sector regions; S4: calculating an average distance of contour points in each sector region to the centroid as a one-dimensional feature in a multi-dimensional feature of a state of the deformable object.

2. The method of claim 1, wherein the state of the deformable object is represented by a set of points in a three-dimensional space. The step S2 specifically comprises: converting the RGB image from an RGB space to an HSV space, obtaining a region of interest of the deformable object by setting thresholds of H, S and V, converting into a binary image, then performing morphological operations with a 5*5 convolution kernel, and finally obtaining the contour of the deformable object by using an edge finding algorithm.

3. The method of claim 1, wherein: In step S3, the entire two-dimensional space is divided into 72 sector regions, each region being 5°, and in step S4, after the average distance is calculated, the average distance is discretized at intervals of 0.25 cm as 72-dimensional features of the state of the deformable object.

4. The method of claim 1, wherein: The shape of the deformable object satisfies the following condition: any ray starting from the centroid of the contour has only one intersection point with the contour.

5. A reverse dynamics model for the state representation method of claim 1, characterized by, The method comprises an encoding layer, a self-attention part, a CNN part and an output part, wherein: The encoding layer is used to encode states of the deformable object at previous and next moments into a three-channel feature map; The self-attention part and the CNN part are connected in parallel, both inputting the feature map obtained by the encoding layer and outputting classification information; The output part is used to combine outputs of the self-attention part and the CNN part to obtain a classification result.

6. The inverse kinetic model of claim 5, wherein, The encoding layer completes the encoding of the input by converting an input state vector into a one-hot encoded matrix and then right-multiplying a weight vector; wherein the input state vector is a current moment state vector, a next moment state vector and a difference value between the state vectors at the previous and next moments.

7. The inverse kinetic model of claim 5, wherein, The self-attention part is composed of 6 self-attention modules connected in series, each of which is composed of a multi-head attention layer, a full connection layer and a residual connection.

8. The inverse kinetic model of claim 5, wherein, The CNN part adopts the structure of RestNet and comprises an input layer, 4 intermediate convolution blocks and a full connection layer, wherein the input layer comprises convolution layers, regularization, activation function layers, pooling layers and Dropout layers connected in sequence; each of the 4 intermediate convolution blocks is composed of n sub-convolution layers, and each of the sub-convolution layers is composed of three convolution layers.

9. The inverse kinetic model of claim 8, wherein, The 4 intermediate convolution blocks are composed of a first convolution block, a second convolution block, a third convolution block and a fourth convolution block connected in sequence, wherein the first convolution block is composed of 3 sub-convolution layers, the second convolution block is composed of 4 sub-convolution layers, the third convolution block is composed of 6 sub-convolution layers, and the fourth convolution block is composed of 3 sub-convolution layers.

10. The inverse kinetic model of claim 5, wherein, The output part is used to add outputs of the self-attention part and the CNN part and then pass through a SoftMax layer to represent classification probabilities of each category.

Images