A robot hand-eye coordination method based on internal model control

Abstract

The application is based on a robot hand-eye coordination method of internal model control, a vision-movement network and a movement-vision network are established, direct models and inverse models of the internal model control system of the mechanical arm are simulated, the vision-movement network converts image input into robot action as an action generator, and the movement-vision network assists the training of the vision-movement network, therefore, the pre-training of the movement-vision network is established through random writing actions of the mechanical arm, the writing of Chinese character strokes can be successfully completed through input of target stroke image, and the structure is simple and the generalization ability is good.

Description

Technical Field

[0001] This invention relates to the field of robotic calligraphy, and more particularly to a robotic hand-eye coordination method based on internal model control. Background Technology

[0002] Learning calligraphy on robots is considered a complex task. In the field of robotic calligraphy, a major problem is how robots learn to write aesthetic characters or letters. To solve this problem, robotic scientists have studied robots learning to write from various aspects. Current research is divided into two types of training methods: offline learning and online learning.

[0003] In offline learning strategies, the training dataset must be collected in advance and remains unchanged during model training. Therefore, the goal of models using offline learning strategies is to fit the training data as accurately as possible. Due to this characteristic, these methods using offline learning strategies have strong fitting capabilities on the dataset. The quality of the dataset is a crucial factor affecting the performance of calligraphy robots, especially when there is insufficient data, making it difficult for this method to achieve optimal performance. Compared to offline learning strategies, many researchers also employ online learning strategies to design calligraphy robots. To enable calligraphy robots to interact with humans, some researchers utilize human-computer interaction strategies to train the robots. These online learning methods have good generative capabilities; however, the learning time of online learning models is longer than that of offline robots, making it difficult to ensure stable operation of online learning robots during the learning phase. Furthermore, these methods must utilize motion information to guide the robot, which is guided by humans. However, collecting motion information also requires a significant amount of human labor.

[0004] It is evident that these methods have limitations such as high computational cost and low diversity of generated results, which restrict the development of calligraphy robots. In order to overcome these limitations, the present invention proposes a robot hand-eye coordination method based on internal model control. Summary of the Invention

[0005] The technical problem solved by this invention is to address the shortcomings of offline and online learning models by proposing a robot hand-eye coordination method based on internal model control.

[0006] The technical solution of the present invention is as follows:

[0007] A robot hand-eye coordination method based on internal model control includes the following steps:

[0008] Step 1: Create a vision-motor network to simulate the direct function of the model;

[0009] Step 2: Develop a motion-vision network to simulate the inverse model;

[0010] Step 3: The robot writing system collects training data, and then implements the proposed model on the robot writing system;

[0011] Step 4: Train the model using the backpropagation algorithm;

[0012] Step 5: Compare and verify the performance of the proposed model.

[0013] Preferably, in step 1, a vision-motor network is created to simulate the function of the direct model, specifically as follows:

[0014] Step 1A: Generate stroke actions based on the stroke image. The visual-motor network consists of two convolutional layers, two pooling layers, and four fully connected layers. The pose is represented by a three-dimensional vector. The writing action is represented by a three-dimensional vector, where i represents the number of poses in the action. Therefore, the output layer of the visual-motor network has 3i neurons. Since the input image size is 28×28 pixels, the input layer of the visual-motor network has 784 neurons. In this method, except for the output layer, all other layers of the visual-motor network use SeLU as the activation function. Since the writing action is represented by a set of continuous values, the output layer of the visual-motor network uses the Sigmoid function as the activation function.

[0015] Step 1B: The mapping relationship from image s to writing action a is modeled using a vision-motor network V. The robot's hand-eye coordination ability is defined as f: A→S and f -1 S→A

[0016] Where, the image representation of the writing motion trajectory is s∈S, and the image representation of the writing motion trajectory is: s j ∈S, the robot's writing action is represented as a=[g1,g2,…,g i ], a∈A, the stroke representation of the robot's writing is a x For any given point A, the hand-eye coordination of a robot is defined as g = f(p), where p ∈ P and p = f -1 (g), g∈G, the robot arm pose is represented as g=[j1,j2,…,j i ], g∈G,

[0017] In this context, the robot arm pose image captured by the robot vision system is represented as p∈P,j i Let g be the angle value of the i-th joint of the robot arm, g be the robot arm posture, and i be the length of the written action.

[0018] Preferably, in step 2, a motion-vision network is developed to simulate the inverse model, which is a model of the mapping relationship between writing actions and images, to assist in the training of the vision-motion network. The specific steps are as follows:

[0019] Step 2A: Construct a motion-visual network. The motion-visual network consists of three fully connected layers and two transposed convolutional layers. The input and output layers of the motion-visual network contain 3i and 784 neurons, respectively, which is the complete opposite of the visual-motor network. However, similar to the visual-motor network, the output layer of the motion-visual network uses the Sigmoid function as its activation function, while the remaining network layers use SeLU as their activation function. After the motion-visual network generates an image M(a) based on the input action, the visual-motor network is trained using the error s between M(a) and the input of the visual-motor network.

[0020] Step 2B: After the motion-visual network is trained, it guides the training of the visual-motor network. The output action 'a' of the visual-motor network is used as the input of the motion-visual network, and the visual-motor network is trained simultaneously. Since each input action corresponds to only one image, the mapping relationship from action to image is simpler than the mapping relationship from image to action. Furthermore, the motion-visual network directly uses image 's' and the generated image as its basis. Training is performed based on the reconstruction error between the two;

[0021] Step 2C: Since the output of the motion-vision network is an image, the cross-entropy function is used as the loss function for the motion-vision network, defined as:

[0022]

[0023] Step 2D: The training of the vision-motor network depends on the guidance of the motion-vision network; therefore, the loss function of the vision-motor network is defined as follows:

[0024]

[0025] Where M(·) represents the motion-visual network; This represents the write action generated by the vision-motor network, which is represented by L. v Come to train, L v Indicates s and The error between the reconstructed images.

[0026] Preferably, in step 3, the robot writing system collects training data, and then implements the proposed model on the robot writing system. This method includes the following steps:

[0027] Step 3A: First, generate a random action;

[0028] Step 3B: Convert the random motion into the robot's joint values ​​using inverse kinematics equations;

[0029] In step 3C, the robot controls the pen to write on the board based on the changes in these joint values;

[0030] Step 3D: The camera captures the robot's writing result and converts it into a standard image. The captured image is used as a label for a random action, which, along with the action, constitutes the training sample. In this model, the robot's actions are represented using a Cartesian coordinate system. The robot's actions can be seen as the robot sequentially passing through a series of points in the Cartesian coordinate system. Therefore, the robot's actions are a = [g1, g2, ..., g...]. i ], also represented as a series of Cartesian coordinate points, a = (d1, d2, ..., dn). i ), where i represents the number of points, d i Let d represent the position of the robot holding the pen in Cartesian space. i =(x i ,y i ,z i Therefore, in the proposed model, an action sequence is represented by 3i neurons. When the robot performs the action generated by the model, the action represented by Cartesian coordinates is converted into the robot's joint values, a = [g1, g2, ..., g...]. i Based on inverse kinematics equations.

[0031] Preferably, the training of the model using the backpropagation algorithm in step 4 specifically involves:

[0032] Utilizing the reconstruction error between the real image and the generated image, To update the motion-visual network;

[0033] Using real images and generated images The error between them is used to update the visual-motor network;

[0034] The training of the model consists of two steps: First, the motion-visual network is trained; after the motion-visual network converges, the motion-visual network is fixed and trained again until it converges.

[0035] Preferably, step five, verifying the performance of the proposed model, specifically involves:

[0036] First, the model was trained on the Random Writing Dataset (RW) and tested on the Chinese Character Stroke Dataset (CCS).

[0037] Then, a series of ablation experiments and a set of comparative experiments were designed to verify the performance of the model proposed by the method of this invention.

[0038] The present invention has the following beneficial effects:

[0039] (1) The structure is simple. The robot calligraphy writing framework consists of only two neural networks. The training is based solely on the robot's random writing. This feature greatly reduces the complexity of implementing robot writing.

[0040] (2) The model proposed in this invention has good generalization ability. In existing methods that use offline learning strategies, the generalization ability is limited by the training data. When the stroke is not in the training dataset, the performance of these methods will be greatly reduced. Although the training data does not contain the motion information of the stroke, the proposed model can write the stroke based on the image of the stroke. Attached Figure Description

[0041] Figure 1 This is a schematic flowchart of the method of the present invention;

[0042] Figure 2 A schematic diagram illustrating the hardware system modules of the robot according to the method of the present invention;

[0043] Figure 3 The model-controlled robot arm, as proposed in this invention, successfully writes strokes on a whiteboard based on the provided stroke image.

[0044] Figure 4 This is a schematic diagram of the training process for a single network model.

[0045] Figure 5 The data provided are performance test data of the method and single network model of this invention on different datasets.

[0046] Figure 6 The FID values ​​of the method of this invention, the GAN model, and the DRL model;

[0047] Figure 7 This is a flowchart illustrating the overall steps of the method of the present invention. Detailed Implementation

[0048] For the purposes of this invention, the foregoing and other technical contents, features and effects are described in conjunction with the appendix below. Figures 1 to 7 The detailed description of the embodiments will make this clear. All structural details mentioned in the following embodiments are based on the accompanying drawings.

[0049] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings.

[0050] Example 1: A robot hand-eye coordination method based on internal model control (IMC). IMC is a novel control strategy based on process instruction models for controller design, and includes the following steps:

[0051] First, a vision-motor network is created to simulate the direct function of the model;

[0052] Develop a motion-vision network to simulate the inverse model;

[0053] Next, the robot writing system collects training data, and then implements the proposed model on the robot writing system;

[0054] The model is trained using the backpropagation algorithm;

[0055] Finally, the performance of the proposed model is compared and verified.

[0056] Example 2, based on Example 1, involves creating a vision-motion network to simulate the function of the direct model, generating stroke actions based on stroke images, specifically as follows:

[0057] The visual-motor network consists of two convolutional layers, two pooling layers, and four fully connected layers. Pose is represented by a three-dimensional vector; the writing action is represented by a three-dimensional vector, where i represents the number of poses in the action. Therefore, the output layer of the visual-motor network has 3i neurons. Since the input image size is 28×28 pixels, the input layer of the visual-motor network has 784 neurons. In this method, all layers of the visual-motor network except the output layer use SeLU as the activation function. Since the writing action is represented by a set of continuous values, the output layer of the visual-motor network uses the Sigmoid function as the activation function.

[0058] The mapping relationship from image s to writing action a is modeled using a visual-motor network V. The robot's hand-eye coordination ability is defined as f: A→S and f -1 :S→A;

[0059] The image representation of the trajectory of the writing action is s∈S, and the image representation of the trajectory of the writing action is: s j ∈S, the robot's writing action is represented as a=[g1,g2,…,g i ], a∈A. The stroke representation of a robot's handwriting is a. x ∈A;

[0060] The hand-eye coordination of a robot is defined as g = f(p), p ∈ P and p = f -1 (g), g∈G. The robot arm pose is represented as g=[j1,j2,…,j i ], g∈G, where the robot arm pose image captured by the robot vision system is represented as p∈P, j i Let g be the angle value of the i-th joint of the robot arm, g be the robot arm posture, and i be the length of the written action.

[0061] Example 3: Based on Example 1, a motion-vision network is developed to simulate the inverse model, which is a model of the mapping relationship between writing actions and images, used to assist in the training of the vision-motion network. Specifically:

[0062] Since multiple actions written on a writing board produce the same result, the visual-motor network cannot distinguish which action corresponds to which writing result. The many-to-one correspondence between images and actions makes the visual-motor network unsuitable for direct training with actions. If the visual-motor network is trained based on the error between action 'a' and the generated action 'a', the reconstruction accuracy 'a' is very low, and the visual-motor network cannot generate the corresponding stroke action. Therefore, an inverse network of the visual-motor network, namely the motion-visual network M, is introduced into the model to assist in training the visual-motor network. The motion-visual network generates the corresponding trajectory image based on the input writing action. The motion-visual network consists of three fully connected layers and two transposed convolutional layers. The input and output layers of the motion-visual network contain 3i and 784 neurons respectively, completely opposite to the visual-motor network. However, similar to the visual-motor network, the output layer of the motion-visual network uses the Sigmoid function as its activation function, while the remaining network layers use SeLU as their activation function.

[0063] After the motion-visual network generates an image M(a) based on the input action, the error s between M(a) and the input of the visual-motor network is used to train the visual-motor network;

[0064] After the motor-visual network is trained, the motor-visual network is used to guide the training of the visual-motor network. The output action 'a' of the visual-motor network is used as the input of the motor-visual network, and the visual-motor network is trained simultaneously.

[0065] Since each write action corresponds to only one image, the mapping from action to image is simpler than the mapping from image to action, and the motion-vision network directly uses image s and the generated image as its basis. Training is performed based on the reconstruction error between the two;

[0066] Since the output of a motion-vision network is an image, the cross-entropy function is used as the loss function for the motion-vision network, defined as:

[0067]

[0068] The training of a vision-motor network depends on the guidance of a motion-vision network; therefore, the loss function of the vision-motor network is defined as follows:

[0069]

[0070] Where M(·) represents the motion-visual network; This represents the write action generated by the vision-motor network. The vision-motor network uses L... v Come to train, L v Indicates s and The error between the reconstructed images.

[0071] Example 4: Based on Example 1, the robot writing system collects training data and then implements the proposed model on the robot writing system, specifically as follows: Figure 2 As shown, the hardware of the calligraphy robot consists of a 4-DOF robotic arm, a writing pen, a writing board, and a camera. The writing pen is fixed to the end effector of the robotic arm, and the other components are fixed to the robot platform. The robotic arm controls the writing pen to write on the writing board, and the calligraphy robot performs the writing action. The camera is used to capture the writing result and convert the result into an image.

[0072] The training data collection process is as follows: First, a random action is generated. Second, the random action is converted into the robot's joint values ​​through the inverse kinematics equation. Then, the robot controls the pen to write on the board based on the changes in these joint values. Finally, the camera captures the robot's writing result and converts it into a standard image. The captured image is used as a label for the random action and together with the action, it constitutes the training sample.

[0073] In this model, the robot's movements are represented using a Cartesian coordinate system. The robot's movements can be viewed as the robot sequentially passing through a series of points in the Cartesian coordinate system. Therefore, the robot's movements are a = [g1, g2, ..., g...]. i ], also represented as a series of Cartesian coordinate points, a = (d1, d2, ..., dn). i ), where i represents the number of points, d i Let d represent the position of the robot holding the pen in Cartesian space. i =(x i ,y i ,z i Therefore, in the proposed model, an action sequence is represented by 3i neurons. When the robot performs the action generated by the model, the action represented by Cartesian coordinates is converted into the robot's joint values, a = [g1, g2, ..., g...]. i Based on inverse kinematics equations.

[0074] Example 5, based on Example 1, describes the training of the model using the backpropagation algorithm. Specifically, the backpropagation process of the model gradient information is as follows: Figure 1 As shown by the dashed lines, the reconstruction error between the real image and the generated image is utilized. To update the motion-visual network;

[0075] Using real images and generated images The error between them is used to update the visual-motor network;

[0076] Since the training of the visual-motor network depends on the motion-visual network, the training of the model is divided into two steps: first, the motion-visual network is trained; after the motion-visual network converges, the motion-visual network is fixed and trained again until it converges.

[0077] Example 6, based on Example 1, specifically verifies the performance of the proposed model by:

[0078] First, the model was trained on the Random Writing Dataset (RW) and tested on the Chinese Character Stroke Dataset (CCS). Then, a series of ablation experiments and a set of comparative experiments were designed to verify the performance of the model proposed by the method of this invention.

[0079] The RW dataset contains 100,000 training samples collected by the robot writing module. This example includes random writing actions and corresponding result images. The random writing dataset is divided into training and validation sets in a 9:1 ratio. In addition, the stroke dataset is used as a test set to verify the performance of the trained model. The CCS dataset contains more than 3,500 data samples, with more than 500 samples for each stroke.

[0080] After training, the proposed model was validated using a test dataset, and its performance was as follows: Figure 3 As shown, the model controls a robotic arm to successfully write strokes on a whiteboard based on provided stroke images. Experimental results demonstrate that the model has effective generalization ability and can generate correct writing actions based on untrained stroke images.

[0081] First, a comparative experiment was designed to verify the effect of the motion-visual network. Then, in order to test the effect of writing action length on untrained stroke samples, we used random writing actions of different lengths to train the proposed model. Finally, by training the model on different training datasets, we examined the effect of training data on the model's generalization ability.

[0082] Since the motion-vision network was removed from the single-network model, the loss function of the vision-motion network was changed to the mean square error between the actual action a and the generated action a, defined as:

[0083]

[0084] Figure 4For the training process of a single network model, although the convergence speed of the vision-motor network in the proposed model is slower than that of a single network model, the proposed model outperforms the single network model in performance. Comparative experiments were conducted on 300 original models and the single network model on validation and test sets. The cosine similarity between a real image s and an image W(a) generated by the robot writing system was used as the evaluation metric for model performance. Figure 5 As shown, the experimental data are as follows: the performance value of the single network model on the validation set is 0.7515, while the performance value of the model proposed by the method of this invention is 0.8742; the performance value of the single network model on the test set is 0.6529, while the performance value of the model proposed by the method of this invention is 0.7930. It can be seen that the proposed model outperforms the single network model on both the validation set and the test set.

[0085] A set of comparative experiments were designed, including one based on Deep Reinforcement Learning (DRL) and another based on Generative Adversarial Network (GAN), with the model proposed in this invention. Since the GAN-based method cannot use cosine similarity to measure its results, this experiment introduced Frenchet Inception Distance (FID) to evaluate all the generated results of these three methods. The more similar the generated result is to the training set, the smaller the FID value. In addition, since the time spent by the robot to perform actions accounts for most of the time spent training the robot to learn the writing task, the number of times the robot writes during the model modeling process is used to roughly estimate the model modeling time.

[0086] To facilitate comparison with the proposed model, the proposed model was trained on training sets with sample sizes of 9,000, 64,500, and 100,000, respectively. Figure 6 As shown, when the model is trained with 9000 samples, its performance is significantly better than the DRL method and close to that of the GAN method. When the model is trained with 64,500 samples, its performance is better than that of the GAN method. Although the performance of the proposed model is also better than that of the GAN method, the performance improvement is not significant. Compared with the two methods, the model has better performance and shorter training time.

[0087] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robot hand-eye coordination method based on internal model control, characterized in that, Includes the following steps: Step 1: Create a vision-motor network V to simulate the direct function of the model, wherein the vision-motor network V is used to model the mapping relationship from image s to writing action a; Step 2: Develop a motion-visual network M to simulate the reverse model. The motion-visual network M takes writing action a as input and writing trajectory image as output, and models the mapping relationship from writing action to image to assist the training of visual-motion network V. Step 3: Collect training data through the robot writing system. The collection of training data includes: generating a random writing action a, converting the random writing action a into the robot's joint values ​​through inverse kinematics equations, controlling the pen to write on the writing board according to the change information of the joint values, capturing the robot's actual writing result with a camera and converting it into a writing trajectory image s, using the writing trajectory image s as a label for the random writing action a, so that the random writing action a and the writing trajectory image s constitute training samples. Step 4: Train the motion-vision network M and the vision-motion network V in two stages using the backpropagation algorithm, wherein: In the first stage, a random writing action 'a' in the training samples is used as the input of the motion-visual network M, and the corresponding writing trajectory image 's' is used as the label. The parameters of the motion-visual network M are updated according to the reconstruction loss Lm between the writing trajectory image 's' and the predicted trajectory image M(a) generated by the motion-visual network M, until the motion-visual network M converges. The reconstruction loss Lm is defined as: Lm=−[s·ln(M(a))+(1−s)·ln(1−M(a))]; In the second stage, the parameters of the converged motion-visual network M are fixed, and the image s is input into the visual-motor network V to obtain the predicted writing action â = V(s). The predicted writing action â is input into the motion-visual network M with fixed parameters to obtain the predicted trajectory image M(â). Backpropagation is performed based on the reconstruction loss Lv between the image s and the predicted trajectory image M(â). While keeping the parameters of the motion-visual network M unchanged, only the parameters of the visual-motor network V are updated until the visual-motor network V converges. The reconstruction loss Lv is defined as: Lv = −[s·ln(M(â)) + (1−s)·ln(1−M(â))]; Step 5: Input the target stroke image into the trained vision-motion network V to generate the target writing action represented by a series of Cartesian coordinate points. Convert the target writing action into the robot's joint values ​​through inverse kinematics equations. Control the robot to complete the writing of the stroke corresponding to the target stroke image on the writing board. Compare and verify the performance of the proposed model.

2. The robot hand-eye coordination method based on internal model control according to claim 1, characterized in that, The vision-motor network V consists of two convolutional layers, two pooling layers, and four fully connected layers. The input image size of the vision-motor network V is 28×28 pixels, and the input layer has 784 neurons. The writing action is represented by a three-dimensional vector sequence including i poses, and the output layer has 3i neurons. All layers of the vision-motor network V except the output layer use SeLU as the activation function, and the output layer uses the Sigmoid function as the activation function. The robot's hand-eye coordination ability is defined as f:A→S and f⁻¹:S→A, where A represents the set of robot writing actions, S represents the set of writing action trajectory images, f:A→S represents the mapping from writing action to writing trajectory image, f⁻¹:S→A represents the mapping from writing trajectory image to writing action, s∈S, a∈A, and i represents the length of the writing action.

3. The robot hand-eye coordination method based on internal model control according to claim 1, characterized in that, The motion-visual network M consists of three fully connected layers and two transposed convolutional layers. The input layer and output layer of the motion-visual network M have 3i neurons and 784 neurons, respectively. The output layer of the motion-visual network M uses the Sigmoid function as the activation function, and the remaining layers use SeLU as the activation function. In the first stage, the motion-visual network M is updated according to the reconstruction loss Lm between the written trajectory image s and the predicted trajectory image M(a). In the second stage, the parameters of the motion-visual network M are kept fixed, and the visual-motor network V is updated according to the reconstruction loss Lv between image s and the predicted trajectory image M(V(s)).

4. The robot hand-eye coordination method based on internal model control according to claim 1, characterized in that, The robot's writing action 'a' is represented as a series of Cartesian coordinate points a=(d1,d2,…,di), where i represents the number of coordinate points, and di represents the position of the robot holding the pen in Cartesian space, di=(xi,yi,zi). A writing action sequence is represented by 3i consecutive values. When the robot performs the writing action generated by the vision-motion network V, each Cartesian coordinate point di is converted into the corresponding robot joint value vector gi through the inverse kinematic equation, forming the robot joint value sequence [g1,g2,…,gi].

Images