Robotic autonomous grasping method and system based on visual and tactile feedback

Abstract

The application is a robot autonomous grasping method and system based on visual and tactile feedback, which first uses a camera to observe the deformation of the rubber at different joint connections of a manipulator in different directions to obtain current manipulator deformation data, and performs global correlation analysis processing after splicing with the manipulator joint position data to obtain a current manipulator global feature vector; then, splices historical manipulator deformation data with the current manipulator deformation data, and performs time sequence feature analysis processing to obtain a manipulator time sequence feature vector; finally, after feature fusion of the current manipulator global feature vector and the manipulator time sequence feature vector, nonlinear processing is performed to obtain a manipulator deformation prediction vector; the manipulator executes a next-time manipulator grasping action according to the manipulator deformation prediction vector, autonomously adjusts the grasping strength of the manipulator, and realizes the continuity and stability of the manipulator grasping action.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a robot autonomous grasping method and system based on vision and tactile feedback. Background Art

[0002] A visual-tactile robot is a robot that combines vision and touch capabilities. The robot's eyes (cameras) can see the movements of the manipulator and interact through tactile perception. This allows it to not only perceive the environment through vision, but also perform precise operations through tactile feedback. It is widely used in robot tolerance testing, the combination of robots and prostheses, precision operation and assembly, and human-computer interaction.

[0003] Affected by the camera's visual perception problems, the manipulator's tactile perception problems, or the imperfect grasping pressure planning algorithm, the current vision-tactile robot has problems with complex calculations and inconsistent grasping action execution when performing grasping tasks, affecting the flexibility and accuracy of the vision-tactile robot's grasping action. Summary of the Invention

[0004] The purpose of the present invention is to provide a robot autonomous grasping method and system based on vision and tactile feedback.

[0005] The technical solutions of the present invention are as follows:

[0006] A robot autonomous grasping method based on vision and tactile feedback includes the following operations:

[0007] S1. Based on the rubber images of different joints of the manipulator at the current moment acquired by the camera, the deformation variables of different joints of the manipulator at the current moment in different directions are obtained to form the current manipulator deformation variable data; the deformation variables in different directions of the joints include: positive and negative rubber offsets in the x-axis direction, contraction or expansion in the y-axis direction, and rubber stretching or compression in the z-axis direction; the current manipulator deformation variable data and the manipulator joint position data are spliced ​​to obtain the current manipulator deformation feature data; the current manipulator deformation feature data is processed by global correlation analysis to obtain the current manipulator global feature vector;

[0008] S2. Obtain the deformation variables of different joints of the manipulator in different directions at consecutive historical moments to obtain the manipulator shape variable data at several historical moments, thereby forming historical manipulator shape variable data; the historical manipulator shape variable data is combined with the current manipulator shape variable data to obtain manipulator comprehensive feature data; the manipulator comprehensive feature data is subjected to time series feature analysis and processing to obtain a manipulator time series feature vector;

[0009] S3, the current robot global feature vector and the robot time sequence feature vector are fused to obtain a robot fusion vector; the robot fusion vector is processed by a nonlinear process to obtain a robot deformation prediction vector; and the robot performs a robot grasping action at the next moment according to the robot deformation prediction vector.

[0010] The operation of the global correlation analysis process in S1 is realized by self-attention processing, feedforward neural network processing, layer normalization and residual connection processing.

[0011] The operation of the time sequence feature analysis process in S2 is as follows: the robot comprehensive feature data is processed by different linear layers to obtain a robot comprehensive linear query vector, a robot comprehensive linear key vector and a robot comprehensive linear value vector; the robot comprehensive linear query vector, the robot comprehensive linear key vector and the robot comprehensive linear value vector are respectively decomposed into a plurality of sub-quantities to obtain a plurality of robot comprehensive linear query sub-vectors, a plurality of robot comprehensive linear key sub-vectors and a plurality of robot comprehensive linear value sub-vectors; the attention score of each robot comprehensive linear query sub-vector and each robot comprehensive linear key sub-vector is obtained, and after being added to the corresponding mask matrix respectively and processed by probability mapping, a plurality of mask attention scores are obtained; the plurality of mask attention scores are respectively weighted and summed with each robot comprehensive linear value sub-vector to obtain a plurality of initial output quantities; after the plurality of initial output quantities are spliced, linear processing and feedforward neural network processing are performed to obtain a robot time sequence feature vector.

[0012] The deformation variables of the current moment robot in different directions at different joint connections in S1 are obtained based on the distribution information of the rubber pixel points in the current moment robot rubber image at different joint connections and the distribution information of the rubber pixel points in the corresponding joint connection natural state rubber image.

[0013] The deformation variables of the current joint connection in the x-axis, y-axis and z-axis directions are the product of the maximum value of the absolute value of the coordinates of the rubber pixel points in the x-axis, y-axis and z-axis of the current joint connection rubber image and the maximum value of the absolute value of the coordinates of the rubber pixel points in the corresponding coordinate axis of the current joint connection natural state rubber image.

[0014] The operation of the feature fusion in S3 can be realized by multi-head attention processing, in which the robot query vector is obtained based on the robot comprehensive feature data, and the robot value vector and the robot key vector are obtained based on the robot time sequence feature vector.

[0015] A robot system based on visual and tactile feedback is used to realize the robot autonomous grasping method based on visual and tactile feedback, comprising:

[0016] A camera is used to obtain an image.

[0017] The mechanical arm comprises joints and rubber at joint connections, and is used for performing a grabbing action according to a mechanical arm action execution signal transmitted by the control unit;

[0018] The computing unit is used for obtaining deformation amounts of different joints of the mechanical arm in different directions at a current time based on images of the rubber at different joint connections of the mechanical arm at the current time acquired by the camera, and forming current mechanical arm deformation data; the current mechanical arm deformation data and mechanical arm joint position data are spliced to obtain current mechanical arm deformation feature data; the current mechanical arm deformation feature data is processed by global correlation analysis to obtain a current mechanical arm global feature vector; the computing unit is also used for obtaining deformation amounts of different joints of the mechanical arm in different directions at historical continuous time, and obtaining a plurality of historical mechanical arm deformation data to form historical mechanical arm deformation data; the historical mechanical arm deformation data and the current mechanical arm deformation data are spliced to obtain mechanical arm comprehensive feature data; the mechanical arm comprehensive feature data is processed by time sequence feature analysis to obtain a mechanical arm time sequence feature vector; the computing unit is also used for fusing the current mechanical arm global feature vector and the mechanical arm time sequence feature vector to obtain a mechanical arm fusion vector; and the mechanical arm fusion vector is processed by non-linear processing to obtain a mechanical arm deformation prediction vector;

[0019] The control unit is used for decomposing the mechanical arm deformation prediction vector transmitted by the computing unit into a mechanical arm action execution signal and transmitting the mechanical arm action execution signal to the mechanical arm.

[0020] The beneficial effects of the present application are as follows:

[0021] The application provides a robot autonomous grabbing method based on visual and tactile feedback, first, a camera is used to obtain rubber images of different joint connecting positions of a manipulator at the current moment, and deformation variables of different joint connecting positions of the manipulator in different directions at the current moment are obtained after analyzing the rubber deformation variables, so that current manipulator deformation variable data are formed; the current manipulator deformation variable data and manipulator joint position data are spliced to obtain current manipulator deformation characteristic data containing manipulator joint position information and applied pressure information; global correlation analysis and processing are performed on the current manipulator deformation characteristic data, the global correlation of the joint connecting positions is captured, and a current manipulator global feature vector is obtained; then, deformation variables of different joint connecting positions of the manipulator in different directions at historical continuous moments are obtained, and historical manipulator deformation variable data reflecting pressure and deformation information when the manipulator joints execute historical actions are obtained; the historical manipulator deformation variable data and the current manipulator deformation variable data are spliced to obtain manipulator comprehensive feature data; the manipulator comprehensive feature data is subjected to time sequence feature analysis and processing, the dynamic pressure and deformation change characteristics of the manipulator action are focused on, and a manipulator time sequence feature vector is obtained; finally, after the current manipulator global feature vector and the manipulator time sequence feature vector are fused, the current manipulator action pressure is analyzed, and the historical manipulator action data are combined to obtain a manipulator fusion vector; the manipulator fusion vector is subjected to nonlinear processing to obtain a manipulator deformation prediction vector; the manipulator executes a manipulator grabbing action at the next moment according to the manipulator deformation prediction vector, autonomously adjusts the grabbing force of the manipulator, ensures that the object can be firmly grabbed and damage caused by excessive force is avoided, and the continuity of the manipulator grabbing action is realized.

[0022] The robot autonomous grabbing method based on visual and tactile feedback provided by the application can observe the deformation characteristics of the manipulator through the camera, analyze the deformation characteristics of the manipulator at the current moment, refer to the deformation information of the manipulator action at the historical moment, perform multi-moment information fusion, predict the operation strategy of the manipulator at the next moment, and realize efficient, stable and continuous grabbing.

[0023] The robot autonomous grabbing method based on visual and tactile feedback provided by the application has stable action planning effect and can be applied to various complex scenes, and can intelligently adjust the grabbing mode to complete the task regardless of whether the target object shape is regular or irregular, and whether the material is hard or soft. BRIEF DESCRIPTION OF DRAWINGS

[0024] The scheme and advantages of the present application will become clear to those skilled in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not considered as limiting the application.

[0025] In the drawings:

[0026] Figure 1A flowchart of the robot autonomous grasping method in the embodiment. DETAILED DESCRIPTION

[0027] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings.

[0028] The embodiment provides a robot system based on visual and tactile feedback, which is used to realize a robot autonomous grasping method based on visual and tactile feedback, and comprises the following.

[0029] a camera configured to acquire images;

[0030] a robot hand comprising joints and rubber at joint connections, and configured to perform grasping actions according to robot hand action execution signals transmitted by the control unit.

[0031] a computing unit configured to obtain deformation variables in different directions of different joint connections of the robot hand at a current moment based on images of the rubber at the different joint connections of the robot hand acquired by the camera, to form current robot hand deformation variable data, to splice the current robot hand deformation variable data and robot hand joint position data to obtain current robot hand deformation feature data, and to process the current robot hand deformation feature data by global correlation analysis to obtain a current robot hand global feature vector, to obtain deformation variables in different directions of different joint connections of the robot hand at a plurality of historical moments to obtain a plurality of historical robot hand deformation variable data, to form historical robot hand deformation variable data, to splice the historical robot hand deformation variable data and the current robot hand deformation variable data to obtain robot hand comprehensive feature data, to process the robot hand comprehensive feature data by time sequence feature analysis to obtain a robot hand time sequence feature vector, and to obtain a robot hand fusion vector by fusing the current robot hand global feature vector and the robot hand time sequence feature vector, and to obtain a robot hand deformation prediction vector by nonlinear processing of the robot hand fusion vector.

[0032] a control unit configured to decompose the robot hand deformation prediction vector transmitted by the computing unit into robot hand action execution signals and transmit the robot hand action execution signals to the robot hand.

[0033] A robot autonomous grasping method based on visual and tactile feedback, as shown in Figure 1 , and the specific steps are as follows.

[0034] S1, based on the camera, acquiring images of the rubber at different joint connections of the robot hand at a current moment, obtaining deformation variables in different directions of different joint connections of the robot hand at the current moment, and forming current robot hand deformation variable data; splicing the current robot hand deformation variable data and robot hand joint position data to obtain current robot hand deformation feature data; and processing the current robot hand deformation feature data by global correlation analysis to obtain a current robot hand global feature vector.

[0035] The camera is used to obtain the rubber image of different joint connections of the robot at the current time, and the deformation of the rubber joint connection of the robot is observed to obtain the deformation of different directions of different joint connections of the robot at the current time, and the current robot deformation data is formed; the current robot deformation data and the robot joint position data are spliced to obtain the current robot deformation feature data containing the robot joint position information and the applied pressure information; the current robot deformation feature data is processed by global correlation analysis to capture the global correlation between the joint connections, and the current robot global feature vector is obtained.

[0036] Firstly, based on the rubber image of different joint connections of the robot at the current time obtained by the camera, the deformation of different directions of different joint connections of the robot at the current time is obtained.

[0037] Among them, the deformation of different directions of different joint connections of the robot at the current time is obtained based on the rubber pixel point distribution information in the rubber image of different joint connections of the robot at the current time and the rubber pixel point distribution information in the natural state rubber image of the corresponding joint connection.

[0038] For example, there are 14 joint connections, which are the joint connections of the 5 fingers of the robot, and the deformation of different directions of each joint connection has 6, including the positive and negative rubber offset in the x-axis direction, the contraction or expansion in the y-axis direction, and the rubber stretching or compression in the z-axis direction.

[0039] Taking the current joint connection as an example, the deformation of the x-axis, y-axis and z-axis directions of the current joint connection of the robot at the current time is the product of the maximum value of the absolute value of the coordinates of the rubber pixel points in the x-axis, y-axis and z-axis of the current joint connection rubber image and the maximum value of the absolute value of the coordinates of the rubber pixel points in the corresponding coordinate axis of the current joint connection natural state rubber image, respectively.

[0040] Then, based on the deformation of different directions of different joint connections of the robot at the current time, the current robot deformation data in the form of 14x6 matrix is formed.

[0041] Next, after splicing the current robot deformation data and the robot joint position data (a matrix formed by the position information of all joint connections of the robot), the current robot deformation feature data containing the joint position information is obtained.

[0042] Finally, the current robot deformation feature data is processed by global correlation analysis to obtain the current robot global feature vector. The operation of global correlation analysis is realized by self-attention processing, feedforward neural network processing, layer normalization and residual connection processing.

[0043] S2, obtain deformation variables of different joints of the robot at different directions at historical continuous time points to obtain historical robot deformation variable data; the historical robot deformation variable data and the current robot deformation variable data are spliced to obtain robot comprehensive feature data; the robot comprehensive feature data is processed by time sequence feature analysis to obtain a robot time sequence feature vector.

[0044] Obtain deformation variables of different joints of the robot at different directions at historical continuous time points to obtain historical robot deformation variable data capable of reflecting pressure deformation information of the robot joints when performing historical actions; splice the historical robot deformation variable data and the current robot deformation variable data to obtain robot comprehensive feature data; process the robot comprehensive feature data by time sequence feature analysis to focus on dynamic pressure and deformation change characteristics of the robot action to obtain a robot time sequence feature vector.

[0045] First, obtain deformation variables of different joints of the robot at different directions at historical continuous time points (preferably the previous 9 historical time points of the current time point) to obtain several historical robot deformation variable data to form historical robot deformation variable data.

[0046] Then, splice the historical robot deformation variable data and the current robot deformation variable data (which can be spliced after being respectively converted into vector form) to obtain robot comprehensive feature data reflecting time sequence information of the robot.

[0047] Finally, process the robot comprehensive feature data by time sequence feature analysis to focus on dynamic change characteristics of the robot action to obtain a robot time sequence feature vector.

[0048] The operation steps of the above time sequence feature analysis processing are as follows.

[0049] First step: process the robot comprehensive feature data by different linear layers (which can be achieved by multiplying the matrix form of the robot comprehensive feature data with a query item parameter matrix, a key item parameter matrix and a value item parameter matrix respectively) to obtain a robot comprehensive linear query vector, a robot comprehensive linear key vector and a robot comprehensive linear value vector.

[0050] Second step: decompose the robot comprehensive linear query vector, the robot comprehensive linear key vector and the robot comprehensive linear value vector into several sub-quantities respectively (which can be achieved by multiplying the robot comprehensive linear query vector, the robot comprehensive linear key vector and the robot comprehensive linear value vector with different parameter weights respectively) to obtain several robot comprehensive linear query sub-vectors, several robot comprehensive linear key sub-vectors and several robot comprehensive linear value sub-vectors.

[0051] Third step: obtain the attention score of each robot comprehensive linear query sub-vector and each robot comprehensive linear key sub-vector, respectively add the corresponding mask matrix after probability mapping processing to obtain a plurality of mask attention scores. The attention score is obtained based on the product of the robot comprehensive linear query sub-vector and the robot comprehensive linear key sub-vector.

[0052] Fourth step: a plurality of mask attention scores are respectively weighted and summed with each robot comprehensive linear value sub-vector to obtain a plurality of initial output quantities; after splicing, linear processing and feedforward neural network processing, a robot time sequence feature vector is obtained.

[0053] S3, the current robot global feature vector and the robot time sequence feature vector are fused to obtain a robot fusion vector; the robot fusion vector is nonlinearly processed to obtain a robot deformation prediction vector; the robot performs the next time robot grasping action according to the robot deformation prediction vector.

[0054] After the current robot global feature vector and the robot time sequence feature vector are fused, the current robot action pressure is analyzed while combining the historical robot action data, the multi-time robot deformation information is fused to obtain a robot fusion vector; the robot fusion vector is nonlinearly processed to obtain a robot deformation prediction vector; the robot performs the next time robot grasping action according to the robot deformation prediction vector.

[0055] First, the current robot global feature vector and the robot time sequence feature vector are fused to obtain a robot fusion vector containing attention current time robot feature information and historical time robot feature information. The operation of feature fusion can be realized by multi-head attention processing. In the multi-head attention processing, the robot query vector is obtained based on the robot comprehensive feature data, and the robot value vector and the robot key vector are obtained based on the robot time sequence feature vector.

[0056] Then, the robot fusion vector is nonlinearly processed to obtain a robot deformation prediction vector. The operation of nonlinear processing can be realized by processing the robot fusion vector through the trained neural network layer.

[0057] Finally, the robot performs the next time robot grasping action according to the predicted rubber deformation amount of each joint connection in the robot deformation prediction vector, autonomously adjusts the grasping force of the robot, ensures that the object can be firmly grasped and avoids damage caused by excessive force, and realizes the continuity of the robot grasping action.

[0058] The embodiment provides a robot autonomous grabbing method based on visual and tactile feedback, first, a camera is used to obtain rubber images of different joint connecting positions of a manipulator at a current moment, and after analyzing rubber deformation variables, the deformation variables of different directions of different joint connecting positions of the manipulator at the current moment are obtained, so that current manipulator deformation variable data are formed; the current manipulator deformation variable data and manipulator joint position data are spliced, so that current manipulator deformation feature data containing manipulator joint position information and applied pressure information are obtained; global correlation analysis processing is performed on the current manipulator deformation feature data, global correlation of joint connecting positions is captured, and a current manipulator global feature vector is obtained; then, deformation variables of different directions of different joint connecting positions of the manipulator at historical continuous moments are obtained, so that historical manipulator deformation variable data reflecting pressure and deformation information when the manipulator joints execute historical actions are obtained; the historical manipulator deformation variable data and the current manipulator deformation variable data are spliced, so that manipulator comprehensive feature data are obtained; the manipulator comprehensive feature data are subjected to time sequence feature analysis processing, dynamic pressure and deformation change features of manipulator actions are focused on, and a manipulator time sequence feature vector is obtained; finally, after feature fusion of the current manipulator global feature vector and the manipulator time sequence feature vector, current manipulator action pressure is analyzed, and historical manipulator action data are combined, so that a manipulator fusion vector is obtained; the manipulator fusion vector is subjected to nonlinear processing, so that a manipulator deformation prediction vector is obtained; the manipulator executes a manipulator grabbing action at a next moment according to the manipulator deformation prediction vector, autonomously adjusts a grabbing force of the manipulator, ensures that an object can be firmly grabbed and damage caused by excessive force is avoided, and continuity of the manipulator grabbing action is realized.

[0059] The robot autonomous grabbing method based on visual and tactile feedback provided by the embodiment can observe manipulator deformation features through a camera, analyze manipulator deformation features at a current moment, refer to manipulator action deformation information at historical moments, perform multi-moment information fusion, predict a manipulator operation strategy at a next moment, and realize efficient, stable and continuous grabbing.

[0060] The robot autonomous grabbing method based on visual and tactile feedback provided by the embodiment has stable action planning effect and can be applied to various complex scenes. Whether a target object is regular in shape or irregular in shape, hard or soft in material, the method can intelligently adjust a grabbing mode to complete a task.

Claims

1. A robot autonomous grasping method based on visual and haptic feedback, characterized in that, The method comprises the following steps: S1, obtaining the deformation variables of different joints of the robot arm in different directions based on the rubber image of the joints obtained by the camera at the current time, to form the current robot arm deformation variable data; The deformation variables of different joints in different directions include: positive and negative rubber offset in the x-axis direction, contraction or expansion in the y-axis direction, and rubber stretching or compression in the z-axis direction; The current robot arm deformation variable data and the robot arm joint position data are spliced to obtain the current robot arm deformation feature data; The current robot arm deformation feature data is processed by global correlation analysis to obtain the current robot arm global feature vector; S2, obtaining the deformation variables of different joints of the robot arm in different directions at historical continuous time, to obtain a plurality of historical robot arm deformation variable data, and forming the historical robot arm deformation variable data; the historical robot arm deformation variable data and the current robot arm deformation variable data are spliced to obtain the robot arm comprehensive feature data; The robot arm comprehensive feature data is processed by time sequence feature analysis to obtain the robot arm time sequence feature vector; S3, the current robot arm global feature vector and the robot arm time sequence feature vector are fused to obtain the robot arm fusion vector; the robot arm fusion vector is processed by nonlinear processing to obtain the robot arm deformation prediction vector; The robot arm performs the robot arm grasping action at the next time according to the robot arm deformation prediction vector.

2. The robot autonomous grasping method based on visual and tactile feedback according to claim 1, characterized in that, The global correlation analysis processing in S1 is realized by self-attention processing, feedforward neural network processing, layer normalization and residual connection processing.

3. The robot autonomous grasping method based on visual and tactile feedback according to claim 1, characterized in that, In S2, the time sequence feature analysis processing is specifically: The robot arm comprehensive feature data is processed by different linear layers to obtain a robot arm comprehensive linear query vector, a robot arm comprehensive linear key vector and a robot arm comprehensive linear value vector; The robot arm comprehensive linear query vector, the robot arm comprehensive linear key vector and the robot arm comprehensive linear value vector are respectively decomposed into a plurality of robot arm comprehensive linear query sub-vectors, a plurality of robot arm comprehensive linear key sub-vectors and a plurality of robot arm comprehensive linear value sub-vectors; The attention scores of each robot arm comprehensive linear query sub-vector and each robot arm comprehensive linear key sub-vector are obtained, and are respectively added to the corresponding mask matrix and processed by probability mapping to obtain a plurality of mask attention scores; The plurality of mask attention scores are respectively weighted and summed with each robot arm comprehensive linear value sub-vector to obtain a plurality of initial output quantities; After splicing, the plurality of initial output quantities are processed by linear processing and feedforward neural network processing to obtain the robot arm time sequence feature vector.

4. The robot autonomous grasping method based on visual and tactile feedback according to claim 1, characterized in that, In S1, the deformation variables of different joints of the robot arm in different directions at the current time are obtained based on the rubber pixel distribution information in the rubber image of the joints at the current time and the rubber pixel distribution information in the natural state rubber image of the corresponding joint.

5. The robot autonomous grasping method based on visual and tactile feedback according to claim 4, characterized in that, The deformation amount of the current joint in the x-axis, y-axis and z-axis directions is the product of the maximum value of the absolute value of the coordinates of the rubber pixels in the x-axis, y-axis and z-axis of the current joint rubber image and the maximum value of the absolute value of the coordinates of the rubber pixels in the corresponding coordinate axis of the current joint natural state rubber image.

6. The robot autonomous grasping method based on visual and tactile feedback according to claim 1, characterized in that, In the S3, the feature fusion operation can be implemented by a multi-head attention processing, in which the robot query vector is obtained based on the robot comprehensive feature data, and the robot value vector and the robot key vector are obtained based on the robot time sequence feature vector.

7. A robot system based on visual and tactile feedback for implementing the robot autonomous grasping method based on visual and tactile feedback according to claim 1, characterized in that it comprises: a camera for acquiring images; a robot comprising joints and joint rubber for performing grasping actions according to the robot action execution signals transmitted by the control unit; a computing unit for obtaining the deformation amount of different joints of the robot in different directions at the current time based on the current joint rubber images of the robot acquired by the camera, forming the current robot deformation data; the current robot deformation data and the robot joint position data are spliced to obtain the current robot deformation feature data; the current robot deformation feature data is processed by global correlation analysis to obtain the current robot global feature vector; for obtaining the deformation amount of different joints of the robot in different directions at the historical continuous time to obtain a plurality of historical robot deformation data, forming the historical robot deformation data; the historical robot deformation data and the current robot deformation data are spliced to obtain the robot comprehensive feature data; the robot comprehensive feature data is processed by time sequence feature analysis to obtain the robot time sequence feature vector; for the current robot global feature vector and the robot time sequence feature vector, the feature fusion is performed to obtain the robot fusion vector; the robot fusion vector is processed by nonlinear processing to obtain the robot deformation prediction vector; a control unit for decomposing the robot deformation prediction vector transmitted by the computing unit into robot action execution signals and transmitting them to the robot.

Images