Control method and device for double-arm cooperation of humanoid robot

Abstract

The invention discloses a control method and device for cooperation of two arms of a humanoid robot. The method comprises the steps that an action image sequence generated in the process that the two arms of the humanoid robot execute a target task is acquired; a first feature vector sequence corresponding to the action image sequence, a mechanical arm joint position vector sequence and a second feature vector sequence corresponding to the target mask image sequence are determined; target style variables corresponding to the first feature vector sequence, the second feature vector sequence, the mechanical arm joint position vector sequence and the action image sequence are coded, and a feature key-value pair sequence is output; the feature key-value pair sequence and the humanoid robot body position vector are jointly used as training samples; and performing supervised imitation learning on the plurality of training samples, and generating a motion control model for double-arm collaboration. Therefore, based on the motion control model, the two arms of the humanoid robot can complete more complex long action sequence tasks, and the success rate of target task execution is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a control method and device for the coordination of two arms of a humanoid robot. Background Art

[0002] With the rapid development of artificial intelligence technology, robotics has made significant progress and has been widely used in various fields. Among them, humanoid robots, as robots that can simulate human behaviors and actions, have extremely high research and application value. However, the dual-arm motion control of humanoid robots has always been a challenging problem. Traditional dual-arm motion control methods mainly rely on precise modeling and complex control algorithms, but due to the complexity and uncertainty of the dual arms of humanoid robots, the control effects of these methods are often unsatisfactory. In addition, these methods usually require a lot of debugging and optimization, resulting in a long development cycle and high cost.

[0003] In recent years, imitation learning, as an emerging machine learning technology, has provided new ideas for robot dual-arm motion control. Imitation learning learns from the action data of human experts and uses continuous action sequence images as the original input of the model, so that the robot can directly imitate the motion trajectory and behavior of humans, thereby performing a series of complex and continuous actions. This method is simple, intuitive and easy to implement, and has achieved good results in some specific scene tasks. However, pure imitation learning also has some limitations; first, imitation learning requires a large amount of high-quality data to train the model, and these data are often difficult to obtain; second, imitation learning may be affected by data bias and noise, resulting in limited generalization ability of the learned model; therefore, a single continuous action sequence image is used as the only input of imitation learning, which makes the content of model learning more divergent, and the action learning of the target object being operated is not focused enough, resulting in reduced generalization ability and robustness of the imitation learning results.

[0004] In the existing end-to-end motion control algorithms for robotic arms based on imitation learning, most of them only use a single camera image sequence as a single image data source for imitation learning model training, and the algorithm is generally applied to traditional robotic arms to perform simple tasks. However, with the development of humanoid robots and the diversification of robotic arm application scenarios, only using image training imitation learning models for end-to-end motion control of robotic arms can no longer meet the needs of humanoid robots to perform long-term dual-arm collaborative tasks in complex scenarios. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention provides a control method and device for the coordination of two arms of a humanoid robot. The method can improve the operation accuracy of the two arms of the humanoid robot when they coordinate to perform a target task, thereby improving the success rate of the humanoid robot in performing the target task.

[0006] According to a first aspect of an embodiment of the present invention, a control method for dual-arm collaboration of a humanoid robot is provided, comprising: obtaining an action image sequence generated during the execution of a target task by the dual arms of the humanoid robot; wherein the action image sequence is used to indicate a set formed by a plurality of original images arranged in chronological order; determining a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of a mask image sequence corresponding to the action image sequence, and a robotic arm joint position vector sequence corresponding to the action image sequence; wherein the mask image sequence includes a plurality of target mask images arranged in chronological order, and the target mask image is used to indicate an image after masking a target object in the original image; encoding the first feature vector sequence, the second feature vector sequence, the robotic arm joint position vector sequence, and the target style variable corresponding to the action image sequence using a Transformer encoder, and outputting a feature key-value pair sequence; using the feature key-value pair sequence and the humanoid robot body position vector together as training samples; and performing supervised imitation learning on a plurality of the training samples based on a Transformer decoder to generate a motion control model for dual-arm collaboration.

[0007] Optionally, the determining of the first feature vector sequence corresponding to the action image sequence and the second feature vector sequence of the mask image sequence corresponding to the action image sequence includes: for any original image in the action image sequence: performing feature extraction processing on the original image and the target mask image corresponding to the original image, respectively, and outputting original image features and target mask image features; performing feature mapping processing on the original image features and the target mask image features, and generating a fused image and mapping features; determining the position information of the mapping features in the fused image; using linear layers to process the original image features and the target mask image features, respectively, and outputting an original image feature vector and a target mask image feature vector; adding the position information to the original image feature vector to generate a first feature vector corresponding to the original image; adding the position information to the target mask image feature vector to generate a second feature vector corresponding to the target mask image; generating a first feature vector sequence based on the first feature vector corresponding to each of the original images in the action image sequence; generating a second feature vector sequence based on the second feature vector corresponding to the target mask image of each of the original images in the action image sequence.

[0008] Optionally, determining the robotic arm joint position vector sequence corresponding to the action image sequence includes: collecting robotic arm joint position information at different moments during the process of the humanoid robot's two arms performing the target task to obtain a robotic arm joint position sequence; time-aligning the robotic arm joint position sequence with the action image sequence to obtain a robotic arm joint position sequence corresponding to the action image sequence; encoding the robotic arm joint position sequence to output a robotic arm joint position vector sequence.

[0009] Optionally, the method also includes: obtaining a sequence of original images corresponding to the process of the humanoid robot's two arms performing the target task; for any original image in the original image sequence: obtaining target object prior knowledge and robotic arm joint position information corresponding to the original image; combining the original image, target object prior knowledge, and robotic arm joint position information into action image knowledge data; wherein the robotic arm joint position information corresponding to the original image is used to indicate the robotic arm joint position coordinates collected at the time of collecting the original image; based on the action image knowledge data corresponding to each of the original images in the original image sequence, generating an action image knowledge sequence; performing action block processing on the action image knowledge sequence to generate an action image sequence.

[0010] Optionally, the method further includes: performing target object detection on the original image to generate a target detection frame and target object category knowledge; performing mask processing on the original image using the target detection frame to generate a mask image; and using the mask image and target object category knowledge together as target object prior knowledge.

[0011] Optionally, the action image knowledge sequence is subjected to action block processing to generate an action image sequence; comprising: at each moment corresponding to the action image knowledge sequence, the action image knowledge sequence is subjected to block processing according to a preset step size, and a plurality of action image blocks are output; wherein the action image blocks include a plurality of action image knowledge data; exponentially weighted processing is performed on the plurality of action image blocks, and a block weight corresponding to each of the action image blocks is output; for any of the action image blocks: based on the block weight, weighted sampling is performed on the action image knowledge data in the action image block, and a sampling result is output; based on the sampling result corresponding to each of the action image blocks, an action image sequence is generated.

[0012] Optionally, the method also includes: generating a target object category sequence based on the target object category knowledge corresponding to each of the original images in the action image sequence; encoding the action image sequence and the target object category knowledge sequence respectively, and outputting a target object category knowledge vector sequence and an action vector sequence; using a transformer encoder to encode the robotic arm joint position vector sequence, the target object category knowledge vector sequence and the action vector sequence respectively, and generate a target style variable.

[0013] Optionally, the method further includes: acquiring a sequence of images to be tested generated when the humanoid robot's arms perform a target task; performing predictive processing on the sequence of images to be tested using the motion control model, and outputting predicted actions of the humanoid robot.

[0014] According to a second aspect of an embodiment of the present invention, a control device for dual-arm coordination of a humanoid robot is further provided, the device comprising: a first acquisition module, used to acquire a sequence of action images generated when the dual-arms of the humanoid robot perform a target task; wherein the action image sequence is used to indicate a set formed by a plurality of original images arranged in chronological order; a first determination module, used to determine a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of a mask image sequence corresponding to the action image sequence, and a sequence of robot arm joint position vectors corresponding to the action image sequence; wherein the mask image sequence includes a plurality of target mask images arranged in chronological order. The target mask image is used to indicate the image after the target object in the original image is masked; the first encoding module is used to use the Transformer encoder to encode the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variables corresponding to the action image sequence, and output a feature key-value pair sequence; the first generation module is used to use the feature key-value pair sequence and the humanoid robot body position vector as training samples; based on the Transformer decoder, supervised imitation learning is performed on several of the training samples to generate a motion control model for dual-arm collaboration.

[0015] According to a third aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in the first aspect.

[0016] According to a fourth aspect of an embodiment of the present invention, there is further provided a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0017] The embodiment of the present invention provides a control method for the dual-arm collaboration of a humanoid robot, the method comprising: first, obtaining an action image sequence generated during the dual-arm execution of a target task by the humanoid robot; wherein the action image sequence is used to indicate a set formed by a plurality of original images arranged in chronological order; secondly, determining a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of a mask image sequence corresponding to the action image sequence, and a robotic arm joint position vector sequence corresponding to the action image sequence; wherein the mask image sequence includes a plurality of target mask images arranged in chronological order, and the target mask image is used to indicate an image after masking a target object in the original image; then, using a Transformer encoder to encode the first feature vector sequence, the second feature vector sequence, the robotic arm joint position vector sequence, and the target style variable corresponding to the action image sequence, and outputting a feature key-value pair sequence; finally, using the feature key-value pair sequence and the humanoid robot body position vector together as training samples; and performing supervised imitation learning on the plurality of the training samples based on a Transformer decoder to generate a motion control model for dual-arm collaboration. This embodiment uses the action image, target mask image, and mechanical arm joint position information corresponding to the same moment in the process of the humanoid robot's dual arms performing the target task as multi-source input for imitation learning; thereby, it can enhance the recognition and positioning capabilities of the humanoid robot's dual arms for the target object, thereby improving the control accuracy of the humanoid robot's dual arms in coordination. This embodiment applies the Transformer architecture to the imitation learning of the humanoid robot's dual arms in coordination, effectively enhancing the humanoid robot's dual arms' coordination capabilities and autonomous task completion capabilities, so that the humanoid robot's dual arms can complete more complex long action sequence tasks, effectively improving the success rate of the humanoid robot in performing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0019] Figure 1 A schematic flow chart of a control method for dual-arm coordination of a humanoid robot provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a process of generating an action image sequence in one embodiment of the present invention;

[0021] Figure 3 A schematic diagram of a process for generating target style variables in one embodiment of the present invention;

[0022] Figure 4 A schematic diagram of a process for generating a first eigenvector and a second eigenvector in one embodiment of the present invention;

[0023] Figure 5 A schematic diagram of the structure of a control device for dual-arm coordination of a humanoid robot provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] In the imitation learning method, by directly mapping the RGB image taken by the visible light camera to the dual-arm motion of the humanoid robot, this mapping from pixels to motion is very suitable for the humanoid robot to perform fine operations. However, fine operation tasks usually involve complex physical properties of the target object being operated, such as category, shape, size, etc. Therefore, the present invention uses the prior knowledge of the target object as supplementary data for the dual-arm collaborative end-to-end motion control algorithm.

[0026] In recent years, the Transformer architecture has been widely used in various fields of deep learning and has achieved good results. The self-attention mechanism is introduced in the Transformer deep learning model architecture, so that the entire model can simultaneously consider all the position information of the input sequence and assign different attention weights according to different parts of the sequence, so as to better capture the semantic information in the sequence data. Therefore, the Transformer architecture is very suitable for processing sequence-to-sequence tasks and has the ability to synthesize information across sequences and generate new sequences.

[0027] Aiming at the complex tasks that may be encountered in the process of coordinated control of the two arms of a humanoid robot, the control method of the present invention first detects and identifies the target object being operated in the process of the two arms of the humanoid robot performing the target task based on the target detection algorithm, and obtains the prior knowledge of the target object (i.e., the target detection frame and the target object category knowledge); then, the target detection frame is used to perform mask cropping on the original image to obtain the mask image; secondly, an imitation learning algorithm based on the Transformer architecture and action block technology is constructed to learn and generate a motion control model for dual-arm coordination on the action sequence. By performing target detection and mask operations on the original image in each action sequence, and combining the position information of the robot arm joints in each action moment, an action image knowledge sequence is formed; finally, the action image knowledge sequence is used as multi-source training data for the end-to-end motion control algorithm.

[0028] The control method of the present invention is based on the Transformer architecture and action blocking technology as the main method of imitation learning, and uses the action image knowledge sequence to train the imitation learning algorithm model, and finally obtains a motion control model of dual-arm coordination, which provides a new control method for the dual-arm coordination of the humanoid robot to perform target tasks, effectively enhances the dual-arm coordination ability and autonomous task completion ability of the humanoid robot, enables the dual-arm of the humanoid robot to complete more complex long action sequence tasks, and effectively improves the success rate of task execution.

[0029] like Figure 1 , which is a flow chart of a control method for dual-arm coordination of a humanoid robot provided by an embodiment of the present invention.

[0030] A control method for the coordination of two arms of a humanoid robot comprises at least the following steps:

[0031] S101, obtaining a sequence of action images generated when the humanoid robot's arms perform a target task; wherein the action image sequence is used to indicate a set of several original images arranged in time sequence;

[0032] S102, determining a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of the mask image sequence corresponding to the action image sequence, and a robot arm joint position vector sequence corresponding to the action image sequence; wherein the mask image sequence includes a plurality of target mask images arranged in chronological order, and the target mask image is used to indicate an image after masking the target object in the original image;

[0033] S103, using a Transformer encoder to encode the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variables corresponding to the action image sequence, and output a feature key-value pair sequence;

[0034] S104, taking the feature key-value pair sequence and the humanoid robot body position vector as training samples; performing supervised imitation learning on the training samples based on the Transformer decoder to generate a motion control model for dual-arm collaboration.

[0035] In S101, there are many application scenarios for the humanoid robot to perform the target task, for example: a humanoid robot used in daily life uses both arms to collaboratively grab an apple, or a humanoid robot used on a production line uses both arms to collaboratively process a device.

[0036] The action image sequence is used to indicate a set of original image-related data groups of key actions of the humanoid robot's arms collected by a visible light camera in a process in which the humanoid robot's arms perform a target task, which are arranged in chronological order.

[0037] The original image related data group includes: the original image, and / or the prior knowledge of the target object corresponding to the original image, and / or the mechanical arm joint position information corresponding to the original image.

[0038] In S102, based on the first feature vector corresponding to each original image in the action image sequence, a first feature vector sequence corresponding to the action image sequence is determined; based on the second feature vector of the target mask image corresponding to each original image in the action image sequence, a second feature vector sequence corresponding to the action image sequence is determined; the robotic arm joint position information corresponding to the acquisition time of each original image in the action image sequence is obtained, and the robotic arm joint position vector corresponding to the robotic arm joint position information is determined as the robotic arm joint position vector corresponding to the original image; based on the robotic arm joint position vector corresponding to each original image in the action image sequence, a robotic arm joint position vector sequence corresponding to the action image sequence is determined.

[0039] The target object is used to indicate the operated object that the two arms of the humanoid robot collaboratively grasp when the two arms perform the target task.

[0040] In S103, the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variables corresponding to the action image sequence are used as inputs of the Transformer encoder, and are processed based on four self-attention modules to generate a feature key-value pair sequence.

[0041] The target style variable is used to indicate the unique features of the target object being manipulated. The use of the target style variable helps the humanoid robot dual-arm collaborative end-to-end motion control model based on imitation learning to learn more information related to the target object and the robot arm joint movements, thereby improving the success rate of the humanoid robot in performing the target task.

[0042] In S104, the humanoid robot body position vector is generated based on the humanoid robot body position information. The humanoid robot body position information is used to indicate the position information of the humanoid robot body relative to the robot arm base.

[0043] The feature key-value pair sequence and the humanoid robot body position vector are used as training samples, and the training samples are used as the input of the Transformer decoder. They are processed using 7 cross-attention modules, and downsampled and projected to k*N (where N is the sum of the number of degrees of freedom of the humanoid robot's arms) by a multi-layer perceptron, corresponding to the target joint position within the next preset time adjacent to the current moment; the target joint position is compared with the target joint position label to generate an L1 loss function. The above operations are performed based on several training samples; when the L1 loss function tends to be minimized, model parameters are generated; the model is adjusted based on the model parameters to generate a motion control model for dual-arm coordination.

[0044] This embodiment uses the action image, target mask image, and mechanical arm joint position information corresponding to the same moment in the process of the humanoid robot's dual arms performing the target task as multi-source input for imitation learning; thereby, it can enhance the recognition and positioning capabilities of the humanoid robot's dual arms for the target object, thereby improving the control accuracy of the humanoid robot's dual arms in coordination. This embodiment applies the Transformer architecture to the imitation learning of the humanoid robot's dual arms in coordination, effectively enhancing the humanoid robot's dual arms' coordination capabilities and autonomous task completion capabilities, so that the humanoid robot's dual arms can complete more complex long action sequence tasks, effectively improving the success rate of the humanoid robot in performing tasks.

[0045] In addition, this embodiment uses an imitation learning strategy based on the Transformer encoding and decoding structure to achieve end-to-end motion control of the humanoid robot's dual arms. The self-attention structure in the encoder can perform high-dimensional mapping of the feature information of multi-source data, and the cross-attention module and L1 loss function in the decoder can accurately model the predicted action sequence of the humanoid robot's dual arms, which can improve the operation accuracy when the dual arms perform tasks in collaboration, thereby improving the overall task execution success rate.

[0046] In a preferred implementation of this embodiment, the method also includes: a prediction stage; obtaining a test image sequence generated when the humanoid robot's arms perform a target task; using the motion control model to perform prediction processing on the test image sequence, and outputting the predicted action of the humanoid robot.

[0047] Here, the image sequence to be tested is used to indicate a set of images to be tested corresponding to each moment within a preset step time including the current moment and before the current moment, which are arranged in time sequence.

[0048] The predicted action of the humanoid robot is used to indicate the target joint position of the humanoid robot corresponding to the next preset step time adjacent to the current moment.

[0049] Therefore, this embodiment accurately predicts the humanoid robot's movements within the next preset step time based on the motion control model, thereby effectively enhancing the coordination ability and autonomous task completion ability of the humanoid robot's arms in the process of performing the target task, allowing the humanoid robot's arms to complete more complex long motion sequence tasks and effectively improve the success rate of task execution.

[0050] In a preferred implementation manner of this embodiment, the determination of the robotic arm joint position vector sequence corresponding to the action image sequence includes: collecting the robotic arm joint position information at different moments during the process of the humanoid robot's two arms performing the target task to obtain a robotic arm joint position sequence; time-aligning the robotic arm joint position sequence with the action image sequence to obtain a robotic arm joint position sequence corresponding to the action image sequence; encoding the robotic arm joint position sequence to output a robotic arm joint position vector sequence.

[0051] Specifically, the acquisition time of each original image in the action image sequence is obtained to generate an acquisition time sequence; for any acquisition time in the acquisition time sequence: the manipulator joint position information corresponding to the acquisition time is selected from the manipulator joint position sequence; the manipulator joint position information is projected into a manipulator joint position vector based on a linear encoder; based on the manipulator joint position vector corresponding to each acquisition time in the acquisition time sequence, a manipulator joint position vector sequence corresponding to the action image sequence is obtained.

[0052] This embodiment uses the robot arm joint position vector sequence as the input of the humanoid robot dual-arm imitation learning, which can improve the accuracy of the mapping learning between the original image and the robot arm movement, thereby improving the success rate of the humanoid robot dual-arm in performing the target task.

[0053] like Figure 2 FIG. 1 is a flow chart of generating an action image sequence in one embodiment of the present invention.

[0054] Generating an action image sequence includes at least the following steps:

[0055] S201, obtaining a sequence of original images corresponding to the process in which the humanoid robot's arms perform a target task;

[0056] S202, for any original image in the original image sequence: obtaining target object prior knowledge and robot arm joint position information corresponding to the original image; combining the original image, target object prior knowledge, and robot arm joint position information into action image knowledge data; wherein the robot arm joint position information corresponding to the original image is used to indicate the robot arm joint position coordinates collected at the time of collecting the original image;

[0057] S203, generating an action image knowledge sequence based on the action image knowledge data corresponding to each original image in the original image sequence;

[0058] S204, performing action block processing on the action image knowledge sequence to generate an action image sequence.

[0059] In S201, when the humanoid robot's arms perform the target task, the imitation learning method is used to perform the coordinated end-to-end motion control of the arms, and it is necessary to obtain the mechanical arm joint position information and the original image of the camera at each moment in the process of performing the target task. The joint module and camera in the humanoid robot system are used to publish the Topic containing the mechanical arm joint position information and the original image through the ROS system, and the topic is subscribed to by the python programming language to obtain the original image sequence and the mechanical arm joint position sequence.

[0060] In S202 and S203, according to the original image, the target object prior knowledge corresponding to the original image is generated based on the model or preset rules. Exemplarily, the target object detection is performed on the original image to generate a target detection frame and target object category knowledge; the original image is masked using the target detection frame to generate a mask image; the mask image and the target object category knowledge are used together as the target object prior knowledge. Here, the target category knowledge is used to indicate the category attribute of the target object being operated.

[0061] In S204, this embodiment does not impose any limitation on the action block processing, for example: the action block processing is performed on the action image knowledge sequence based on a model or a preset rule to generate an action image sequence.

[0062] Exemplarily, at each moment corresponding to the action image knowledge sequence, the action image knowledge sequence is divided into blocks according to a preset step size, and a number of action image blocks are output; wherein the action image blocks include a number of action image knowledge data; the several action image blocks are subjected to exponential weighting processing, and the block weight corresponding to each of the action image blocks is output; for any of the action image blocks: based on the block weight, the action image knowledge data in the action image block is weighted sampled, and the sampling result is output; based on the sampling result corresponding to each of the action image blocks, an action image sequence is generated.

[0063] Here, the preset step length is determined based on the movements of the two arms of the humanoid robot during the execution of the target task.

[0064] The algorithm of the present invention processes the action of the humanoid robot's arms when performing the target task according to a specific step length k, that is, an action observation is performed every k steps, the next k-step action is generated, and it is executed in sequence. At the same time, in order to avoid the unstable robot motion caused by the discrete action after the action is divided into blocks, a time set method is adopted to perform a block process at each moment, and an exponential weighting scheme is used to perform a weighted average on the prediction of the next moment. In this way, overlapping action image blocks are obtained, and there will be multiple predicted actions within a given time step.

[0065] Assume that the time sequence corresponding to the action image knowledge sequence is [1min, 2min, 3min, 4min, 5min, 6min, 7min, 8min, 9min, 10min], and form an action image sequence based on the original image corresponding to each moment in the time sequence. If the action image sequence is divided into blocks according to 3 time steps at each moment in the time sequence, 8 action image blocks are output; each action image block includes three original images. The block weight of each action image block is determined based on the exponential weighting method. Since each original image in the action image block has a corresponding image weight, the block weight and the image weight are multiplied to obtain the acquisition weight corresponding to each original image. Then, the original image with an acquisition weight greater than the preset threshold is selected from each action image block as the acquisition result; based on the original image corresponding to the acquisition result of each action image block in several action image blocks, an action image sequence is generated.

[0066] This embodiment uses motion block processing and time aggregation to perform continuous and smooth block division and exponential weighted prediction of the humanoid robot's dual-arm movements, thereby enhancing the smoothness of the robot's dual-arm predicted movements and avoiding discrete switching of dual-arm movements due to discontinuous blocking, thereby making the movement of the robot's dual-arms more stable.

[0067] like Figure 3 FIG. 1 is a flow chart of generating target style variables in one embodiment of the present invention.

[0068] Generating the target style variable includes at least the following steps:

[0069] S301, generating a target object category sequence based on target object category knowledge corresponding to each of the original images in the action image sequence;

[0070] S302, encoding the action image sequence and the target object category knowledge sequence respectively, and outputting a target object category knowledge vector sequence and an action vector sequence;

[0071] S303, using a transformer encoder to encode the robot arm joint position vector sequence, the target object category knowledge vector sequence and the action vector sequence respectively to generate a target style variable.

[0072] Specifically, the robot arm joint position vector sequence, the target object category knowledge vector sequence, and the action vector sequence are used as the input of the Transformer encoder, and 4 self-attention modules and linear layers are used to obtain the target style variable Z (mean, std), where mean is the mean and std is the variance.

[0073] It should be noted that the target object category knowledge vector sequence, the action vector sequence, and the robot arm joint position vector sequence have a corresponding relationship in time.

[0074] Figure 4 The figure is a schematic diagram of a process of generating a first eigenvector and a second eigenvector in one embodiment of the present invention.

[0075] S401, performing feature extraction processing on the original image and the target mask image corresponding to the original image, and outputting the original image features and the target mask image features;

[0076] S402, performing feature mapping processing on the original image features and the target mask image features to generate a fused image and mapping features;

[0077] S403, determining position information of the mapping feature in the fused image;

[0078] S404, using a linear layer pair to process the original image features and the target mask image features respectively, and output an original image feature vector and a target mask image feature vector;

[0079] S405, adding the position information to the original image feature vector to generate a first feature vector corresponding to the original image;

[0080] S406, adding the position information to the target mask image feature vector to generate a second feature vector corresponding to the target mask image.

[0081] Specifically, the original image and the target mask image are normalized by using a convolutional neural network, and the normalized original image and the normalized target mask image are output; the deep features of the normalized original image and the normalized target mask image are extracted, and the original image features and the target mask image features are output; wherein the original image features contain the global information in the target task scene, and the target mask image features contain the deeper features of the operated target object. Feature mapping is performed on these two features, and the features are projected into embedded vectors through a linear layer, and sinusoidal position information is added to the feature mapping embedded vector to save the feature space position information, and finally the first feature vector corresponding to the original image and the second feature vector corresponding to the target mask image are obtained, thereby improving the accuracy of the motion control model in predicting the dual-arm action.

[0082] A control method for dual-arm coordination of a humanoid robot provided by this embodiment will be described in detail below in conjunction with a specific application scenario.

[0083] A control method for the coordination of two arms of a humanoid robot comprises at least the following steps:

[0084] S1, obtain the original image sequence corresponding to the process of the humanoid robot's dual arms performing the target task.

[0085] S2, for any original image in the original image sequence: perform target object detection on the original image to generate a target detection frame and target object category knowledge; use the target detection frame to mask the original image to generate a mask image; use the mask image and the target object category knowledge together as the target object prior knowledge corresponding to the original image; obtain the robot arm joint position information corresponding to the original image; combine the original image, the target object prior knowledge, and the robot arm joint position information into action image knowledge data; wherein the robot arm joint position information corresponding to the original image is used to indicate the robot arm joint position coordinates collected at the time of collecting the original image.

[0086] S3, generating an action image knowledge sequence based on the action image knowledge data corresponding to each original image in the original image sequence.

[0087] S4, at each moment corresponding to the action image knowledge sequence, the action image knowledge sequence is processed into blocks according to a preset step size, and a number of action image blocks are output; wherein the action image blocks include a number of action image knowledge data; exponential weighting is performed on a number of the action image blocks, and a block weight corresponding to each action image block is output; for any action image block: based on the block weight, weighted sampling is performed on the action image knowledge data in the action image block, and a sampling result is output; based on the sampling result corresponding to each action image block, an action image sequence is generated.

[0088] S5, collecting the manipulator arm joint position information at different moments during the process of the humanoid robot's two arms performing the target task, and obtaining the manipulator arm joint position sequence; time-aligning the manipulator arm joint position sequence with the action image sequence, and obtaining the manipulator arm joint position sequence corresponding to the action image sequence; encoding the manipulator arm joint position sequence, and outputting the manipulator arm joint position vector sequence.

[0089] S6, based on the target object category knowledge corresponding to each original image in the action image sequence, generate a target object category knowledge sequence; encode the action image sequence and the target object category knowledge sequence respectively, and output a target object category knowledge vector sequence and an action vector sequence; use a transformer encoder to encode the robot arm joint position vector sequence, the target object category knowledge vector sequence and the action vector sequence respectively to generate a target style variable.

[0090] S7, for any original image in the action image sequence: perform feature extraction processing on the original image and the target mask image corresponding to the original image, and output the original image features and the target mask image features; perform feature mapping processing on the original image features and the target mask image features, and generate a fused image and mapping features; determine the position information of the mapping features in the fused image; use a linear layer to process the original image features and the target mask image features, and output the original image feature vector and the target mask image feature vector; add the position information to the original image feature vector to generate the first feature vector corresponding to the original image; add the position information to the target mask image feature vector to generate the second feature vector corresponding to the target mask image. Based on the first feature vector corresponding to each original image in the action image sequence, generate a first feature vector sequence; based on the second feature vector corresponding to the target mask image of each original image in the action image sequence, generate a second feature vector sequence.

[0091] S8, using the Transformer encoder to encode the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variables corresponding to the action image sequence, and output a feature key-value pair sequence.

[0092] S9, taking the feature key-value pair sequence and the humanoid robot body position vector as training samples; performing supervised imitation learning on several training samples based on the Transformer decoder to generate a motion control model for dual-arm collaboration.

[0093] S10, obtaining a sequence of images to be tested generated when the two arms of the humanoid robot perform a target task; using a motion control model for dual-arm collaboration to perform prediction processing on the sequence of images to be tested, and outputting a predicted action of the humanoid robot.

[0094] This embodiment is aimed at the humanoid robot dual-arm collaborative end-to-end motion control algorithm, using the original image, target object prior knowledge, and mechanical arm joint position information as the main training data of the imitation learning algorithm based on the Transformer architecture. On the basis of the original image data features, the mask image, target object category knowledge and mechanical arm joint position information are used to strengthen the recognition and positioning capabilities of the mechanical arm imitation learning, thereby improving the accuracy of the mechanical arm motion control. The target object category knowledge is conducive to the mechanical arm motion control algorithm to autonomously predict the execution of operations such as target grabbing in an unseen target state. The target mask image can enhance the mechanical arm's perception of the operated object, thereby achieving more accurate recognition and positioning. Therefore, the control method for the humanoid robot dual-arm collaboration in this embodiment can effectively enhance the humanoid robot's dual-arm collaboration ability and autonomous task completion ability, so that the humanoid robot dual-arm can complete more complex long action sequence tasks, effectively improving the success rate of target task execution.

[0095] like Figure 5 , which is a schematic diagram of the structure of a control device for dual-arm coordination of a humanoid robot provided by an embodiment of the present invention.

[0096] A control device for the coordination of two arms of a humanoid robot, the device 500 comprising: a first acquisition module 501, used to acquire an action image sequence generated during the execution of a target task by two arms of the humanoid robot; wherein the action image sequence is used to indicate a set formed by arranging a plurality of original images in chronological order; a first determination module 502, used to determine a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of a mask image sequence corresponding to the action image sequence, and a robot arm joint position vector sequence corresponding to the action image sequence; wherein the mask image sequence comprises a plurality of target mask images arranged in chronological order, the target mask images being the first feature vector sequence corresponding to the action image sequence, and the second feature vector sequence of the mask image sequence corresponding to the action image sequence. The marked mask image is used to indicate the image after the target object in the original image is masked; the first encoding module 503 is used to use the Transformer encoder to encode the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variables corresponding to the action image sequence, and output a feature key-value pair sequence; the first generation module 504 is used to use the feature key-value pair sequence and the humanoid robot body position vector as training samples; based on the Transformer decoder, supervised imitation learning is performed on several of the training samples to generate a motion control model for dual-arm collaboration.

[0097] In a preferred implementation manner of this embodiment, the first determination module includes: a first determination unit, which is used to: for any original image in the action image sequence: normalize the original image and the target mask image corresponding to the original image respectively, and output the normalized original image and the normalized target mask image; perform deep feature extraction on the normalized original image and the normalized target mask image respectively to generate original image features and target mask image features; perform feature mapping processing on the original image features and the target mask image features to generate a fused image and mapping features; determine the position information of the mapping features in the fused image; use a linear layer to respectively map the original image features and the target mask image features; The original image features and the target mask image features are processed to output an original image feature vector and a target mask image feature vector; the position information is added to the original image feature vector to generate a first feature vector corresponding to the original image; the position information is added to the target mask image feature vector to generate a second feature vector corresponding to the target mask image; a second determining unit is used to generate a first feature vector sequence based on the first feature vector corresponding to each of the original images in the action image sequence; a third determining unit is used to generate a second feature vector sequence based on the second feature vector corresponding to the target mask image of each of the original images in the action image sequence.

[0098] In a preferred implementation of this embodiment, the first determination module also includes: an acquisition unit, used to acquire the mechanical arm joint position information at different times during the execution of the target task by the two arms of the humanoid robot, and obtain a mechanical arm joint position sequence; a time alignment unit, used to time-align the mechanical arm joint position sequence with the action image sequence, and obtain a mechanical arm joint position sequence corresponding to the action image sequence; an encoding processing unit, used to encode the mechanical arm joint position sequence and output a mechanical arm joint position vector sequence.

[0099] In a preferred implementation manner of this embodiment, the device also includes: a second acquisition module, used to acquire the original image sequence corresponding to the process of the humanoid robot's two arms performing the target task; a second determination module, used to, for any original image in the original image sequence: acquire the target object prior knowledge and the robotic arm joint position information corresponding to the original image; and combine the original image, the target object prior knowledge, and the robotic arm joint position information into action image knowledge data; wherein the robotic arm joint position information corresponding to the original image is used to indicate the robotic arm joint position coordinates collected at the time of collecting the original image; a second generation module, further used to generate an action image knowledge sequence based on the action image knowledge data corresponding to each of the original images in the original image sequence; a block processing module, used to perform action block processing on the action image knowledge sequence to generate an action image sequence.

[0100] In a preferred implementation of this embodiment, the device also includes: a target detection module, which is used to perform target object detection on the original image and generate a target detection frame and target object category knowledge; a mask processing module, which is used to perform mask processing on the original image using the target detection frame and generate a mask image; and a third determination module, which is used to use the mask image and target object category knowledge together as target object prior knowledge.

[0101] In a preferred implementation manner of this embodiment, the block processing module includes: a block processing unit, which is used to perform block processing on the action image knowledge sequence according to a preset step size at each moment corresponding to the action image knowledge sequence, and output a plurality of action image blocks; wherein the action image blocks include a plurality of action image knowledge data; an exponential weighted processing unit, which is used to perform exponential weighted processing on the plurality of action image blocks, and output a block weight corresponding to each of the action image blocks; a weighted sampling unit, which is used to perform weighted sampling on the action image knowledge data in the action image block based on the block weight, and output a sampling result for any of the action image blocks; and a generation unit, which is used to generate an action image sequence based on the sampling result corresponding to each of the action image blocks.

[0102] In a preferred implementation manner of this embodiment, the device also includes: a third generation module, used to generate a target object category sequence based on the target object category knowledge corresponding to each of the original images in the action image sequence; a second encoding processing module, used to encode the action image sequence and the target object category knowledge sequence, respectively, and output a target object category knowledge vector sequence and an action vector sequence; a third encoding processing module, used to use a transformer encoder to encode the robotic arm joint position vector sequence, the target object category knowledge vector sequence and the action vector sequence, respectively, to generate a target style variable.

[0103] In a preferred implementation of this embodiment, the device also includes: a third acquisition module, used to acquire the test image sequence generated when the humanoid robot's arms perform the target task; a prediction processing module, used to use the motion control model to perform prediction processing on the test image sequence and output the predicted action of the humanoid robot.

[0104] The above device can execute a control method for the coordination of two arms of a humanoid robot provided by an embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing a control method for the coordination of two arms of a humanoid robot. For technical details not described in detail in this embodiment, please refer to a control method for the coordination of two arms of a humanoid robot provided by an embodiment of the present invention.

[0105] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a control method for dual-arm coordination of a humanoid robot described in the present invention.

[0106] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0107] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0108] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to the following embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0109] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0110] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0111] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0112] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0113] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0114] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0115] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0116] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0117] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A control method for the coordination of two arms of a humanoid robot, characterized in that: include: Acquire a sequence of action images generated when the humanoid robot's arms perform a target task; wherein the sequence of action images is used to indicate a set of several original images arranged in time sequence; Determine a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of a mask image sequence corresponding to the action image sequence, and a robot arm joint position vector sequence corresponding to the action image sequence; wherein the mask image sequence includes a plurality of target mask images arranged in chronological order, and the target mask image is used to indicate an image after masking the target object in the original image; Using a Transformer encoder to encode the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variables corresponding to the action image sequence, and output a feature key-value pair sequence; The feature key-value pair sequence and the humanoid robot body position vector are used together as training samples; supervised imitation learning is performed on several of the training samples based on the Transformer decoder to generate a motion control model for dual-arm collaboration.

2. The method according to claim 1, characterized in that The step of determining a first feature vector sequence corresponding to the action image sequence and a second feature vector sequence of a mask image sequence corresponding to the action image sequence comprises: For any original image in the action image sequence: perform feature extraction processing on the original image and the target mask image corresponding to the original image respectively, and output the original image features and the target mask image features; perform feature mapping processing on the original image features and the target mask image features to generate a fused image and mapping features; determine the position information of the mapping features in the fused image; use linear layers to process the original image features and the target mask image features respectively, and output the original image feature vector and the target mask image feature vector; add the position information to the original image feature vector to generate a first feature vector corresponding to the original image; add the position information to the target mask image feature vector to generate a second feature vector corresponding to the target mask image; Generate a first feature vector sequence based on a first feature vector corresponding to each of the original images in the action image sequence; A second feature vector sequence is generated based on the second feature vector corresponding to the target mask image of each original image in the action image sequence.

3. The method according to claim 1, characterized in that The step of determining the robot arm joint position vector sequence corresponding to the action image sequence comprises: Collecting the mechanical arm joint position information at different moments during the process of the two arms of the humanoid robot performing the target task, and obtaining the mechanical arm joint position sequence; Temporally aligning the robot arm joint position sequence with the action image sequence to obtain a robot arm joint position sequence corresponding to the action image sequence; The robot arm joint position sequence is encoded and a robot arm joint position vector sequence is output.

4. The method according to claim 1, characterized in that: Also includes: Acquire a sequence of original images corresponding to the process in which the humanoid robot's two arms perform a target task; For any original image in the original image sequence: obtaining target object prior knowledge and robot arm joint position information corresponding to the original image; combining the original image, target object prior knowledge, and robot arm joint position information into action image knowledge data; wherein the robot arm joint position information corresponding to the original image is used to indicate the robot arm joint position coordinates collected at the time of collecting the original image; generating an action image knowledge sequence based on the action image knowledge data corresponding to each of the original images in the original image sequence; The action image knowledge sequence is subjected to action block processing to generate an action image sequence.

5. The method according to claim 4, characterized in that Also includes: Performing target object detection on the original image to generate target detection frames and target object category knowledge; Using the target detection frame to perform mask processing on the original image to generate a mask image; The mask image and target object category knowledge are used together as target object prior knowledge.

6. The method according to claim 4, characterized in that The step of performing action block processing on the action image knowledge sequence to generate an action image sequence comprises: At each moment corresponding to the action image knowledge sequence, the action image knowledge sequence is processed into blocks according to a preset step length, and a plurality of action image blocks are output; wherein the action image blocks include a plurality of action image knowledge data; Performing exponential weighting processing on the plurality of action image blocks, and outputting a block weight corresponding to each action image block; For any of the action image blocks: based on the block weights, weighted sampling is performed on the action image knowledge data in the action image block, and a sampling result is output; Based on the sampling results corresponding to each of the action image blocks, an action image sequence is generated.

7. The method according to claim 1, characterized in that Also includes: generating a target object category sequence based on target object category knowledge corresponding to each of the original images in the action image sequence; Encoding the action image sequence and the target object category knowledge sequence respectively, and outputting a target object category knowledge vector sequence and an action vector sequence; The robot arm joint position vector sequence, the target object category knowledge vector sequence and the action vector sequence are respectively encoded using a transformer encoder to generate a target style variable.

8. The method according to claim 1, characterized in that Also includes: Acquire a sequence of images to be tested generated when the humanoid robot's arms perform a target task; The motion control model is used to perform prediction processing on the image sequence to be tested, and the predicted action of the humanoid robot is output.

9. A control device for the coordination of two arms of a humanoid robot, characterized in that: include: A first acquisition module is used to acquire a sequence of action images generated when the humanoid robot's arms perform a target task; wherein the sequence of action images is used to indicate a set of several original images arranged in time sequence; A first determination module is used to determine a first feature vector sequence corresponding to the action image sequence, a second feature vector sequence of a mask image sequence corresponding to the action image sequence, and a robot arm joint position vector sequence corresponding to the action image sequence; wherein the mask image sequence includes a plurality of target mask images arranged in chronological order, and the target mask image is used to indicate an image after mask processing is performed on a target object in the original image; A first encoding module is used to encode the first feature vector sequence, the second feature vector sequence, the robot arm joint position vector sequence, and the target style variable corresponding to the action image sequence using a Transformer encoder, and output a feature key-value pair sequence; The first generation module is used to use the feature key-value pair sequence and the humanoid robot body position vector as training samples; based on the Transformer decoder, supervised imitation learning is performed on several of the training samples to generate a motion control model for dual-arm collaboration.

10. A computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images