Robot action generation model training method and action generation method

Abstract

The invention relates to a robot action generation model training method and an action generation method, and the model training comprises the steps: constructing an automobile wire harness operation data set, and extracting a human body posture; constructing an instruction set based on the data set and the human body posture; pre-training a motion word segmentation device; obtaining a text-action vocabulary; pre-training a language model based on the data set, the instruction set and the text-action vocabulary; and constructing an operation data set, and performing fine tuning on the pre-trained language model based on the operation data set to complete training. According to the generation method, a humanoid action sequence is output through a model trained through a model training method, and the humanoid action sequence is redirected to a robot to complete robot action generation. Compared with the prior art, the generality of the generation model is improved, the robot action more conforming to the automobile wire harness operation can be generated, and the flexibility of the robot for operating the automobile wire harness is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of humanoid robots, and in particular to a robot action generation model training method and an action generation method. Background Art

[0002] At present, wiring harnesses are crucial in modern cars, especially electric cars, which are used to transmit power and signals to achieve key functions. With the rapid development of smart cars, the number of wiring harnesses in multiple parts such as car doors, engines and vehicle bodies has increased exponentially. However, the current assembly of automotive wiring harnesses relies on a large amount of manual labor, which puts great pressure on both operators and factories. An obvious solution is to reduce labor demand and improve productivity through robotic automation. Deformable automotive wiring harnesses require robots to be dexterous and intelligent, so it is crucial to realize the intelligent and dexterous assembly of wiring harnesses by robots.

[0003] Generative methods have achieved remarkable results in the field of robot skill learning, and have been proven to be effective in improving robot operations on a variety of physical robotic arms. China's patent application "CN106600000A" discloses a method for mapping human-robot motion data, which uses human motion data as the input of a deep learning model and robot sample data as the expected output of the deep learning model. Although the mapping relationship between human-robot motion data is achieved, this method does not combine text information to guide robot motion generation, is not suitable for automotive wiring harness assembly, and has low generalization.

[0004] Therefore, it is a problem that needs to be solved to provide a robot motion generation method that can be applied to automobile wiring harness assembly and has generalization. Summary of the invention

[0005] The purpose of the present invention is to provide a robot motion generation model training method and a motion generation method in order to overcome the defects of the above-mentioned prior art. On the basis of language-motion understanding, prompt data is constructed with human posture, which effectively improves the accuracy of robot motion generation.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to a first aspect of the present invention, a method for training a robot motion generation model is provided, wherein the robot motion generation model comprises a motion segmenter, a language model module and a vocabulary integration module, and the steps include:

[0008] Constructing an automotive wiring harness operation data set, the data set includes an action data set and a text data set when a worker operates the wiring harness, and extracting human body posture based on the action data;

[0009] Building an instruction set based on the text data set, the action data set and the human body posture, wherein the data types of the instruction set include the function description, function label, function category, input and output of the language model;

[0010] Obtaining a text vocabulary in the language model module, pre-training the motion segmenter based on the text vocabulary and the motion data set, and generating a text vocabulary and an motion vocabulary containing motion semantics;

[0011] Based on the text vocabulary, the text vocabulary containing action semantics and the action vocabulary, the vocabulary integration module is used to integrate and obtain a text-action vocabulary;

[0012] Pre-training a language model based on the dataset, instruction set, and text-action vocabulary;

[0013] An operation data set is constructed, and a pre-trained language model is fine-tuned based on the operation data set to complete the training. The data types in the operation data set include human automobile wiring harness routing, wiring harness terminal plugging, and wiring harness winding operation data.

[0014] As a preferred technical solution, the human body posture is three-dimensional data.

[0015] As a preferred technical solution, the pre-training method of the motion word segmenter includes:

[0016] Randomly select multiple action sequences from the action data set as a training set;

[0017] The action sequence is expressed as Where F represents the F-frame action sequence, i represents the i-th action sequence, and x represents the action;

[0018] Discretize the action sequence into action discrete tokens of preset length Wherein, t represents a single data in a discrete word unit; i represents the i-th data in a discrete word unit; f represents a preset length and f=F / l, and l represents a sampling time;

[0019] Decode the action discrete words into action sequences Based on the decoded action sequence With action sequence m 1:F Calculate the loss;

[0020] The motion tokenizer is optimized based on the loss.

[0021] As a preferred technical solution, the method for generating the action vocabulary is: using a pre-trained motion segmenter to discretize each action sequence in the action data set into action discrete word units, and integrating all action discrete word units to obtain the action vocabulary.

[0022] As a preferred technical solution, the method for obtaining the text vocabulary containing action semantics is: randomly giving the motion segmenter a language text description containing time information, and repeatedly performing text-action matching based on the text vocabulary and the action vocabulary until the text sequence of the text vocabulary and the action sequence in the motion coding book maintain a consistent order in time.

[0023] As a preferred technical solution, the method for obtaining the text-action vocabulary is:

[0024] Obtaining the text vocabulary and all text discrete words in the text vocabulary containing action semantics;

[0025] Encode all the discrete words of the text into basic units;

[0026] All action discrete word units in the action vocabulary are obtained, and the action discrete word units and basic units are input into a vocabulary integration module for integration in chronological order.

[0027] As a preferred technical solution, the pre-training of the language model uses log-likelihood to calculate the pre-training loss, which is expressed as:

[0028]

[0029] in, Represents the i-th discrete word in the action sequence at time t; Represents the first i discrete words in the action sequence at time t; Lt Indicates the length of the current action sequence; represents the probability at time t.

[0030] According to a second aspect of the present invention, a robot action generation method is provided, the method using the robot action generation model trained by the above generation model training method to generate robot actions, comprising:

[0031] Obtaining the instruction of the operation harness and inputting it into the motion word segmenter, and outputting discrete action words;

[0032] Input the action word into the language model module, and output a 3D humanoid action sequence;

[0033] The robot motion is generated using a redirection method based on the 3D humanoid motion sequence.

[0034] As a preferred technical solution, the method for generating robot actions is:

[0035] The minimum distance between the current position of the robot's end effector and the target position is taken as the objective function, and its expression is:

[0036] F(θ)=||p target -p(θ)|| 2 ,

[0037] Among them, p target represents the target position, and p(θ) represents the position of the end effector at the current joint angle;

[0038] Obtaining the position of each end effector based on the objective function;

[0039] The positions of each end effector are integrated to generate the robot motion.

[0040] As a preferred technical solution, the method for obtaining the position of each joint key point based on the objective function is: using the gradient descent method to find the position of the end effector that makes the objective function the minimum value.

[0041] Compared with the prior art, the present invention has the following advantages:

[0042] 1) In the process of training the generation model, the present invention uses human posture as prompt data of the language model, and integrates the instructions including text to text, text to action and action to action in the process of generating robot actions by the action generation model to construct an instruction set. The robot action generation model is trained based on the instruction set and human posture, so that it can generate actions that are more in line with the automotive wiring harness based on understanding the action description grammar. The model training method provided by the present invention can ensure that the robot action generation model can effectively learn and generalize to a variety of automotive wiring harness related tasks;

[0043] 2) In the process of robot motion generation, the present invention maps the motion generated by the robot motion generation model to the humanoid robot in a redirected manner, which not only improves the accuracy of robot motion generation, but also generates richer automobile wiring harness operation motions and improves the flexibility of the robot in operating the automobile wiring harness;

[0044] 3) The present invention pre-trains the robot action generation model so that when operating the automobile wiring harness, it does not need to perform online matching and training for given task text instructions, which effectively saves time and operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a general flow chart of the model training and action generation method of the present invention;

[0046] Figure 2 A schematic diagram of the robot motion generation model training of the present invention;

[0047] Figure 3 A framework diagram is generated for the actions of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0049] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0050] The present invention relates to a humanoid robot motion generation model training method and a humanoid robot motion generation method for dexterous operation of automobile wiring harnesses. Aiming at the large labor demand for operation tasks in automobile wiring harness production lines, a humanoid robot motion generation model is constructed using Wensheng motions. Aiming at the motion requirements of the operation tasks, human posture data is used as a prompt to improve the motion generated by the model. The detailed process of the present invention is as follows Figure 1 As shown in the figure, it is divided into two modules: model training and action generation.

[0051] Example 1

[0052] This embodiment provides a method for training a robot action generation model, which uses human posture data in the process of automobile wiring harness operation as prompt data of the robot action generation model, combines text data and action data in the process of automobile wiring harness to build an instruction set, and pre-trains the robot action generation model based on the instruction set, human posture data, action data and text data, without the need for online matching and training for specific texts, saving model training time and training efficiency. Specifically, the robot action generation model includes a motion segmenter, a language model Llama module and a vocabulary integration module, wherein the vocabulary integration module can implement functions with the help of the SentencePiece model.

[0053] The framework of the training method of the robot action generation model is as follows Figure 2 As shown, the following steps are included:

[0054] S1. Arrange the automotive wiring harness operation scene and collect the action data set and text data set of workers when operating the wiring harness. The action data set and text data set are collectively referred to as the data set. Extract human posture based on the action data set as prompt data for the subsequent robot action generation model, and the human posture data is in a three-dimensional space format.

[0055] The method of obtaining motion data and text data includes at least a motion capture camera, an RGB camera and an IMU sensor, and the data types collected include at least: joint coordinates in three-dimensional directions, pitch-roll-rotation angles of joints, angular accelerations and a complete operation video of an operation task.

[0056] S2. Instruction set construction:

[0057] An instruction set is constructed based on text datasets, action datasets, and human postures, and saved in json file format. The data types of the instruction set include the function description of the language model Llama, which contains multiple action semantic text language descriptions, function labels such as "text-to-pose", "text-to-motion", and "text-to-text", function categories such as "t2p", "t2m", and "t2t", inputs, and outputs. The instruction set containing text-to-text, text-to-action, and action-to-action instructions is constructed to help the language model Llama better understand the text and generate corresponding humanoid action sequences.

[0058] During the pre-training process of the language model Llama, the input and output templates of different tasks are defined through the instruction set, providing the language model Llama with clear task structure and data format guidance. The model parses and processes different types of task data according to the content in the file, fills the actual data into the placeholder, learns the patterns and features of different tasks, and learns how to map from input to expected output. Therefore, the instruction set mainly plays the role of defining the task structure and standardizing the data format to ensure that the language model Llama can effectively learn and generalize to a variety of sports-related tasks.

[0059] Specifically, the code for constructing the instruction set is shown in Table 1:

[0060] Table 1 Instruction set construction code table

[0061]

[0062]

[0063] S3. Obtain a text vocabulary in the language model module, pre-train a motion segmenter based on the text vocabulary and the motion data set, the motion segmenter includes a motion decoder D and an motion encoder E, and generate a text vocabulary and an action vocabulary containing action semantics.

[0064] The specific steps are:

[0065] S31, randomly selecting multiple action sequences in the action data set as a training set;

[0066] S32, the action sequence is expressed as Where F represents the F-frame action sequence, i represents the i-th action sequence, and x represents the action;

[0067] S33, using the action encoder to discretize the action sequence into action discrete words of preset length Wherein, t represents a single data in a discrete word unit; i represents the i-th data in a discrete word unit; f represents a preset length and f=F / l, and l represents a sampling time;

[0068] S34, decoding the action discrete word into an action sequence using an action decoder Based on the decoded action sequence With action sequence m 1:F Calculate the loss;

[0069] S35. Loss-based optimization of motion segmenter.

[0070] S4. Based on the text vocabulary, the text vocabulary containing action semantics, and the action vocabulary, the SentencePiece module is used to integrate them to obtain the text-action vocabulary.

[0071] S41. Generate an action vocabulary: Use a pre-trained motion tokenizer to discretize each action sequence in the action data set into action discrete tokens, and integrate all action discrete tokens to obtain the action vocabulary.

[0072] S42. Obtain a text vocabulary containing action semantics: randomly give the motion segmenter VQ-VAE a language text description containing time information and of length N, and repeat text-action matching based on the text vocabulary and the action vocabulary until the text sequence in the text vocabulary and the action sequence in the motion coding book maintain a consistent temporal order.

[0073] S43, constructing a text-action vocabulary: obtaining all text discrete words in the text vocabulary and the text vocabulary containing action semantics, encoding all text discrete words into basic units; obtaining all action discrete words in the action vocabulary, and inputting the action discrete words and basic units into the SentencePiece module for integration in chronological order, and in the integration process, the chronological order of the text and the action needs to be kept consistent.

[0074] S5. Pre-trained language model.

[0075] The main purpose of language model-based action generation is to learn the semantic coupling between action and text. Therefore, its pre-training is to pre-train the language model based on the data set, instruction set and text-action vocabulary obtained in steps S1-S4. In order to achieve better performance in the generation task and improve the language model's understanding and generation of results similar to the samples in the training data set, the probability of predicting the next token is measured using the log-likelihood, and it is converted into a probability in the form of a transformation product and presented in the form of a logarithmic sum, so as to process long sequence data such as action sequences, avoid numerical overflow caused by the exponential reduction of the probability product as the sequence length decreases, and facilitate the optimization of model parameters by maximizing the log-likelihood, and easy to calculate the gradient. The model parameters are adjusted by the calculated gradient so that the model output probability distribution is as consistent as possible with the actual data distribution, that is, the joint distribution of the text-action sequence in the text-action vocabulary, so as to better understand and generate text similar to the training data samples.

[0076] Specifically, the log-likelihood is used to calculate the pre-training loss, which is expressed as:

[0077]

[0078] in, Represents the i-th discrete word in the action sequence at time t; Represents the first i discrete words in the action sequence at time t; Lt Indicates the length of the current action sequence; represents the probability at time t.

[0079] S6. Construct an operation dataset including human automobile wiring harness routing, wiring harness terminal plugging and wiring harness winding operation data, and fine-tune the pre-trained language model based on the operation dataset to enhance the performance of the model in related tasks and complete the training.

[0080] Example 2

[0081] This embodiment provides a robot action generation method, which generates robot actions using a robot action generation model trained by the model training method provided by the above embodiment. The process is as follows: Figure 3 ,include:

[0082] A1. Get the command of the operation harness and input it into the motion word segmenter, which outputs discrete action words.

[0083] A2. Input the action word into the language model module and output the 3D humanoid action sequence.

[0084] A3. Generate robot motions using the redirection method based on 3D humanoid motion sequences.

[0085] A31. The minimum distance between the current position of the robot's end effector and the target position is taken as the objective function, and its expression is:

[0086] F(θ)=||p target -p(θ)||2,

[0087] Among them, p t arg et represents the target position, and p(θ) represents the position of the end effector at the current joint angle;

[0088] A32. Based on the objective function, use the gradient descent method to find the position of the end effector corresponding to the minimum objective function of each end effector;

[0089] A33. Integrate the position of each end effector to generate robot motion.

[0090] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A robot action generation model training method, characterized in that: The robot motion generation model includes a motion segmenter, a language model module and a vocabulary integration module, and the steps include: Constructing an automotive wiring harness operation data set, the data set includes an action data set and a text data set when a worker operates the wiring harness, and extracting human body posture based on the action data; Building an instruction set based on the text data set, the action data set and the human body posture, wherein the data types of the instruction set include the function description, function label, function category, input and output of the language model; Obtaining a text vocabulary in the language model module, pre-training the motion segmenter based on the text vocabulary and the motion data set, and generating a text vocabulary and an motion vocabulary containing motion semantics; Based on the text vocabulary, the text vocabulary containing action semantics and the action vocabulary, the vocabulary integration module is used to integrate and obtain a text-action vocabulary; Pre-training a language model based on the dataset, instruction set, and text-action vocabulary; An operation data set is constructed, and a pre-trained language model is fine-tuned based on the operation data set to complete the training. The data types in the operation data set include human automobile wiring harness routing, wiring harness terminal plugging, and wiring harness winding operation data.

2. A robot action generation model training method according to claim 1, characterized in that: The human body posture is three-dimensional data.

3. A robot action generation model training method according to claim 1, characterized in that: The pre-training method of the motion word segmenter includes: Randomly select multiple action sequences from the action data set as a training set; The action sequence is expressed as Where F represents the F-frame action sequence, i represents the i-th action sequence, and x represents the action; Discretize the action sequence into action discrete tokens of preset length Wherein, t represents a single data in a discrete word unit; i represents the i-th data in a discrete word unit; f represents a preset length and f=F / l, and l represents a sampling time; Decode the action discrete words into action sequences Based on the decoded action sequence With action sequence m 1:F Calculate the loss; The motion tokenizer is optimized based on the loss.

4. A robot action generation model training method according to claim 1, characterized in that: The method for generating the action vocabulary is: using a pre-trained motion segmenter to discretize each action sequence in the action data set into action discrete word units, and integrating all action discrete word units to obtain the action vocabulary.

5. A robot action generation model training method according to claim 1, characterized in that: The method for obtaining the text vocabulary containing action semantics is: randomly giving the language text description containing time information of the motion segmenter, and repeatedly performing text-action matching based on the text vocabulary and the action vocabulary until the text sequence of the text vocabulary and the action sequence in the motion coding book maintain a consistent order in time.

6. A robot action generation model training method according to claim 5, characterized in that: The method for obtaining the text-action vocabulary is: Obtaining the text vocabulary and all text discrete words in the text vocabulary containing action semantics; Encode all the discrete words of the text into basic units; All action discrete words in the action vocabulary are obtained, and the action discrete words and basic units are input into a vocabulary integration module for integration in chronological order.

7. A robot action generation model training method according to claim 1, characterized in that: In the pre-training of the language model, the log-likelihood is used to calculate the pre-training loss, and its expression is: in, Represents the i-th discrete word in the action sequence at time t; represents the first i discrete words in the action sequence at time t; L t Indicates the length of the current action sequence; represents the probability at time t.

8. A robot motion generation method, characterized in that: The method generates robot motions using a robot motion generation model trained by the method according to any one of claims 1 to 7, comprising: Obtaining the instruction of the operation harness and inputting it into the motion word segmenter, and outputting discrete action words; Input the action word into the language model module, and output a 3D humanoid action sequence; The robot motion is generated using a redirection method based on the 3D humanoid motion sequence.

9. A robot motion generation method according to claim 8, characterized in that: The method for generating robot actions is: The distance between the current position of the robot's end effector and the target position is taken as the objective function, and its expression is: F(θ)=||p target -p(θ)|| 2 , Among them, p target represents the target position, and p(θ) represents the position of the end effector at the current joint angle; Obtaining the position of each end effector based on the objective function; The positions of each end effector are integrated to generate the robot motion.

10. A robot motion generation method according to claim 9, characterized in that: The method for obtaining the position of each end effector based on the objective function is: using the gradient descent method to find the position of the end effector that makes the objective function the minimum value.

Images