A method, system and medium for generating teaching actions of an assisted embodied intelligent robot

Abstract

The present invention discloses a method, system, and medium for generating teaching actions for an embodied intelligent robot. The teaching action generation method comprises: obtaining a first data set and a second data set; obtaining an action generation model based on the first and second data sets; obtaining a teaching action to be processed, and generating a target teaching action based on the teaching action to be processed using the action generation model. The present invention uses a deep learning method to train the action generation model, and then uses the action generation model to generate standardized target teaching actions. This method can effectively improve the quality of the teaching action dataset, thereby enhancing the training effect of the imitation learning model, and can be widely applied in the field of artificial intelligence technology.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, system, and medium for generating teaching actions to assist embodied intelligent robots. Background Art

[0002] When an embodied intelligent robot learns human actions in a specific scenario, such as learning to pick up an apple in an office setting, there are two main technical methods for the robot to learn: reinforcement learning in a simulation environment and imitation learning in a real scenario.

[0003] Current imitation learning methods typically collect teaching motion data through multiple expert demonstrations to gather sufficient valid data as a teaching dataset. However, these expert demonstrations can suffer from issues such as lag, jitter, and irregular movements. In practical projects, experts are often instructed to minimize these lags and jitters, but this issue is not completely eliminated. This reduces the quality of the collected teaching data, impacting the effectiveness of imitation learning model training. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a method, system and medium for generating teaching actions for an assisted embodied intelligent robot with high data set quality and good training effect.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application provides a method for generating teaching actions for an embodied intelligent robot, comprising the following steps:

[0006] Obtain a first data set and a second data set;

[0007] Obtaining an action generation model based on the first data set and the second data set;

[0008] Acquire a teaching action to be processed, and generate a target teaching action according to the teaching action to be processed using the action generation model;

[0009] The first data set includes a plurality of first teaching action samples that meet preset specifications, and the second data set includes a plurality of second teaching action samples that do not meet the preset specifications. Obtaining the action generation model based on the first data set and the second data set specifically includes:

[0010] Setting each of the first teaching motion samples and each of the second teaching motion samples as a two-dimensional matrix of motion dimension multiplied by motion length;

[0011] Setting a target action length value, and when the action length in the two-dimensional matrix is ​​less than the target action length value, padding a corresponding position in the two-dimensional matrix with a zero element to obtain the preprocessed first data set and the second data set;

[0012] Constructing a first neural network model, inputting the preprocessed first data set and the second data set into the first neural network model to perform action specification judgment, and outputting corresponding binary classification judgment results;

[0013] Optimizing the parameters of the first neural network model according to the binary classification result to obtain a trained action judgment model;

[0014] The action generation model is trained based on the first data set, the second data set and the action judgment model.

[0015] In some embodiments, obtaining the first data set and the second data set specifically includes:

[0016] Setting a teaching action sample, and breaking the teaching action sample into a plurality of teaching actions;

[0017] The first data set and the second data set are acquired according to the teaching action.

[0018] In some embodiments, acquiring the first data set and the second data set according to the teaching action specifically includes:

[0019] Constructing an action environment to control embodied intelligent robots;

[0020] In the action environment, controlling the embodied intelligent robot to perform a plurality of the teaching actions according to the preset specification, and recording the action data of the embodied intelligent robot to obtain the first data set;

[0021] In the action environment, the embodied intelligent robot is controlled to perform several of the teaching actions, and the action data of the embodied intelligent robot that does not meet the preset specifications is recorded to obtain the second data set.

[0022] In some embodiments, the training of the action generation model based on the first data set, the second data set, and the action judgment model specifically includes:

[0023] Constructing a second neural network model, inputting the preprocessed second data set into the second neural network model to generate a standard action, and outputting a predicted standard action and an end-effector state corresponding to the predicted standard action;

[0024] Obtain a first loss value according to the state of the end effector, obtain a second loss value according to the binary classification result, and obtain a third loss value according to the predicted standard action and the first teaching action sample;

[0025] Performing a weighted summation of the first loss value, the second loss value, and the third loss value according to a preset allocation weight;

[0026] Parameters of the second neural network model are optimized according to the weighted summation result to obtain the trained action generation model.

[0027] In some embodiments, obtaining the teaching action to be processed and generating the target teaching action according to the teaching action to be processed by the action generation model specifically includes:

[0028] Obtaining the teaching action to be processed, and breaking the teaching action to be processed into a plurality of teaching action segments;

[0029] Inputting each of the teaching action segments into the action generation model, and outputting a corresponding number of standard teaching segments;

[0030] The target teaching action is obtained by splicing the standard teaching segments.

[0031] To achieve the above objectives, another aspect of the present application provides a system for generating teaching actions for an embodied intelligent robot, comprising:

[0032] The first module is used to obtain a first data set and a second data set;

[0033] A second module is configured to obtain an action generation model based on the first data set and the second data set;

[0034] The third module is used to obtain the teaching action to be processed, and generate the target teaching action according to the teaching action to be processed through the action generation model;

[0035] The first data set includes a plurality of first teaching action samples that meet preset specifications, and the second data set includes a plurality of second teaching action samples that do not meet the preset specifications. Obtaining the action generation model based on the first data set and the second data set specifically includes:

[0036] Setting each of the first teaching motion samples and each of the second teaching motion samples as a two-dimensional matrix of motion dimension multiplied by motion length;

[0037] Setting a target action length value, and when the action length in the two-dimensional matrix is ​​less than the target action length value, padding a corresponding position in the two-dimensional matrix with a zero element to obtain the preprocessed first data set and the second data set;

[0038] Constructing a first neural network model, inputting the preprocessed first data set and the second data set into the first neural network model to perform action specification judgment, and outputting corresponding binary classification judgment results;

[0039] Optimizing the parameters of the first neural network model according to the binary classification result to obtain a trained action judgment model;

[0040] The action generation model is trained based on the first data set, the second data set and the action judgment model.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and runnable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the auxiliary embodied intelligent robot teaching action generation method as described above is realized.

[0042] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the auxiliary embodied intelligent robot teaching action generation method as described above.

[0043] The beneficial effects of the present invention are as follows: the method, system, and medium for generating teaching actions for an assisted embodied intelligent robot first obtain a first data set and a second data set, then obtain a motion generation model based on the first and second data sets, and finally obtain a pending teaching action. The motion generation model generates a target teaching action based on the pending teaching action. The present invention uses a deep learning method to train the motion generation model, and then uses the motion generation model to generate standardized target teaching actions. This can effectively improve the quality of the teaching action data set, thereby improving the training effect of the imitation learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1A flowchart of the steps of a method for generating teaching actions for an assisted embodied intelligent robot provided by one embodiment of the present invention;

[0046] Figure 2 A schematic structural diagram of a first neural network model provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of training an action generation model provided by one embodiment of the present invention;

[0048] Figure 4 A schematic diagram of the steps for generating a target teaching action provided by an embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the structure of a system for generating teaching actions for an assisted embodied intelligent robot provided by an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0052] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0053] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0054] When an embodied intelligent robot learns human actions in a specific scenario, such as learning to pick up an apple in an office setting, there are two main technical methods for the robot to learn: reinforcement learning in a simulation environment and imitation learning in a real scenario.

[0055] The basic process of imitation learning, using wearable devices as a sensing method, is as follows: After an expert puts on the wearable sensor device, it records the movements of various body parts in real time. These recorded movements are then used to control a robot to perform the same movements in real time, enabling it to pick up an apple. During this process, the robot's movements—the motor motion states of each joint, the position of the arm, and the corresponding movements of the dexterous hand—are recorded, forming the series of movements required for the robot to complete the scenario. This series of movements is called an action sample. For imitation learning, dozens or even hundreds of such action samples need to be recorded. These hundreds of recorded action samples serve as standard demonstrations for the apple-picking scenario to train the imitation learning model. Once the model is trained, the robot can be controlled to perform these movements autonomously, and the robot is considered to have "learned" the apple-picking action. It is obvious from this process that the quality of these hundreds of expert-demonstrated movements, that is, the quality of the training data, directly affects the subsequent training of the imitation learning model.

[0056] Furthermore, when experts use wearable sensors to remotely control robots in real time, the process often isn't entirely smooth. This is due to several factors: wearable devices can't perfectly capture human movements; when the movements recorded by wearable sensors are broken down into machine joints, they need to adapt to the robot's joint parameters, discarding or estimating approximate joint motion parameters and losing some information; data transmission from wearable sensors is delayed, causing a lag between the robot's movements and the expert's; and the expert's field of view is somewhat obstructed during remote control. Consequently, the movement sequence may contain pauses, discontinuities, irregular movement curves, and jitter. These issues reduce the quality of the collected teaching data, and thus the effectiveness of the imitation learning model training.

[0057] To this end, an embodiment of the present invention proposes a method for generating teaching actions to assist embodied intelligent robots. First, a first data set and a second data set are obtained. Then, a motion generation model is obtained based on the first data set and the second data set. Finally, a teaching action to be processed is obtained. The target teaching action is generated based on the teaching action to be processed by the motion generation model. The present invention obtains an action generation model through training using a deep learning method, and then generates standardized target teaching actions through the action generation model. This can effectively improve the quality of the teaching action data set, thereby improving the training effect of the imitation learning model. The teaching action generation method can be applied to robots such as embodied intelligent robots, industrial robots, service robots, rehabilitation and assistance robots, but is not limited thereto.

[0058] Reference Figure 1 , Figure 1 This is a flowchart of a method for generating teaching actions for an assisted embodied intelligent robot provided by an embodiment of the present invention. The embodiment of the present invention provides a method for generating teaching actions for an assisted embodied intelligent robot, the method comprising steps S101 to S103:

[0059] S101, obtaining a first data set and a second data set;

[0060] As an optional implementation, the step of obtaining the first data set and the second data set can be specifically divided into the following steps S1011 and S1012:

[0061] S1011, setting a teaching action sample, and breaking the teaching action sample into several teaching actions;

[0062] S1012, acquiring a first data set and a second data set according to the teaching action;

[0063] The first data set includes a number of first teaching action samples that meet the preset specifications, and the second data set includes a number of second teaching action samples that do not meet the preset specifications.

[0064] Specifically, the jamming and jittering in the teaching action are largely caused by the lack of synchronization between the machine and the expert. Therefore, after the embodiment of the present invention breaks down a complex teaching action sample into many small teaching actions, for each teaching action, when performing teaching and collecting data, the expert does not need to pay too much attention to the current state of the machine, but can directly complete a smooth small action, such as moving the arm a certain distance, grabbing an apple, etc. Therefore, the embodiment of the present invention collects a relatively smooth standard action data set (i.e., the first data set) and a non-standard action data set (i.e., the second data set) for these disassembled teaching actions. Among them, the preset standard refers to smooth movements without jamming or jitter, and the final posture error of the end effector is less than a threshold (such as the distance between the fingertip and the target object during grasping is <1cm).

[0065] As an optional implementation, the step of obtaining the first data set and the second data set according to the teaching action can be specifically divided into the following steps S10121 and S10123:

[0066] S10121. Constructing an action environment for controlling embodied intelligent robots;

[0067] S10122. In the action environment, control the embodied intelligent robot to perform a plurality of teaching actions according to preset specifications, and record the action data of the embodied intelligent robot to obtain a first data set;

[0068] In some optional embodiments, standardized motion data is collected for disassembled teaching movements. These disassembled small movements may include: moving the arm a certain distance, turning the wrist, bending the fingers, opening the fingers, grabbing an object, and releasing an object. The objects are small everyday objects, including but not limited to pens, plastic cups, mugs, mice, spoons, bowls, plates, toothbrushes, books, keys, coins, batteries, apples, boxed milk, towels, staplers, folders, tape rolls, wrenches, tape measures, wires, data cables, and mobile phone chargers. Because these disassembled teaching movements are very simple small movements, experts can relatively easily control the robot arm to complete smooth movements, thereby obtaining such standardized movements, that is, movements without lags and jitters, to form the first data set.

[0069] S10123. In the action environment, control the embodied intelligent robot to perform several teaching actions, and record the action data of the embodied intelligent robot that does not meet the preset specifications to obtain a second data set.

[0070] Specifically, for the first teaching motion sample in the first dataset, a second dataset containing stuttering and jitter was recorded and organized, either through expert production or by capturing it from a wider range of other motions. When collecting the second dataset, it was matched to the first teaching motion sample in the first dataset. That is, for the same small motion, there was a teaching motion sample in both the first dataset and the second dataset, facilitating subsequent training of the motion generation model.

[0071] S102, obtaining an action generation model according to the first data set and the second data set;

[0072] As an optional implementation, the step of obtaining the action generation model according to the first data set and the second data set can be specifically divided into the following steps S1021 and S1022:

[0073] S1021. Train an action judgment model based on the first data set and the second data set;

[0074] S1022: Train an action generation model based on the first data set, the second data set, and the action judgment model.

[0075] Specifically, a first data set of standardized actions and a second data set of unplanned actions are used to train an action judgment model that can determine whether an action is standardized. This model is used to guide the training process of the action generation model to generate standardized target teaching actions.

[0076] As a further optional implementation, the first data set includes a plurality of first teaching action samples that meet preset specifications, and the second data set includes a plurality of second teaching action samples that do not meet the preset specifications. The step of training the action judgment model based on the first data set and the second data set can be specifically divided into the following steps S10211 to S10214:

[0077] S10211, setting each first teaching motion sample and each second teaching motion sample to a two-dimensional matrix of motion dimension multiplied by motion length;

[0078] S10212: Setting a target action length value. When the action length in the two-dimensional matrix is ​​less than the target action length value, zero elements are added to the corresponding positions in the two-dimensional matrix to obtain the preprocessed first and second data sets.

[0079] S10213: Construct a first neural network model, input the preprocessed first data set and second data set into the first neural network model to perform action standard judgment, and output a corresponding binary classification judgment result;

[0080] S10214. Optimize the parameters of the first neural network model according to the binary classification judgment result to obtain a trained action judgment model.

[0081] In some optional embodiments, the action judgment model itself can be a general neural network, which does not need to be too complex and can adopt a common transformer structure. For example, Figure 2The figure shows a schematic diagram of the structure of a first neural network model provided by an embodiment of the present invention. The first neural network model includes a multi-layer perceptron, a transformer module, and a fully connected layer. The input of the model is a two-dimensional matrix, namely the action dimension D*action length L. When constructing the model, the action length L is set to a fixed value (i.e., the action length target value) that is long enough. When the length of an action is less than the action length target value, 0 is added to the corresponding position to ensure the temporal integrity of the original action. Each action unit of the input action is processed by the multi-layer perceptron and then input into the transformer module. The final output is a binary classification judgment result output through a fully connected layer. Finally, the backpropagation algorithm is used to optimize the parameters of the first neural network model based on the binary classification judgment result, the teaching action samples in the first data set, and the second data set. When the corresponding optimization condition is met (such as the training reaches a preset number of iterations), the training is stopped to obtain a trained action judgment model.

[0082] As an optional implementation, the step of training the action generation model based on the first data set, the second data set, and the action judgment model can be specifically divided into the following steps S10221 to S10224:

[0083] S10221: Construct a second neural network model, input the preprocessed second data set into the second neural network model to generate a standard action, and output a predicted standard action and an end-effector state corresponding to the predicted standard action;

[0084] S10222: Obtain a first loss value based on the end effector state, obtain a second loss value based on the binary classification result, and obtain a third loss value based on the predicted standard action and the first teaching action sample;

[0085] S10223. Perform a weighted sum of the first loss value, the second loss value, and the third loss value according to a preset distribution weight;

[0086] S10224. Optimize the parameters of the second neural network model according to the weighted summation result to obtain a trained action generation model.

[0087] Specifically, if Figure 3The figure shows a training diagram of an action generation model provided by an embodiment of the present invention. The action generation model is used to input a non-standard action and generate a corresponding standard action. The loss during training is divided into three parts. One part is the action end execution loss, that is, whether the end is executed in place; the second part is the binary judgment result output by the action judgment model, which is used as the judgment loss of whether the action is standard; the third part is the difference directly obtained between the generated action and the GT action. Among them, the GT action represents the corresponding first teaching action sample in the first data set. During the training stage, the three losses are weighted and assigned weights during training. The assigned weights can be adjusted according to the actual training situation. For example, the weight of the first loss value can be set to 0.5, the weight of the second loss value can be set to 1, and the weight of the third loss value can be set to 0.1. When training the action generation model, the input of the model is the second teaching action sample in the second data set that does not meet the preset specifications. The basic model of the action generation model is a pre-built second neural network model, which includes a first multi-layer perceptron, a transformer module and a second multi-layer perceptron. After the action is decomposed into a two-dimensional matrix and mapped by the first multi-layer perceptron, it is sent to the model main module, namely the transformer module for processing. After processing by the transformer module, it is passed through the second multi-layer perceptron to form various action sequences and generate generated actions. Finally, the parameters of the second neural network model are optimized through the back propagation algorithm combined with the weighted sum of the first loss value, the second loss value and the third loss value. When the corresponding optimization conditions are met (such as the training reaches a preset number of iterations), the training is stopped to obtain a trained action generation model.

[0088] S103, obtaining a pending teaching action, and generating a target teaching action based on the pending teaching action using an action generation model;

[0089] As an optional implementation, the step of obtaining the teaching action to be processed and generating the target teaching action according to the teaching action to be processed by the action generation model can be specifically divided into the following steps S1031 to S1033:

[0090] S1031, obtaining a teaching action to be processed, and breaking the teaching action to be processed into a plurality of teaching action segments;

[0091] S1032, inputting each teaching action segment into the action generation model, and outputting a corresponding number of standard teaching segments;

[0092] S1033: Splice the standard teaching segments to obtain the target teaching action.

[0093] Specifically, if Figure 4The figure shows a schematic diagram of the steps for generating a target teaching action according to one embodiment of the present invention. First, a pending teaching action is taken. The pending teaching action is then broken down into smaller actions, resulting in several teaching action segments. The resulting segments are then run through the action generation model one by one to generate a standardized target teaching action. The generated standardized target teaching actions are then reassembled into a complete teaching action until all pending teaching actions are processed.

[0094] The above describes the method for generating teaching motions for an assisted embodied intelligent robot according to an embodiment of the present invention. It can be appreciated that, compared to the teaching motion data collected by current imitation learning, the embodiment of the present invention breaks down a complete teaching motion sample into several small teaching motions, constructs a motion judgment model, and trains the motion judgment model to generate a motion generation model. Finally, the motion generation model optimizes each small teaching motion, removing irregular pauses, jitter, and other issues, generating standardized target teaching motions. This effectively improves the quality of the teaching motion dataset, thereby enhancing the training effectiveness of the imitation learning model.

[0095] Reference Figure 5 , an embodiment of the present invention further provides a system for generating teaching actions for an embodied intelligent robot, comprising:

[0096] The first module is used to obtain a first data set and a second data set;

[0097] The second module is used to obtain an action generation model based on the first data set and the second data set;

[0098] The third module is used to obtain the teaching action to be processed, and generate the target teaching action according to the teaching action to be processed through the action generation model.

[0099] The contents of the above-mentioned auxiliary embodied intelligent robot teaching action generation method embodiment are all applicable to the embodiment of this auxiliary embodied intelligent robot teaching action generation system. The functions specifically implemented by this auxiliary embodied intelligent robot teaching action generation system embodiment are the same as those in the above-mentioned auxiliary embodied intelligent robot teaching action generation method embodiment, and the beneficial effects achieved are also the same as the beneficial effects achieved by the above-mentioned auxiliary embodied intelligent robot teaching action generation method embodiment.

[0100] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the method for generating teaching actions for an assisted embodied intelligent robot is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0101] like Figure 6 FIG2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention, referring to FIG2 Figure 6 , an embodiment of the present invention provides an electronic device, including:

[0102] The processor 1001 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0103] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called by the processor 1001 to execute the teaching action generation method for an assisted embodied intelligent robot according to the embodiments of the present invention.

[0104] Input / output interface 1003, used to implement information input and output;

[0105] Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0106] Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 );

[0107] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0108] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned auxiliary embodied intelligent robot teaching action generation method.

[0109] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0110] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0111] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0112] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0113] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods of various embodiments of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0114] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0115] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0116] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0117] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for generating teaching actions for an embodied intelligent robot, characterized in that: The following steps are involved: Obtain a first data set and a second data set; Obtaining an action generation model based on the first data set and the second data set; Acquire a teaching action to be processed, and generate a target teaching action according to the teaching action to be processed using the action generation model; The first data set includes a plurality of first teaching action samples that meet preset specifications, and the second data set includes a plurality of second teaching action samples that do not meet the preset specifications. Obtaining the action generation model based on the first data set and the second data set specifically includes: Setting each of the first teaching motion samples and each of the second teaching motion samples as a two-dimensional matrix of motion dimension multiplied by motion length; Setting a target action length value, and when the action length in the two-dimensional matrix is ​​less than the target action length value, padding a corresponding position in the two-dimensional matrix with a zero element to obtain the preprocessed first data set and the second data set; Constructing a first neural network model, inputting the preprocessed first data set and the second data set into the first neural network model to perform action specification judgment, and outputting corresponding binary classification judgment results; Optimizing the parameters of the first neural network model according to the binary classification result to obtain a trained action judgment model; The action generation model is trained based on the first data set, the second data set and the action judgment model.

2. The method for generating teaching actions for an assisted embodied intelligent robot according to claim 1, characterized in that: The obtaining of the first data set and the second data set specifically includes: Setting a teaching action sample, and breaking the teaching action sample into a plurality of teaching actions; The first data set and the second data set are acquired according to the teaching action.

3. The method for generating teaching actions for an assisted embodied intelligent robot according to claim 2, characterized in that: The acquiring of the first data set and the second data set according to the teaching action specifically includes: Constructing an action environment to control embodied intelligent robots; In the action environment, controlling the embodied intelligent robot to perform a plurality of the teaching actions according to the preset specification, and recording the action data of the embodied intelligent robot to obtain the first data set; In the action environment, the embodied intelligent robot is controlled to perform several of the teaching actions, and the action data of the embodied intelligent robot that does not meet the preset specifications is recorded to obtain the second data set.

4. The method for generating teaching actions for an assisted embodied intelligent robot according to claim 1, characterized in that: The training of the action generation model according to the first data set, the second data set, and the action judgment model specifically includes: Constructing a second neural network model, inputting the preprocessed second data set into the second neural network model to generate a standard action, and outputting a predicted standard action and an end-effector state corresponding to the predicted standard action; Obtain a first loss value based on the end effector state, obtain a second loss value based on the binary classification result, and obtain a third loss value based on the predicted standard action and the first teaching action sample; Performing a weighted summation of the first loss value, the second loss value, and the third loss value according to a preset allocation weight; Parameters of the second neural network model are optimized according to the weighted summation result to obtain the trained action generation model.

5. A method for generating teaching actions for an assisted embodied intelligent robot according to any one of claims 1 to 4, characterized in that: The acquiring of the teaching action to be processed and generating a target teaching action according to the teaching action to be processed by the action generation model specifically includes: Obtaining the teaching action to be processed, and breaking the teaching action to be processed into a plurality of teaching action segments; Inputting each of the teaching action segments into the action generation model, and outputting a corresponding number of standard teaching segments; The target teaching action is obtained by splicing the standard teaching segments.

6. A system for generating teaching actions for an embodied intelligent robot, characterized in that: include: The first module is used to obtain a first data set and a second data set; A second module is configured to obtain an action generation model based on the first data set and the second data set; The third module is used to obtain the teaching action to be processed, and generate the target teaching action according to the teaching action to be processed through the action generation model; The first data set includes a plurality of first teaching action samples that meet preset specifications, and the second data set includes a plurality of second teaching action samples that do not meet the preset specifications. Obtaining the action generation model based on the first data set and the second data set specifically includes: Setting each of the first teaching motion samples and each of the second teaching motion samples as a two-dimensional matrix of motion dimension multiplied by motion length; Setting a target action length value, and when the action length in the two-dimensional matrix is ​​less than the target action length value, padding a corresponding position in the two-dimensional matrix with a zero element to obtain the preprocessed first data set and the second data set; Constructing a first neural network model, inputting the preprocessed first data set and the second data set into the first neural network model to perform action specification judgment, and outputting corresponding binary classification judgment results; Optimizing the parameters of the first neural network model according to the binary classification result to obtain a trained action judgment model; The action generation model is trained based on the first data set, the second data set and the action judgment model.

7. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the method for generating teaching actions of an assisted embodied intelligent robot as described in any one of claims 1 to 5 are realized.

8. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the auxiliary embodied intelligent robot teaching action generation method as described in any one of claims 1 to 5.

Images