Demonstration action generation method and system for assisting intelligent robot with body and medium

Abstract

The invention discloses a teaching action generation method and system for assisting an intelligent robot with a body and a medium. The teaching action generation method comprises the steps that a first data set and a second data set are acquired; obtaining an action generation model according to the first data set and the second data set; and obtaining a to-be-processed teaching action, and generating a target teaching action according to the to-be-processed teaching action through the action generation model. The action generation model is obtained through training of the deep learning method, then the standard target teaching action is generated through the action generation model, the quality of the teaching action data set can be effectively improved, the training effect of the simulation learning model is improved, and the method can be widely applied to the technical field of artificial intelligence.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, system, and medium for generating teaching actions to assist embodied intelligent robots. Background Art

[0002] When an embodied intelligent robot learns human actions in a specific scenario, such as learning to pick up an apple in an office setting, there are two main technical methods for the robot to learn: reinforcement learning in a simulation environment and imitation learning in a real scenario.

[0003] Current imitation learning methods typically collect teaching motion data through multiple expert demonstrations to gather sufficient valid data as a teaching dataset. However, these expert demonstrations can suffer from issues such as lag, jitter, and irregular movements. In practical projects, experts are often instructed to minimize these lags and jitters, but this issue is not completely eliminated. This reduces the quality of the collected teaching data, impacting the effectiveness of imitation learning model training. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a method, system and medium for generating teaching actions for an assisted embodied intelligent robot with high data set quality and good training effect.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application provides a method for generating teaching actions for an embodied intelligent robot, comprising the following steps: Obtain a first data set and a second data set; Obtaining an action generation model based on the first data set and the second data set; Acquire a teaching action to be processed, and generate a target teaching action according to the teaching action to be processed using the action generation model; The first data set includes a plurality of first teaching action samples that meet preset specifications, and the second data set includes a plurality of second teaching action samples that do not meet the preset specifications. Obtaining the action generation model based on the first data set and the second data set specifically includes: Setting each of the first teaching motion samples and each of the second teaching motion samples as a two-dimensional matrix of motion dimension multiplied by motion length; Setting a target action length value, and when the action length in the two-dimensional matrix is less than the target action length value, padding a corresponding position in the two-dimensional matrix with a zero element to obtain the preprocessed first data set and the second data set; Construct a first neural network model, input the preprocessed first dataset and the second dataset into the first neural network model for action specification judgment, and output corresponding binary classification judgment results; Optimize the parameters of the first neural network model according to the binary classification judgment results to obtain a trained action judgment model; Train the action generation model according to the first dataset, the second dataset, and the action judgment model.

[0006] In some embodiments, the obtaining of the first dataset and the second dataset specifically includes: Set a taught action sample, and disassemble the taught action sample into a plurality of taught actions; Obtain the first dataset and the second dataset according to the taught actions.

[0007] In some embodiments, the obtaining of the first dataset and the second dataset according to the taught actions specifically includes: Construct an action environment for controlling an embodied intelligent robot; In the action environment, control the embodied intelligent robot to execute a plurality of the taught actions according to the preset specifications, and record the action data of the embodied intelligent robot to obtain the first dataset; In the action environment, control the embodied intelligent robot to execute a plurality of the taught actions, and record the action data of the embodied intelligent robot that does not conform to the preset specifications to obtain the second dataset.

[0008] In some embodiments, the training of the action generation model according to the first dataset, the second dataset, and the action judgment model specifically includes: Construct a second neural network model, input the preprocessed second dataset into the second neural network model for generating a specified action, and output a predicted specified action and the end effector state corresponding to the predicted specified action; Obtain a first loss value according to the end effector state, obtain a second loss value according to the binary classification judgment results, and obtain a third loss value according to the predicted specified action and the first taught action sample; Perform weighted summation on the first loss value, the second loss value, and the third loss value according to a preset allocation weight; Optimize the parameters of the second neural network model according to the weighted summation result to obtain the trained action generation model.

[0009] In some embodiments, the obtaining of the teachable action to be processed, and the generating of the target teachable action by the action generation model according to the teachable action to be processed specifically include: Obtain the teachable action to be processed, and disassemble the teachable action to be processed into a plurality of teachable action segments; Input each of the teachable action segments into the action generation model, and output a corresponding plurality of standardized teachable segments; Splice each of the standardized teachable segments to obtain the target teachable action.

[0010] To achieve the above object, another aspect of the embodiments of the present application proposes an assistive embodied intelligent robot teachable action generation system, including: A first module for obtaining a first data set and a second data set; A second module for obtaining an action generation model according to the first data set and the second data set; A third module for obtaining a teachable action to be processed, and generating a target teachable action by the action generation model according to the teachable action to be processed; The first data set includes a plurality of first teachable action samples that conform to a preset specification, and the second data set includes a plurality of second teachable action samples that do not conform to the preset specification. The obtaining of the action generation model according to the first data set and the second data set specifically includes: Set each of the first teachable action samples and each of the second teachable action samples as a two-dimensional matrix of action dimension multiplied by action length; Set a target value for the action length. When the action length in the two-dimensional matrix is less than the target value of the action length, fill zero elements at the corresponding position in the two-dimensional matrix to obtain the preprocessed first data set and second data set; Construct a first neural network model, input the preprocessed first data set and second data set into the first neural network model for action specification judgment, and output corresponding binary classification judgment results; Optimize the parameters of the first neural network model according to the binary classification judgment results to obtain a trained action judgment model; Train to obtain the action generation model according to the first data set, the second data set, and the action judgment model.

[0011] To achieve the above object, on the other hand, an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing the connection and communication between the processor and the memory. When the program is executed by the processor, it realizes the method for generating teaching actions of the assistive embodied intelligent robot as described above.

[0012] To achieve the above object, on the other hand, an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the method for generating teaching actions of the assistive embodied intelligent robot as described above.

[0013] The beneficial effects of the present invention are as follows: The method, system and medium for generating teaching actions of the assistive embodied intelligent robot of the present invention first obtain a first data set and a second data set, then obtain an action generation model according to the first data set and the second data set, and finally obtain a teaching action to be processed. The action generation model generates a target teaching action according to the teaching action to be processed. The present invention trains an action generation model through a deep learning method, and then generates a standard target teaching action through the action generation model, which can effectively improve the quality of the teaching action data set, thereby improving the training effect of the imitation learning model. Description of the Drawings

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduces the drawings required to be used in the embodiments of the present invention. It should be understood that the drawings introduced below only conveniently and clearly represent some embodiments of the technical solutions in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 It is a flowchart of the steps of the method for generating teaching actions of the assistive embodied intelligent robot provided by an embodiment of the present invention; Figure 2 It is a schematic structural diagram of the first neural network model provided by an embodiment of the present invention; Figure 3 It is a training schematic diagram of the action generation model provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of the steps of generating a target teaching action provided by an embodiment of the present invention; Figure 5 It is a schematic structural diagram of the system for generating teaching actions of the assistive embodied intelligent robot provided by an embodiment of the present invention; Figure 6 Schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0016] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application detailed in the appended claims.

[0017] It can be understood that the terms "first", "second", etc. used in the present application may be used in this document to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if", "when" used herein may be interpreted as "when...", "when...", or "in response to determining".

[0018] The terms "at least one", "multiple", "each", "any one", etc. used in the present application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any one refers to any one of the multiple.

[0019] When an embodied intelligent robot learns human actions in a specific scenario, for example, learning to pick up an apple in an office scenario, the methods for the robot to learn are mainly divided into two technical routes: reinforcement learning in a simulation environment and imitation learning in a real scenario.

[0020] The basic process of imitation learning is as follows. Taking the use of wearable devices as a sensing method as an example, after an expert wears the wearable sensing device, this sensing device records the movements of various parts of the human body in real time. These recorded movements will control the robot to perform the same actions in real time, enabling the robot to complete the action of picking up an apple. In this process, the movements of the robot, that is, the motor movement states of each joint, the position of the arm, and the corresponding actions of the dexterous hand, will be recorded to form a series of actions required for the robot to complete this scenario. This series of actions is called an action sample. For imitation learning, dozens or hundreds of such action samples need to be recorded. These hundreds of recorded action samples will be used as the standard demonstration samples for the actions in the apple-picking scenario to train the imitation learning model. After the model is trained, the robot can be controlled to autonomously complete such actions, which means the robot "learns" the action of picking up an apple. In this process, it is obvious that the quality of the hundreds of actions taught by the expert, that is, the quality of the training data, will directly affect the quality of the subsequent training of the imitation learning model.

[0021] In addition, when an expert performs real-time teleoperation on a robot through a wearable sensing device, it is often not very smooth and silky. The reasons are as follows: The wearable device cannot perfectly record human movements; when the movements recorded by the wearable sensing device are decomposed into robot joints, it is necessary to adapt to the robot joint parameters, and some information will be lost by discarding or estimating a rough joint action parameter; there is a delay in the data transmission of the wearable sensing device, resulting in a delay between the action execution of the robot and the expert's action; there will be some obstructions in the expert's field of view during teleoperation. The consequences of these are that there will be some action pauses, discontinuous actions, non-standard action movement curves, and jitters during the action series. These problems will reduce the quality of the collected teaching data, and thus also affect the training effect of the imitation learning model.

[0022] Therefore, the embodiment of the present invention proposes an assistive embodied intelligent robot teaching action generation method. First, obtain the first data set and the second data set, then obtain an action generation model according to the first data set and the second data set, and finally obtain the to-be-processed teaching action. The action generation model generates a target teaching action according to the to-be-processed teaching action. The present invention trains an action generation model through a deep learning method, and then generates a standard target teaching action through this action generation model, which can effectively improve the quality of the teaching action data set, thereby improving the training effect of the imitation learning model. This teaching action generation method can be applied to robots such as embodied intelligent robots, industrial robots, service robots, rehabilitation and assistive robots, etc., but is not limited thereto.

[0023] Refer to Figure 1 , Figure 1The figure is a flowchart of the steps of an embodiment of an auxiliary embodied intelligent robot teaching action generation method provided by the present invention. An embodiment of the present invention proposes an auxiliary embodied intelligent robot teaching action generation method, which includes steps S101 to S103: S101. Obtain a first data set and a second data set; Further as an optional implementation manner, the step of obtaining the first data set and the second data set can be specifically divided into the following steps S1011 and S1012: S1011. Set a teaching action sample and disassemble the teaching action sample into several teaching actions; S1012. Obtain the first data set and the second data set according to the teaching actions; Among them, the first data set includes several first teaching action samples that meet the preset specifications, and the second data set includes several second teaching action samples that do not meet the preset specifications.

[0024] Specifically, the jerks and jitters in the teaching actions are largely caused by the out-of-sync between the machine and the expert. Therefore, in an embodiment of the present invention, after disassembling a complex teaching action sample into many small teaching actions, for each teaching action, when collecting teaching data, the expert does not have to pay too much attention to the current state of the machine and can directly complete a smooth small action. For example, the arm moves a certain distance and grabs an apple, etc. Therefore, in an embodiment of the present invention, for these disassembled teaching actions, a relatively smooth standard action data set (i.e., the first data set) and a non-standard action data set (i.e., the second data set) are respectively collected. Among them, the preset specification means that the action is smooth without jerks, without jitters, and the final pose error of the end effector is less than a threshold (such as the distance between the fingertip and the target object is <1 cm) when grasping, etc.

[0025] Further as an optional implementation manner, the step of obtaining the first data set and the second data set according to the teaching actions can be specifically divided into the following steps S10121 and S10123: S10121. Construct an action environment for controlling the embodied intelligent robot; S10122. In the action environment, control the embodied intelligent robot to execute several teaching actions according to the preset specifications and record the action data of the embodied intelligent robot to obtain the first data set; In some alternative embodiments, for the taught actions of disassembly, standardized action data is collected. These small disassembly actions may include: the arm moving a certain distance, the wrist flipping, the fingers bending, the fingers opening, grasping an object, and releasing an object, etc. The object is a daily small object, including but not limited to a pen, a plastic cup, a mug, a mouse, a spoon, a bowl, a plate, a toothbrush, a book, a key, a coin, a battery, an apple, a boxed milk, a towel, a stapler, a folder, a tape roll, a wrench, a tape measure, a wire, a data cable, and a mobile phone charger, etc. Since these taught actions obtained by disassembly are very simple small actions, experts can relatively easily control the manipulator to complete smooth actions, thereby obtaining such standardized actions, that is, actions without jamming and jitter, to form the first data set.

[0026] S10123. In the action environment, control the embodied intelligent robot to execute a number of taught actions, and record the action data of the embodied intelligent robot that does not conform to the preset specification to obtain the second data set.

[0027] Specifically, for the first taught action sample in the first data set, by means of deliberate production by experts or intercepting from other large-scale actions, record and sort out the actions with jamming and jitter to form the second data set. When collecting the second data set, it corresponds to the first taught action sample in the first data set, that is, for the same small action, there is its taught action sample in the first data set and there is also a corresponding taught action sample in the second data set, so as to facilitate the subsequent training of the action generation model.

[0028] S102. Obtain an action generation model according to the first data set and the second data set; Further as an alternative implementation manner, the step of obtaining an action generation model according to the first data set and the second data set can be specifically divided into the following steps S1021 and S1022: S1021. Train to obtain an action judgment model according to the first data set and the second data set; S1022. Train to obtain an action generation model according to the first data set, the second data set and the action judgment model.

[0029] Specifically, use the first data set with standardized actions and the second data set with non-standardized actions to train an action judgment model that can judge whether an action is standardized, and guide the training process of the action generation model through this model to generate standardized target taught actions.

[0030] As a further optional implementation, the first data set includes a number of first teaching action samples that meet the preset specifications, and the second data set includes a number of second teaching action samples that do not meet the preset specifications. The step of training an action judgment model based on the first data set and the second data set can be specifically divided into the following steps S10211 to S10214: S10211. Set each first teaching action sample and each second teaching action sample as a two-dimensional matrix of action dimension multiplied by action length; S10212. Set the target value of the action length. When the action length in the two-dimensional matrix is less than the target value of the action length, fill in zero elements at the corresponding positions in the two-dimensional matrix to obtain the preprocessed first data set and second data set; S10213. Construct a first neural network model, input the preprocessed first data set and second data set into the first neural network model for action specification judgment, and output the corresponding binary classification judgment result; S10214. Optimize the parameters of the first neural network model according to the binary classification judgment result to obtain a trained action judgment model.

[0031] In some optional embodiments, the action judgment model itself can collect a general neural network and does not need to be too complex. A common transformer structure can be used. Exemplarily, as Figure 2 shown in the structural schematic diagram of the first neural network model provided by an embodiment of the present invention. The first neural network model includes a multi-layer perceptron, a transformer module, and a fully connected layer. The input of the model is a two-dimensional matrix, that is, action dimension D * action length L. When constructing the model, the action length L is set as a sufficiently long fixed value (i.e., the target value of the action length). When the length of a certain action is less than the target value of the action length, 0 is filled in at the corresponding position to ensure the temporal integrity of the original action. Each action unit of the input action is processed by the multi-layer perceptron and then input into the transformer module, and finally the output passes through a fully connected layer to output the binary classification judgment result. Finally, according to the binary classification judgment result, the first data set, and the teaching action samples in the second data set, the parameters of the first neural network model are optimized by the backpropagation algorithm. When the corresponding optimization conditions are met (such as training reaching the preset number of iterations), the training is stopped to obtain a trained action judgment model.

[0032] As a further optional implementation, the step of training an action generation model based on the first data set, the second data set, and the action judgment model can be specifically divided into the following steps S10221 to S10224: S10221. Construct a second neural network model, input the preprocessed second dataset into the second neural network model to generate standard actions, and output the predicted standard actions and the corresponding end effector states of the predicted standard actions; S10222. Obtain the first loss value according to the end effector state, obtain the second loss value according to the binary classification judgment result, and obtain the third loss value according to the predicted standard action and the first teaching action sample; S10223. Perform weighted summation on the first loss value, the second loss value, and the third loss value according to the preset allocation weights; S10224. Optimize the parameters of the second neural network model according to the weighted summation result to obtain a trained action generation model.

[0033] Specifically, as Figure 3 shown in the training schematic diagram of the action generation model provided by an embodiment of the present invention. The action generation model is used to input an irregular action and generate a corresponding standard action. The loss during training is divided into three parts. One part is the end effector execution loss of the action, that is, whether the end is executed in place; the second part is the binary classification judgment result output by the action judgment model, which is used as the judgment loss of whether the action is standard; the third part is the difference directly obtained between the generated action and the GT action. Among them, the GT action represents the first teaching action sample corresponding in the first dataset. During the training stage, these three losses are weighted and allocated weights during training, and the allocation weights can be adjusted according to the actual training situation. Exemplarily, the weight of the first loss value can be set to 0.5, the weight of the second loss value can be set to 1, and the weight of the third loss value can be set to 0.1. When training this action generation model, the input of the model is the second teaching action sample that does not meet the preset specifications in the second dataset. The basic model of the action generation model is a pre-constructed second neural network model. The second neural network model includes a first multi-layer perceptron, a transformer module, and a second multi-layer perceptron. After the action is disassembled into a two-dimensional matrix and mapped by the first multi-layer perceptron, it is sent to the model main body module, that is, the transformer module for processing. After the transformer module processes it, it passes through the second multi-layer perceptron to form each action sequence and form the generated action. Finally, through the backpropagation algorithm, combined with the weighted summation result of the first loss value, the second loss value, and the third loss value, the parameters of the second neural network model are optimized. When the corresponding optimization conditions are met (such as the training reaches the preset number of iterations), the training is stopped to obtain a trained action generation model.

[0034] S103. Obtain the teaching action to be processed, and generate a target teaching action through the action generation model according to the teaching action to be processed; As a further optional implementation, the step of obtaining the teach action to be processed and generating the target teach action according to the teach action to be processed through the action generation model can be specifically divided into the following steps S1031 to S1033: S1031. Obtain the teach action to be processed and disassemble the teach action to be processed into several teach action segments; S1032. Input each teach action segment into the action generation model and output the corresponding several standardized teach segments; S1033. Concatenate each standardized teach segment to obtain the target teach action.

[0035] Specifically, as Figure 4 shown in the schematic diagram of the steps for generating the target teach action provided by an embodiment of the present invention, first take a teach action to be processed, then disassemble the teach action to be processed into small actions to obtain several teach action segments, and then pass the disassembled teach action segments through the action generation model one by one to generate a standardized target teach action. Furthermore, the generated standardized target teach actions are reassembled into a complete teach action until all the teach actions to be processed are completed.

[0036] The above describes the method for generating the teach action of the assistive embodied intelligent robot according to the embodiments of the present invention. It can be recognized that compared with the teach action data collected by current imitation learning, the embodiments of the present invention disassemble the complete teach action sample into several small teach actions, construct an action judgment model, train the action generation model through the action judgment model, and finally, through the action generation model, optimize each small teach action to remove problems such as non-standard pauses and jitters, and generate standardized target teach actions, which can effectively improve the quality of the teach action data set and thus improve the training effect of the imitation learning model.

[0037] Referring to Figure 5 , the embodiments of the present invention also provide an assistive embodied intelligent robot teach action generation system, including: The first module is used to obtain the first data set and the second data set; The second module is used to obtain the action generation model according to the first data set and the second data set; The third module is used to obtain the teach action to be processed and generate the target teach action according to the teach action to be processed through the action generation model.

[0038] The content in the embodiments of the above-described method for generating teaching actions of an assisted embodied intelligent robot is applicable to the embodiments of the present assisted embodied intelligent robot teaching action generation system. The functions specifically implemented by the embodiments of the present assisted embodied intelligent robot teaching action generation system are the same as those of the above-described method embodiments for generating teaching actions of an assisted embodied intelligent robot, and the beneficial effects achieved are also the same as those of the above-described method embodiments for generating teaching actions of an assisted embodied intelligent robot.

[0039] Embodiments of the present invention also provide an electronic device, which includes: a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. When the program is executed by the processor, it implements the above-described method for generating teaching actions of an assisted embodied intelligent robot. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0040] As Figure 6 shown is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. Referring to Figure 6 , embodiments of the present invention provide an electronic device, including: A processor 1001, which can be implemented in ways such as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention; A memory 1002, which can be implemented in forms such as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1002 and are called by the processor 1001 to execute the method for generating teaching actions of an assisted embodied intelligent robot according to the embodiments of the present invention; An input / output interface 1003, which is used to implement information input and output; A communication interface 1004, which is used to implement communication interaction between this device and other devices, and can implement communication through a wired method (such as USB, network cable, etc.) or through a wireless method (such as a mobile network, WIFI, Bluetooth, etc.); A bus 1005, which transmits information between various components of the device (such as the processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004); Among them, the processor 1001, the memory 1002, the input / output interface 1003, and the communication interface 1004 are communicatively connected to each other inside the device through the bus 1005.

[0041] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned method for generating teaching actions of an assistive embodied intelligent robot.

[0042] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0043] An embodiment of the present invention also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 the method shown.

[0044] In some alternative embodiments, the functions / operations mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two consecutive blocks shown may actually be executed substantially simultaneously or the above blocks can sometimes be executed in the reverse order. In addition, the embodiments presented and described in the flowcharts of the present invention are provided by way of example for the purpose of providing a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated, in which the order of various operations is changed and the sub-operations described as part of a larger operation are executed independently.

[0045] In addition, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the above-described functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It should also be understood that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of such modules would be understood within the routine skills of an engineer. Thus, those skilled in the art can implement the present invention as set forth in the claims without undue experimentation. It should also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0046] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods of the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0047] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0048] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0049] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

[0050] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. An assistive embodied intelligent robot teaching action generation method, characterized in that, It includes the following steps: Obtain a first data set and a second data set; Obtain an action generation model according to the first data set and the second data set; Obtain a teach action to be processed, and generate a target teach action according to the teach action to be processed through the action generation model; The first data set includes a number of first teach action samples that conform to a preset specification, and the second data set includes a number of second teach action samples that do not conform to the preset specification. Obtaining the action generation model according to the first data set and the second data set specifically includes: Set each of the first teach action samples and each of the second teach action samples as a two-dimensional matrix of action dimension multiplied by action length; Set a target value for the action length. When the action length in the two-dimensional matrix is less than the target value of the action length, fill zero elements at the corresponding position in the two-dimensional matrix to obtain the preprocessed first data set and second data set; Construct a first neural network model, input the preprocessed first data set and second data set into the first neural network model for action specification judgment, and output corresponding binary classification judgment results; Optimize the parameters of the first neural network model according to the binary classification judgment results to obtain a trained action judgment model; Train to obtain the action generation model according to the first data set, the second data set, and the action judgment model.

2. The method for generating a teaching action of an assistive embodied intelligent robot according to claim 1, wherein, [[ID=~1]] The obtaining of the first data set and the second data set specifically includes: Set teach action samples, and disassemble the teach action samples into a number of teach actions; Obtain the first data set and the second data set according to the teach actions.

3. The method for generating a teaching action of an assistive embodied intelligent robot according to claim 2, wherein The obtaining of the first data set and the second data set according to the teach actions specifically includes: Construct an action environment for controlling an embodied intelligent robot; In the action environment, control the embodied intelligent robot to execute a number of the teach actions according to the preset specification, and record the action data of the embodied intelligent robot to obtain the first data set; In the action environment, control the embodied intelligent robot to execute a number of the teach actions, and record the action data of the embodied intelligent robot that does not conform to the preset specification to obtain the second data set.

4. The method for generating a teaching action of an assistive embodied intelligent robot according to claim 1, wherein, The training to obtain the action generation model according to the first data set, the second data set, and the action judgment model specifically includes: Construct a second neural network model, input the preprocessed second data set into the second neural network model for generating a standard action, and output a predicted standard action and the end effector state corresponding to the predicted standard action; Obtain a first loss value according to the end effector state, obtain a second loss value according to the binary classification judgment results, and obtain a third loss value according to the predicted standard action and the first teach action sample; Perform weighted summation on the first loss value, the second loss value, and the third loss value according to a preset allocation weight; Optimize the parameters of the second neural network model according to the weighted summation result to obtain the trained action generation model.

5. A method for generating teaching actions of an assistive embodied intelligent robot according to any one of claims 1 to 4, characterized in that, The obtaining of the teachable action to be processed, and the generation of the target teachable action by the action generation model according to the teachable action to be processed specifically includes: Obtain the teachable action to be processed, and disassemble the teachable action to be processed into a plurality of teachable action segments; Input each of the teachable action segments into the action generation model, and output a corresponding plurality of standardized teachable segments; Splice each of the standardized teachable segments to obtain the target teachable action.

6. An assistive embodied intelligent robot teaching action generation system, characterized in that, It includes: A first module for obtaining a first data set and a second data set; A second module for obtaining an action generation model according to the first data set and the second data set; A third module for obtaining a teachable action to be processed, and generating a target teachable action by the action generation model according to the teachable action to be processed; The first data set includes a plurality of first teachable action samples that conform to a preset specification, and the second data set includes a plurality of second teachable action samples that do not conform to the preset specification. The obtaining of the action generation model according to the first data set and the second data set specifically includes: Set each of the first teachable action samples and each of the second teachable action samples as a two-dimensional matrix of action dimension multiplied by action length; Set a target value for the action length. When the action length in the two-dimensional matrix is less than the target value of the action length, fill zero elements at the corresponding position in the two-dimensional matrix to obtain the preprocessed first data set and second data set; Construct a first neural network model, input the preprocessed first data set and second data set into the first neural network model for action specification judgment, and output corresponding binary classification judgment results; Optimize the parameters of the first neural network model according to the binary classification judgment results to obtain a trained action judgment model; Train to obtain the action generation model according to the first data set, the second data set, and the action judgment model.

7. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing the connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the method for assisting the teachable action generation of an embodied intelligent robot according to any one of claims 1 to 5 are realized.

8. A storage medium, the storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the steps of the method for assisting the teachable action generation of an embodied intelligent robot according to any one of claims 1 to 5.

Images