Robot teaching learning method and system and electronic equipment

Abstract

The invention discloses a robot teaching learning method and system and electronic equipment, and belongs to the technical field of robot control. The method comprises the following steps: performing time synchronization and de-noising processing on an original teaching signal to obtain a first teaching signal, and inputting the first teaching signal to a modal encoder to obtain a hidden state feature; linear weighting is carried out based on the hidden state features, and unified fusion features are generated through a nonlinear activation function; on the basis of the unified fusion feature, calculating the minimum accumulated cost through dynamic planning, determining an alignment path and a stage boundary, and segmenting the unified fusion feature into sub-tasks on the basis of the alignment path and the stage boundary; aiming at the subtasks, performing model optimization by using weighted behavior cloning loss to obtain an optimization model; and carrying out self-supervised learning on the optimization model by adopting self-supervised time sequence consistency loss. According to the invention, the robustness and accuracy of robot teaching learning are improved.

Description

Technical Field

[0001] The present application belongs to the field of robot control technology, and in particular relates to a robot teaching and learning method, system, and electronic equipment. Background Art

[0002] Currently, multimodal fusion is one of the core technologies in the fields of artificial intelligence and robotics. However, existing teaching systems rarely integrate multimodal teaching signals. Most of them process the data of each modality in a serial or simple splicing manner, which cannot effectively capture the correlation between different modalities. In addition, complex operations often require the teaching process to be divided into several subtasks or stages (such as grasping, handling, assembly, etc.). However, traditional teaching alignment methods usually require manual labeling or rely on preset similarity metrics for alignment, lacking automation and robustness. Existing algorithms use various information such as speed and force as the basis for segmentation to perform teaching segmentation, but how to achieve accurate alignment in multimodality remains a challenge.

[0003] In response to the above problems, the robot teaching learning method, system and electronic equipment in this application are proposed. Summary of the Invention

[0004] In order to address the deficiencies of the prior art, the present application provides a robot teaching learning method, system and electronic device to solve the problems in the prior art that traditional robot teaching methods usually require manual labeling or rely on preset similarity metrics for alignment, lack automation and robustness, etc.

[0005] The technical effects to be achieved by this application are achieved through the following solutions: In a first aspect, the present application provides a robot teaching and learning method, the method comprising: Performing time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, and inputting the first teaching signal into a modal encoder to obtain a latent state feature; Perform linear weighting based on the hidden state features and generate unified fusion features through a nonlinear activation function; Based on the unified fusion feature, a minimum cumulative cost is calculated by dynamic programming to determine an alignment path and a stage boundary, and the unified fusion feature is divided into subtasks based on the alignment path and the stage boundary; For the subtask, weighted behavioral cloning loss is used to optimize the training model to obtain an optimized model; Self-supervised temporal consistency loss is used to perform self-supervised learning on the optimization model.

[0006] In some embodiments, the latent state feature is , ,in Indicates the Mode Indicates at time , Indicates the first teaching signal of the mode at time . Indicates the mode encoder corresponding to the mode

[0007] In some embodiments, the unified fusion feature is , where represents the total number of modes, represents the fusion weight of the mode, satisfying , represents the linear mapping matrix of the mode, represents the bias vector, represents the non-linear activation function

[0008] In some embodiments, calculating the minimum cumulative cost based on the unified fusion feature by dynamic programming includes: Obtaining the cost matrix , where A and B represent two different teaching sequences respectively, represents the feature of the mode in sequence A and the time index is , represents the feature of the mode in sequence B and the time index is , ‖ ‖ represents calculating the norm of the vector

[0009] In some embodiments, calculating the minimum cumulative cost based on the unified fusion feature by dynamic programming further includes: Based on the cost matrix obtaining the minimum cumulative cost where the boundary conditions are , for any or , .

[0010] In some embodiments, the weighted behavior cloning loss is , , where represents the total number of subtasks, represents the time sequence length of the subtask Denote the true action vector of the -th subtask at time . Denote the action vector of the -th subtask predicted by the training model at time . Denote the weight of the -th subtask, satisfying .

[0011] In some embodiments, the self-supervised temporal consistency loss is , , where denotes the length of the unified fusion feature sequence, denotes the unified fusion feature at time +1, and denotes the temporal feature prediction network.

[0012] In some embodiments, A represents a reference teaching sequence, and B represents a teaching sequence to be aligned.

[0013] In a second aspect, the present application provides a robot teaching learning system, which includes: a data preprocessing and modality encoding module, a multi-modal teaching signal fusion module, an automatic segmentation and alignment module, a multi-task behavior cloning module, and a self-supervised temporal consistency module. The robot teaching learning system can implement the method described in any one of the foregoing.

[0014] In a third aspect, the present application provides an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method described in any one of the foregoing is implemented.

[0015] Through the robot teaching learning method, system, and electronic device provided by the present application, the method is based on multi-modal fusion, and through teaching alignment, multi-task loss design, and model optimization and adjustment of temporal consistency, the robustness and accuracy of robot teaching learning are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1Flowchart of a robot teaching learning method in an embodiment of the present application; Figure 2 Schematic structural diagram of a robot teaching learning system in an embodiment of the present application; Figure 3 Schematic block diagram of an electronic device in an embodiment of the present application. Detailed implementation manners

[0018] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part rather than all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0019] It should be noted that unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should be the ordinary meanings understood by those of ordinary skill in the art to which the present application belongs. The "first", "second" and similar terms used in one or more embodiments of the present application do not denote any order, quantity or importance, but are only used to distinguish different components. The terms such as "include" or "comprise" mean that the elements or items appearing before this term cover the elements or items listed after this term and their equivalents, without excluding other elements or items. The terms such as "connect" or "couple" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0020] Related robot teaching learning methods usually rely on single-modal information (such as only relying on vision or action recording) for task learning, and it is difficult to make full use of the multi-source information contained in the teaching process. For example, in the field of industrial collaborative robots, the fusion of multi-modal information (vision, force sense, voice, etc.) is of great significance for fully understanding the teaching intention.

[0021] In addition, the commonly used Behavior Cloning method in teaching learning adopts a unified loss function for different tasks. However, in multi-task or multi-objective scenarios, it is difficult for a single loss function to take into account the optimization of each task. The traditional mean squared error loss is vulnerable to competition between tasks, resulting in poor performance of some tasks. Finally, the teaching data has obvious temporal dependence, but the existing learning mechanisms often ignore the self-supervised constraints of sequence information, and phenomena such as discontinuous prediction or jitter are likely to occur. Introducing temporal consistency in self-supervised learning has been proven to be able to mine rich context information of temporal data, but it has not been fully utilized in the field of teaching learning.

[0022] In summary, the existing technologies have deficiencies in multi-modal fusion, teaching alignment, multi-task loss design, and temporal consistency training, etc., and new technical solutions are urgently needed for improvement.

[0023] This application proposes a multi-modal based robot teaching learning method for the scenario of industrial collaborative robot teaching learning.

[0024] The core ideas of this application include: 1. Construct a multi-modal encoding structure for teaching signals, extract features from teaching signals of multiple modalities such as vision, dynamics, and force sense respectively, and perform fusion. 2. Design a segmented teaching alignment algorithm to automatically identify the stage boundaries in the teaching process and align the corresponding task segments. 3. Introduce a multi-task behavior cloning loss function to simultaneously optimize the behavior cloning objectives of multiple sub-tasks in a weighted manner. 4. Adopt a self-supervised temporal consistency training mechanism to improve the stability and generalization ability of teaching encoding by constraining the continuity of teaching features in the time dimension.

[0025] The above innovative combinations constitute the multi-modal teaching learning sub-module in the embodied intelligent robot system based on the hybrid memory bank, which improves the utilization efficiency of teaching data and the learning effect from the system architecture level.

[0026] This application uses five key modules: data preprocessing and modal encoding module, multi-modal teaching signal fusion module, automatic segmented alignment module, multi-task behavior cloning module, and self-supervised temporal consistency module to form a complete model training process. The training includes the following five steps: 1. Data preprocessing and modal encoding: Synchronize and clean the multi-modal signals such as vision, dynamics, force sense, and speech collected, and use feature encoders to extract primary hidden state features respectively.

[0027] 2. Multi-modal feature fusion: Fuse the hidden states of each modality into a unified temporal feature representation according to weights to generate a fused feature sequence .

[0028] 3. Segment teaching automatic alignment: Based on the fused features, the dynamic time warping (DTW) method is used to identify the phase boundaries in the teaching process and complete the global optimal alignment of multiple teaching sequences.

[0029] 4. Multi-task behavior cloning training: Design a weighted behavior cloning loss function, and perform supervised learning through the sub-task sequences divided after alignment to optimize the network action prediction.

[0030] 5. Self-supervised temporal consistency optimization: Based on behavior cloning, add a temporal prediction network to perform self-supervised constraints on the cross-moment feature coherence and improve the temporal stability of the model.

[0031] Next, in conjunction with the accompanying drawings, various non-limiting embodiments of the present application will be described in detail.

[0032] First, with reference to Figure 1 , the robot teaching learning method of the present application will be described in detail.

[0033] The present application provides a robot teaching learning method, and the method includes: S1: Perform time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, and input the first teaching signal into a modal encoder to obtain hidden state features; S2: Perform linear weighting based on the hidden state features and generate unified fused features through a non-linear activation function; S3: Based on the unified fused features, calculate the minimum cumulative cost through dynamic programming, determine the alignment path and phase boundaries, and segment the unified fused features into sub-tasks based on the alignment path and phase boundaries; S4: For the sub-tasks, use the weighted behavior cloning loss to optimize the training model to obtain an optimized model; S5: Perform self-supervised learning on the optimized model using the self-supervised temporal consistency loss.

[0034] The above robot teaching learning method is based on multi-modal fusion, and through teaching alignment, multi-task loss design, and model optimization and adjustment of temporal consistency, it improves the robustness and accuracy of robot teaching learning.

[0035] Exemplarily, the training model refers to an action prediction model (such as a hierarchical behavior prediction model based on LSTM, a multi-head attention action prediction model based on Transformer, or a lightweight sequence model based on GRU), which is used for cloning training and can also be called a behavior policy network model.

[0036] In some embodiments, the hidden state features are , , where represents the th mode, represents at time , represents the th mode at time of the first teaching signal, represents the th mode corresponding to the mode encoder.

[0037] Exemplarily, the first teaching signal may include a visual image , joint angles , end force sense , voice commands , etc., represents at time .

[0038] Exemplarily, the mode encoder may be a convolutional neural network (CNN), a recurrent neural network (RNN), or a pre-trained model.

[0039] The above hidden state features are to provide an accurate basis for subsequent feature fusion.

[0040] In some embodiments, the unified fusion feature is , where represents the total number of modes, represents the th mode fusion weight, satisfying , represents the th mode linear mapping matrix, represents the bias vector, represents the non-linear activation function.

[0041] Exemplarily, can be automatically normalized through layers or dynamically adjusted using a learning strategy.

[0042] In some embodiments, based on the unified fusion feature, calculating the minimum cumulative cost through dynamic programming includes: Obtaining the cost matrix , where A and B respectively represent two different teaching sequences, represents the th mode in sequence A and the feature at time index , represents the in the modality and with a time index of feature ‖ ‖ represents the modulus of the computed vector, i.e., the norm of the vector.

[0043] Exemplarily, represents the time index of sequence A, with a value range of .

[0044] Exemplarily, represents the time index of sequence B, with a value range of .

[0045] In some embodiments, A represents a reference teaching sequence, such as a standard action; B represents a teaching sequence to be aligned, such as another teaching sequence that needs to be time-aligned with the standard action.

[0046] In some embodiments, calculating the minimum cumulative cost through dynamic programming based on the unified fusion feature further includes: Based on the cost matrix obtain the minimum cumulative cost where the boundary conditions are , for any or , .

[0047] In some embodiments, the weighted behavior cloning loss is , , where represents the total number of subtasks, represents the temporal length of the th subtask, represents the true action vector of the th subtask at time , represents the action vector of the th subtask predicted by the model at time , represents the weight of the th subtask, satisfying .

[0048] Exemplarily, the weight can be determined by Bayesian optimization, uniform initialization, or an adaptive algorithm.

[0049] In some embodiments, the self-supervised temporal consistency loss is , , where represents the length of the unified fusion feature sequence, Indicates a moment The unified fusion feature of +1 Indicates the temporal feature prediction network

[0050] Exemplarily, the temporal feature prediction network can select structures such as fully connected layers, recurrent neural networks (LSTM / GRU), etc

[0051] Based on the existing multi-modal encoding and behavior cloning models, this application introduces a temporal feature prediction network to perform self-supervised learning on continuous fusion features to enhance the temporal coherence of the features

[0052] The robot teaching learning method of this application improves the temporal smoothness and robustness of the teaching encoding while ensuring the accuracy of robot motion prediction

[0053] In a second aspect, as Figure 2 shown, this application provides a robot teaching learning system, and the robot teaching learning system includes: a data preprocessing and modal encoding module, a multi-modal teaching signal fusion module, an automatic segmentation and alignment module, a multi-task behavior cloning module, and a self-supervised temporal consistency module. The robot teaching learning system can implement the method described in any one of the foregoing

[0054] The above-mentioned robot teaching learning system can achieve all the technical effects of the robot teaching learning method, which will not be elaborated here

[0055] It should be noted that the method of one or more embodiments of this application can be executed by a single device, such as a computer or a server, etc. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In this case of a distributed scenario, one of the multiple devices can only execute one or more steps of the method of one or more embodiments of this application, and these multiple devices will interact with each other to complete the described method

[0056] It should be noted that the specific embodiments of this application are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multi-task processing and parallel processing are also possible or may be advantageous

[0057] Based on the same inventive concept, corresponding to the method of any of the above embodiments, this application also discloses an electronic device. Specifically Figure 3The figure shows a schematic hardware structure diagram of an electronic device for a robot teaching and learning method provided in this embodiment. The device may include: a processor 310, a memory 320, an input / output interface 330, a communication interface 340, and a bus 350. Among them, the processor 310, the memory 320, the input / output interface 330, and the communication interface 340 are communicatively connected to each other inside the device through the bus 350.

[0058] The processor 310 may be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0059] The memory 320 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 320 may store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of the present application through software or firmware, the relevant program codes are stored in the memory 320 and are called and executed by the processor 310.

[0060] The input / output interface 330 is used to connect to an input / output module to implement information input and output. The input / output module may be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Among them, the input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0061] The communication interface 340 is used to connect to a communication module (not shown in the figure) to implement communication interaction between this device and other devices. Among them, the communication module may implement communication in a wired manner (for example, USB, network cable, etc.) or in a wireless manner (for example, mobile network, WIFI, Bluetooth, etc.).

[0062] The bus 350 includes a path for transmitting information between various components of the device (for example, the processor 310, the memory 320, the input / output interface 330, and the communication interface 340).

[0063] It should be noted that although the above device only shows the processor 310, the memory 320, the input / output interface 330, the communication interface 340, and the bus 350, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary to implement the solution of the embodiments of the present application, and does not necessarily include all the components shown in the figure.

[0064] The electronic device in the above embodiment is used to implement the corresponding robot teaching and learning method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described herein again.

[0065] Based on the same inventive concept, corresponding to the method in any of the above embodiments, one or more embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the robot teaching and learning method described in any of the foregoing embodiments.

[0066] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0067] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the robot teaching and learning method described in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be described herein again.

[0068] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary, and is not intended to imply that the scope of the present application is limited to these examples; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of one or more embodiments of the present application as described above, and they are not provided in detail for the sake of brevity.

[0069] Additionally, for simplicity of explanation and discussion, and so as not to render one or more embodiments of the present application difficult to understand, well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Further, the devices may be shown in block diagram form in order to avoid rendering one or more embodiments of the present application difficult to understand, and this also takes into account the fact that details regarding the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application are to be implemented (i.e., these details should be fully within the understanding of those of ordinary skill in the art). In cases where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it will be apparent to those of ordinary skill in the art that one or more embodiments of the present application may be practiced without these specific details or with variations of these specific details. Accordingly, these descriptions should be regarded as illustrative rather than restrictive.

[0070] Although the present application has been described in connection with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0071] One or more embodiments of the present application are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the appended claims. Accordingly, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of the present application shall be included within the scope of protection of the present application.

Claims

1. A robot teaching and learning method, characterized in that, Including: Performing time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, and inputting the first teaching signal into a modal encoder to obtain hidden state features; Performing linear weighting based on the hidden state features and generating unified fusion features through a non-linear activation function; Based on the unified fusion features, calculating the minimum cumulative cost through dynamic programming, determining the alignment path and stage boundaries, and segmenting the unified fusion features into subtasks based on the alignment path and stage boundaries; For the subtasks, using a weighted behavior cloning loss to optimize the training model to obtain an optimized model; Performing self-supervised learning on the optimized model using a self-supervised temporal consistency loss.

2. The robot teaching learning method according to claim 1, wherein, The hidden state feature is , , where represents the th modality, represents at time , represents the th modality's first teaching signal at time , and represents the modality encoder corresponding to the th modality.

3. The robot teaching learning method according to claim 2, characterized in that The unified fusion feature is , where represents the total number of modalities, represents the fusion weight of the th modality, satisfying , represents the linear mapping matrix of the th modality, represents the bias vector, represents the non-linear activation function.

4. The robot teaching learning method according to claim 3, wherein The calculating the minimum cumulative cost through dynamic programming based on the unified fusion features includes: Obtain the cost matrix , where A and B respectively represent two different teaching sequences, Indicates the mode in sequence A and the time index is feature of Indicating the mode in sequence B and the time index is feature of ‖ ‖ represents calculating the norm of a vector.

5. The robot teaching learning method according to claim 4, characterized in that, The calculating the minimum cumulative cost through dynamic programming based on the unified fusion features further includes: Based on the cost matrix obtain the minimum cumulative cost wherein, the boundary condition is , for any or , .

6. The robot teaching learning method according to claim 5, characterized in that, The weighted behavior cloning loss is , , Among them, represents the total number of subtasks, Indicates the timing length of the Indicates the subtask at the moment true action vector, Indicates the action vector of the subtask predicted by the training model at the moment , Indicates the weight of the subtask, satisfying 7. The robot teaching learning method according to claim 1, wherein The self-supervised temporal consistency loss is , , Among them, represents the length of the unified fusion feature sequence, Indicating a moment The unified fusion feature of Represents a timing feature prediction network.

8. The robot teaching and learning method according to claim 4, characterized in that, A represents the reference teaching sequence, and B represents the teaching sequence to be aligned.

9. A robot teaching and learning system, characterized in that, The robot teaching learning system includes: a data preprocessing and modal encoding module, a multi-modal teaching signal fusion module, an automatic segmentation and alignment module, a multi-task behavior cloning module, and a self-supervised temporal consistency module. The robot teaching learning system can implement the method described in any one of claims 1 to 8.

10. An electronic device, the electronic device comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method described in any one of claims 1 to 8 when executing the computer program.

Images