Robot teaching learning method, system and electronic equipment

Abstract

The present application discloses a robot teaching learning method, system and electronic equipment, which belongs to the field of robot control technology. The method includes: performing time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, inputting the first teaching signal into a modal encoder to obtain a hidden state feature; performing linear weighting based on the hidden state feature, and generating a unified fusion feature through a nonlinear activation function; based on the unified fusion feature, calculating the minimum cumulative cost through dynamic programming, determining the alignment path and stage boundary, and dividing the unified fusion feature into subtasks based on the alignment path and stage boundary; for the subtask, using weighted behavioral cloning loss to perform model optimization to obtain an optimized model; using self-supervised temporal consistency loss to perform self-supervised learning on the optimized model. The present application improves the robustness and accuracy of robot teaching learning.

Description

Technical Field

[0001] The present application belongs to the field of robot control technology, and in particular relates to a robot teaching and learning method, system, and electronic equipment. Background Art

[0002] Currently, multimodal fusion is one of the core technologies in the fields of artificial intelligence and robotics. However, existing teaching systems rarely integrate multimodal teaching signals. Most of them process the data of each modality in a serial or simple splicing manner, which cannot effectively capture the correlation between different modalities. In addition, complex operations often require the teaching process to be divided into several subtasks or stages (such as grasping, handling, assembly, etc.). However, traditional teaching alignment methods usually require manual labeling or rely on preset similarity metrics for alignment, lacking automation and robustness. Existing algorithms use various information such as speed and force as the basis for segmentation to perform teaching segmentation, but how to achieve accurate alignment in multimodality remains a challenge.

[0003] In response to the above problems, the robot teaching learning method, system and electronic equipment in this application are proposed. Summary of the Invention

[0004] In order to address the deficiencies of the prior art, the present application provides a robot teaching learning method, system and electronic device to solve the problems in the prior art that traditional robot teaching methods usually require manual labeling or rely on preset similarity metrics for alignment, lack automation and robustness, etc.

[0005] The technical effects to be achieved by this application are achieved through the following solutions:

[0006] In a first aspect, the present application provides a robot teaching and learning method, the method comprising:

[0007] Performing time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, and inputting the first teaching signal into a modal encoder to obtain a latent state feature;

[0008] Perform linear weighting based on the hidden state features and generate unified fusion features through a nonlinear activation function;

[0009] Based on the unified fusion feature, a minimum cumulative cost is calculated by dynamic programming to determine an alignment path and a stage boundary, and the unified fusion feature is divided into subtasks based on the alignment path and the stage boundary;

[0010] For the subtask, weighted behavioral cloning loss is used to optimize the training model to obtain an optimized model;

[0011] Self-supervised temporal consistency loss is used to perform self-supervised learning on the optimization model.

[0012] In some embodiments, the latent state feature is , ,in Indicates the modal, Indicates at time , Indicates the Mode at time The first teaching signal, Indicates the The modal encoder corresponding to the mode.

[0013] In some embodiments, the unified fusion feature is , in represents the total number of modes, Indicates the The fusion weight of the modality satisfies , Indicates the The linear mapping matrix of the mode, represents the bias vector, represents a non-linear activation function.

[0014] In some embodiments, the calculating the minimum cumulative cost by dynamic programming based on the unified fusion feature includes:

[0015] Get the cost matrix ,

[0016] Among them, A and B represent two different teaching sequences.

[0017] Indicates the first modal and the moment index is Features,

[0018] Indicates the first modal and the moment index is Features,

[0019] ‖ ‖ represents the modulus of the calculated vector.

[0020] In some embodiments, the calculating the minimum cumulative cost by dynamic programming based on the unified fusion feature further includes:

[0021] Based on the cost matrix Get the minimum cumulative cost The boundary conditions are , for any or , .

[0022] In some embodiments, the weighted behavioral cloning loss is , ,in, Indicates the total number of subtasks,

[0023] Indicates the The time length of each subtask, Indicates the Subtasks at time The true motion vector of Represents the first Subtasks at time The action vector, Indicates the The weight of each subtask satisfies .

[0024] In some embodiments, the self-supervised temporal consistency loss is , ,

[0025] in, represents the length of the unified fusion feature sequence, Indicates time +1 for the unified fusion feature, Represents the time series feature prediction network.

[0026] In some embodiments, A represents a reference teaching sequence, and B represents a teaching sequence to be aligned.

[0027] In the second aspect, the present application provides a robot teaching and learning system, which includes: a data preprocessing and modal encoding module, a multimodal teaching signal fusion module, an automatic segmentation alignment module, a multi-task behavior cloning module and a self-supervised temporal consistency module. The robot teaching and learning system can implement any of the methods described above.

[0028] In a third aspect, the present application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the aforementioned methods when executing the computer program.

[0029] The robot teaching learning method, system and electronic device provided in this application are based on multimodal fusion and improve the robustness and accuracy of robot teaching learning through teaching alignment, multi-task loss design and temporal consistency model optimization and adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 This is a flowchart of a robot teaching and learning method in one embodiment of the present application;

[0032] Figure 2 Schematic diagram of the structure of a robot teaching and learning system in one embodiment of the present application;

[0033] Figure 3 This is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0035] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0036] Related robot teaching and learning methods typically rely on a single modality (e.g., vision or motion recordings alone) for task learning, making it difficult to fully utilize the multi-source information inherent in the teaching process. For example, in the field of industrial collaborative robots, the integration of multimodal information (vision, force perception, voice, etc.) is crucial for fully understanding the teaching intent.

[0037] Furthermore, the behavior cloning method commonly used in demonstration-based learning often uses a unified loss function for different tasks. However, in multi-task or multi-objective scenarios, a single loss function cannot optimize all tasks simultaneously. Traditional mean squared error loss is susceptible to competition between tasks, resulting in poor performance on some tasks. Finally, while demonstration data has significant temporal dependencies, existing learning mechanisms often ignore the self-supervisory constraints of sequence information, making predictions discontinuous or jittery. Introducing temporal consistency in self-supervised learning has been shown to exploit the rich contextual information of time series data, but this has not yet been fully utilized in the demonstration-based learning domain.

[0038] In summary, existing technologies have shortcomings in multimodal fusion, teaching alignment, multi-task loss design and temporal consistency training, and new technical solutions are urgently needed to improve them.

[0039] This application proposes a multimodal robot teaching and learning method for the scenario of industrial collaborative robot teaching and learning.

[0040] The core ideas of this application include:

[0041] 1. Construct a multimodal encoding structure for teaching signals, extract features from teaching signals in multiple modalities such as vision, dynamics, and force perception, and fuse them;

[0042] 2. Design a segmented teaching alignment algorithm to automatically identify the stage boundaries during the teaching process and align the corresponding task segments;

[0043] 3. Introducing a multi-task behavior cloning loss function to simultaneously optimize the behavior cloning objectives of multiple subtasks in a weighted manner;

[0044] 4. A self-supervised temporal consistency training mechanism is adopted to improve the stability and generalization ability of the teaching encoding by constraining the continuity of the teaching features in the time dimension.

[0045] The above-mentioned innovative combinations constitute a multimodal teaching and learning sub-module in the embodied intelligent robot system based on a hybrid memory bank, which improves the efficiency of teaching data utilization and learning effects from the system architecture level.

[0046] This application uses five key modules: data preprocessing and modal encoding module, multimodal teaching signal fusion module, automatic segment alignment module, multi-task behavior cloning module and self-supervised temporal consistency module to form a complete model training process. The overall training includes the following five steps:

[0047] 1. Data preprocessing and modal encoding: Synchronize and clean the collected multimodal signals, including vision, dynamics, force perception, and speech, and use feature encoders to extract primary latent state features.

[0048] 2. Multimodal feature fusion: Fusion of the hidden states of each modality into a unified temporal feature representation according to the weights to generate a fused feature sequence .

[0049] 3. Automatic alignment of segmented teaching: Based on fused features, the dynamic time warping (DTW) method is used to identify stage boundaries during the teaching process and achieve global optimal alignment for multiple teaching sequences.

[0050] 4. Multi-task behavior cloning training: Design a weighted behavior cloning loss function, perform supervised learning on aligned subtask sequences, and optimize network action prediction.

[0051] 5. Self-supervised temporal consistency optimization: Based on behavioral cloning, a temporal prediction network is added to perform self-supervisory constraints on cross-time feature coherence, improving the temporal stability of the model.

[0052] Various non-limiting embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0053] First, refer to Figure 1 , the robot teaching learning method of this application is described in detail.

[0054] The present application provides a robot teaching and learning method, the method comprising:

[0055] S1: performing time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, and inputting the first teaching signal into a modal encoder to obtain a hidden state feature;

[0056] S2: linearly weighting the latent state features and generating a unified fusion feature through a nonlinear activation function;

[0057] S3: Based on the unified fusion feature, calculate the minimum cumulative cost through dynamic programming, determine the alignment path and stage boundaries, and divide the unified fusion feature into subtasks based on the alignment path and stage boundaries;

[0058] S4: For the subtask, use weighted behavior cloning loss to optimize the training model to obtain an optimized model;

[0059] S5: Use self-supervised temporal consistency loss to perform self-supervised learning on the optimized model.

[0060] The above-mentioned robot teaching learning method is based on multimodal fusion, and improves the robustness and accuracy of robot teaching learning through teaching alignment, multi-task loss design and temporal consistency model optimization and adjustment.

[0061] For example, the training model refers to an action prediction model (such as a hierarchical action prediction model based on LSTM, a multi-head attention action prediction model based on Transformer, or a lightweight sequence model based on GRU), which is used for cloning training and can also be called a behavior policy network model.

[0062] In some embodiments, the latent state feature is , ,in Indicates the modal, Indicates at time , Indicates the Mode at time The first teaching signal, Indicates the The modal encoder corresponding to the mode.

[0063] Exemplarily, the first teaching signal may include a visual image , joint angle , terminal force perception , voice commands wait, Indicates at time .

[0064] Exemplarily, the modality encoder may be a convolutional neural network (CNN), a recurrent neural network (RNN), or a pre-trained model.

[0065] The above hidden state features are It provides an accurate basis for subsequent feature fusion.

[0066] In some embodiments, the unified fusion feature is , in represents the total number of modes, Indicates the The fusion weight of the modality satisfies , Indicates the The linear mapping matrix of the mode, represents the bias vector, represents a non-linear activation function.

[0067] For example, Can be achieved through Layers are automatically normalized or dynamically adjusted using learning strategies.

[0068] In some embodiments, the calculating the minimum cumulative cost by dynamic programming based on the unified fusion feature includes:

[0069] Get the cost matrix ,

[0070] Among them, A and B represent two different teaching sequences.

[0071] Indicates the first modal and the moment index is Features,

[0072] Indicates the first modal and the moment index is Features,

[0073] ‖ ‖ represents the modulus of the calculated vector, that is, the norm of the vector.

[0074] For example, Indicates the time index of sequence A, the value range is .

[0075] For example, Indicates the time index of sequence B, the value range is .

[0076] In some embodiments, A represents a reference teaching sequence, such as a standard action; B represents a teaching sequence to be aligned, such as another teaching sequence that needs to be time-aligned with the standard action.

[0077] In some embodiments, the calculating the minimum cumulative cost by dynamic programming based on the unified fusion feature further includes:

[0078] Based on the cost matrix Get the minimum cumulative cost The boundary conditions are , for any or , .

[0079] In some embodiments, the weighted behavioral cloning loss is , ,

[0080] in, Indicates the total number of subtasks, Indicates the The time length of each subtask, Indicates the Subtasks at time The true motion vector of The model predicts the Subtasks at time The action vector, Indicates the The weight of each subtask satisfies .

[0081] For example, the weight It can be determined by Bayesian optimization, uniform initialization, or adaptive algorithms.

[0082] In some embodiments, the self-supervised temporal consistency loss is , ,in, represents the length of the unified fusion feature sequence, Indicates time +1 for the unified fusion feature, Represents the time series feature prediction network.

[0083] For example, the time series feature prediction network can use structures such as fully connected layers and recurrent neural networks (LSTM / GRU).

[0084] Based on the existing multimodal encoding and behavior cloning models, this application introduces a temporal feature prediction network to perform self-supervised learning on continuous fusion features to enhance the temporal coherence of features.

[0085] The robot teaching learning method of the present application not only ensures the accuracy of robot motion prediction, but also improves the timing smoothness and robustness of the teaching coding.

[0086] Second, as Figure 2 As shown, the present application provides a robot teaching and learning system, which includes: a data preprocessing and modal encoding module, a multimodal teaching signal fusion module, an automatic segmentation alignment module, a multi-task behavior cloning module and a self-supervised temporal consistency module. The robot teaching and learning system can implement any of the methods described above.

[0087] The above-mentioned robot teaching and learning system can achieve all the technical effects of the robot teaching and learning method, which will not be repeated here.

[0088] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the described method.

[0089] It should be noted that the above description is of specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0090] Based on the same inventive concept, corresponding to any of the above embodiments and methods, this application also discloses an electronic device. Specifically, Figure 3 The following is a schematic diagram showing the hardware structure of an electronic device for a robot teaching and learning method provided in this embodiment. The device may include: a processor 310, a memory 320, an input / output interface 330, a communication interface 340, and a bus 350. The processor 310, the memory 320, the input / output interface 330, and the communication interface 340 are connected to each other within the device via the bus 350.

[0091] The processor 310 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0092] The memory 320 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 320 can store an operating system and other application programs. When the technical solutions provided in the embodiments of the present application are implemented through software or firmware, the relevant program code is stored in the memory 320 and is called and executed by the processor 310.

[0093] The input / output interface 330 is used to connect to an input / output module to enable information input and output. The input / output module can be configured as a component within the device (not shown) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc. Output devices may include a display, speaker, vibrator, indicator light, etc.

[0094] The communication interface 340 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via wired means (e.g., USB, network cable, etc.) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth, etc.).

[0095] The bus 350 comprises a pathway that transmits information between the various components of the device (eg, the processor 310 , the memory 320 , the input / output interface 330 , and the communication interface 340 ).

[0096] It should be noted that although the above device only shows the processor 310, the memory 320, the input / output interface 330, the communication interface 340, and the bus 350, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in the figure.

[0097] The electronic device of the above embodiment is used to implement the corresponding robot teaching and learning method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0098] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, one or more embodiments of the present application also provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the robot teaching and learning method described in any of the above embodiments.

[0099] The computer-readable media of this embodiment includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0100] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the robot teaching and learning method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0101] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. Within the scope of the present application, the technical features of the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0102] Additionally, to simplify the description and discussion, and to avoid obscuring the understanding of one or more embodiments of the present application, well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. Furthermore, devices may be shown in block diagram form to avoid obscuring the understanding of one or more embodiments of the present application, and this also takes into account the fact that the implementation details of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (i.e., such details should be fully understood by those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it will be apparent to those skilled in the art that one or more embodiments of the present application can be implemented without or with variations in these specific details. Accordingly, these descriptions should be considered illustrative rather than restrictive.

[0103] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the discussed embodiments.

[0104] The one or more embodiments of the present application are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the one or more embodiments of the present application shall be included within the scope of protection of the present application.

Claims

1. A robot teaching learning method, characterized in that: include: Performing time synchronization and denoising processing on the original teaching signal to obtain a first teaching signal, and inputting the first teaching signal into a modal encoder to obtain a latent state feature; Perform linear weighting based on the hidden state features and generate unified fusion features through a nonlinear activation function; Based on the unified fusion feature, a minimum cumulative cost is calculated by dynamic programming to determine an alignment path and a stage boundary, and the unified fusion feature is divided into subtasks based on the alignment path and the stage boundary; For the subtask, weighted behavioral cloning loss is used to optimize the training model to obtain an optimized model; Self-supervised temporal consistency loss is used to perform self-supervised learning on the optimization model.

2. The robot teaching and learning method according to claim 1, characterized in that: The hidden state feature is , ,in Indicates the modal, Indicates at time , Indicates the Mode at time The first teaching signal, Indicates the The modal encoder corresponding to the mode.

3. The robot teaching and learning method according to claim 2, characterized in that: The unified fusion feature is , in represents the total number of modes, Indicates the The fusion weight of the modality satisfies , Indicates the The linear mapping matrix of the mode, represents the bias vector, represents a non-linear activation function.

4. The robot teaching and learning method according to claim 3, characterized in that: The calculating the minimum cumulative cost by dynamic programming based on the unified fusion feature includes: Get the cost matrix , Among them, A and B represent two different teaching sequences. Indicates the first modal and the moment index is Features, Indicates the first modal and the moment index is Features, ‖ ‖ represents the modulus of the calculated vector.

5. The robot teaching and learning method according to claim 4, characterized in that: The calculating the minimum cumulative cost by dynamic programming based on the unified fusion feature further includes: Based on the cost matrix Get the minimum cumulative cost The boundary conditions are , for any or , .

6. The robot teaching and learning method according to claim 5, characterized in that: The weighted behavioral cloning loss is , , in, Indicates the total number of subtasks, Indicates the The time length of each subtask, Indicates the Subtasks at time The true motion vector of Represents the first Subtasks at time The action vector, Indicates the The weight of each subtask satisfies .

7. The robot teaching and learning method according to claim 1, characterized in that: The self-supervised temporal consistency loss is , , in, represents the length of the unified fusion feature sequence, Indicates time The unified fusion characteristics of Represents the time series feature prediction network.

8. The robot teaching and learning method according to claim 4, characterized in that: A represents the reference teaching sequence, and B represents the teaching sequence to be aligned.

9. A robot teaching and learning system, characterized in that: The robot teaching and learning system includes: a data preprocessing and modal encoding module, a multimodal teaching signal fusion module, an automatic segmentation alignment module, a multi-task behavior cloning module and a self-supervised temporal consistency module. The robot teaching and learning system can implement the method described in any one of claims 1 to 8.

10. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images