Entropy-aware robot control method and system based on expert demonstration

Abstract

The application provides an entropy-aware robot control method and system based on expert demonstration, and relates to the field of embodied intelligence technology, which comprises obtaining a task instruction input by a user, observation data and ontology perception data of a robot at a current time; cleaning and feature extracting the data to obtain multiple features, and retrieving a reference video in an expert reference video database; encoding and splicing the multiple features to generate a multi-modal feature representation; feature extracting the reference video to obtain reference video features; processing the multi-modal feature representation and the reference video features through a skill generation model to obtain a discrete skill codebook index probability; calculating an information entropy of the probability, and determining a sampling candidate number of a current candidate skill based on the information entropy; generating a skill token sequence based on the discrete skill codebook index probability and the sampling candidate number; and finally decoding the skill token sequence to obtain a robot action sequence. The control method provided by the application can guarantee accurate control of a robot in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence technology, specifically to an entropy-sensing robot control method and system based on expert demonstrations. Background Technology

[0002] In scenarios such as industrial automation, home services, and medical assistance, robots need to be able to learn fine motor skills from human demonstrations in order to meet the diverse task requirements in open environments. The core challenge lies in how to efficiently model the distribution of multimodal behaviors and long-term dependencies from high-dimensional continuous actions.

[0003] The existing mainstream technology adopts a two-stage architecture of "discretization + autoregressive generation". First, the continuous action is compressed into a discrete codebook index through vector quantization (VQ). Then, the discrete sequence distribution under the observation conditions is modeled by Transformer. Finally, the continuous control instructions are reconstructed by the pre-trained decoder.

[0004] However, the discretization process of the above methods relies on pre-trained static codebooks, which are difficult to dynamically adapt to changes in task semantics or environment, resulting in limited skill generalization ability. In particular, insufficient codebook coverage and semantic mismatch are prone to occur when transferring with few samples or fine-tuning task semantics. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides an entropy-aware robot control method and system based on expert demonstrations, which solves the problem of insufficient robustness of existing robot control methods in real and complex scenarios.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] In a first aspect, the present invention provides an entropy-aware robot control method based on expert demonstration, the method comprising:

[0010] Acquire the task instructions input by the user, as well as the robot's current observation data and proprioceptive perception data;

[0011] The task instructions, observation data, and ontology perception data are cleaned and feature extracted to obtain task instruction features, observation features, and ontology perception features; reference videos are retrieved from the expert reference video database based on the task instructions, observation data, and ontology perception data.

[0012] The task instruction features, observation features, and ontology perception features are encoded and concatenated to generate a multimodal feature representation; feature extraction is performed on the reference video to obtain reference video features;

[0013] The discrete skill codebook index probability is obtained by processing the multimodal feature representation and the reference video features through a skill generation model.

[0014] Calculate the information entropy of the discrete skill codebook index probability, and dynamically determine the number of sampling candidates for the current candidate skill based on the information entropy;

[0015] A skill token sequence is generated based on the discrete skill codebook index probability and the number of sampled candidates;

[0016] The skill token sequence is decoded by a decoder of a pre-trained skill variational autoencoder to obtain a robot action sequence.

[0017] Preferably, the method for constructing the expert reference video database includes:

[0018] Spatiotemporal feature vectors are obtained by processing expert reference demonstration videos in the training set through a pre-trained video representation network.

[0019] The spatiotemporal feature vectors and their corresponding metadata are indexed and stored in a vector database to obtain the expert reference video database.

[0020] Preferably, the skill generation model includes a backbone network and an auxiliary network, and the step of processing the multimodal feature representation and the reference video features through the skill generation model to obtain the discrete skill codebook index probability includes:

[0021] The backbone network uses the multimodal feature representation as a conditional input to obtain a predicted skill token sequence; the auxiliary network processes the reference video features to obtain a residual correction signal.

[0022] The backbone network processes the predicted skill token sequence through a self-attention mechanism to obtain a temporally correlated optimized predicted skill token sequence.

[0023] The backbone network uses a cross-attention mechanism to query relevant reference video features from the reference video features based on the temporally correlated optimized predicted skill token sequence, and then merges the queried relevant reference video features with the temporally correlated optimized predicted skill token sequence to obtain the initial discrete skill codebook index probability.

[0024] The backbone network adjusts the initial discrete skill codebook index probability by superimposing the residual correction signal, thereby obtaining the discrete skill codebook index probability.

[0025] Preferably, the auxiliary network processes the reference video features to obtain the residual correction signal, including:

[0026] High-level guidance signals are extracted from the features of the reference video through deep interaction;

[0027] The residual correction signal is obtained by processing the higher-level guidance signal through a linear layer with zero initialization.

[0028] Preferred options also include:

[0029] Data for robot demonstrations was collected based on a high-fidelity robot simulation environment.

[0030] The robot demonstration data is cleaned to obtain valid demonstration data;

[0031] The effective demonstration data is normalized and features are extracted to obtain a long-term action feature sequence.

[0032] Local feature extraction and temporal downsampling are performed on the long-term action feature sequence to capture long-distance action dependencies and generate low-dimensional latent variables;

[0033] The low-dimensional latent variables are mapped using a finite scalar quantization mechanism to obtain a skill feature sequence;

[0034] The skill feature sequence is decoded and reconstructed to obtain the restored continuous action sequence.

[0035] Preferably, the method includes:

[0036] The skill generation model is trained by the continuous action sequence to obtain the trained skill generation model.

[0037] Preferably, the training objective of the skill generation model includes minimizing a consistency constraint, and the method for constructing the consistency constraint includes:

[0038] The discrete skill codebook index probability is processed by differentiable expected sampling technique to obtain the expected skill feature vector; the corresponding spatiotemporal feature vector is extracted from the expert demonstration reference video;

[0039] Calculate the mean square error of the expected skill feature vector and the spatiotemporal feature vector, and use the mean square error as a consistency constraint for the skill generation model.

[0040] Secondly, the present invention also provides an entropy-sensing robot control system based on expert demonstration, comprising:

[0041] The data acquisition module acquires the task instructions input by the user, as well as the robot's current observation data and proprioceptive perception data;

[0042] The first processing module cleans and extracts features from the task instructions, observation data, and ontology perception data to obtain task instruction features, observation features, and ontology perception features; and retrieves reference videos from the expert reference video database based on the task instructions, observation data, and ontology perception data.

[0043] The second processing module encodes and concatenates the task instruction features, the observation features, and the ontology perception features to generate a multimodal feature representation; and performs feature extraction on the reference video to obtain reference video features.

[0044] The probability generation module processes the multimodal feature representation and the reference video features through the skill generation model to obtain the discrete skill codebook index probability;

[0045] The sampling selection module calculates the information entropy of the discrete skill codebook index probability and dynamically determines the number of sampling candidates for the current candidate skill based on the information entropy.

[0046] The skill generation module generates a skill token sequence based on the discrete skill codebook index probability and the number of sampled candidates;

[0047] The skill decoding module decodes the skill token sequence using a decoder of a pre-trained skill variational autoencoder to obtain a robot action sequence.

[0048] Thirdly, the present invention provides a computer-readable storage medium storing a computer program for entropy-aware robot control based on expert demonstration, wherein the computer program causes a computer to execute the entropy-aware robot control method based on expert demonstration as described above.

[0049] Fourthly, the present invention also provides an electronic device, comprising:

[0050] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing an entropy-aware robot control method based on an expert demonstration as described in any of the entropy examples.

[0051] (III) Beneficial Effects

[0052] This invention provides an entropy-aware robot control method and system based on expert demonstrations. Compared with existing technologies, it has the following advantages:

[0053] 1. This invention designs a dual-stream generation architecture based on Retrieval Augmentation (RAG). It retrieves the reference video that best matches the semantics of the current task from an external knowledge base and uses a specially designed auxiliary network to extract high-level operational logic from the reference video. The auxiliary network performs deep analysis of the reference video and injects correction signals into the backbone network in the form of residuals. This "backbone-auxiliary" decoupling design allows the model to retain basic motion control capabilities while also acquiring the ability to extract important information from the reference video. Even in data-scarce scenarios, the model can infer the correct operational process based solely on the retrieved reference video, greatly reducing the dependence on large-scale labeled data and significantly improving the robustness of the robot control method in real-world complex scenarios.

[0054] 2. This invention proposes to calculate the information entropy of discrete skill codebook index probability and dynamically determine the number of sampling candidates for the current candidate skill based on the information entropy. This entropy-based adaptive sampling mechanism dynamically adjusts the sampling range by calculating the information entropy of the predicted distribution in real time. When the model is "hesitant" (high entropy), it automatically expands the search space to explore feasible paths. When the model is "confident" (low entropy), it automatically converges to the optimal solution to ensure accuracy. Thus, a dynamic balance is achieved between "exploration" and "utilization", which improves both the accuracy and efficiency of robot action generation. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating the entropy-sensing robot control method provided in an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] This application provides an entropy-aware robot control method and system based on expert demonstrations, which solves the problem that existing robots are prone to motion deformation or logical confusion in complex environments. It enables robots to deeply understand task intentions and coherently generate action sequences in a real world full of interference and dynamic changes.

[0059] The technical solution in this application is to solve the above-mentioned technical problems, and the general idea is as follows:

[0060] 1. Current robot control methods struggle to simultaneously achieve both precise motion generation and cross-task versatility. Some methods introduce continuous "offsets" to correct discrete codes in order to ensure smooth and accurate (high-fidelity) motions. While discrete "skill tokens" can improve transferability, they require extremely high codebook utilization and complex causal architectures to maintain motion quality. This approach leads to models becoming overly reliant on task-specific fine-tuning, compromising the versatility of discrete latent variables. Therefore, existing discretization methods are either too coarse, losing motion details, or overly reliant on continuous corrections, sacrificing spatial understanding. They cannot achieve the ideal representation form, like token representations in natural language processing, which possess both high semantic generalization and accurate information reconstruction.

[0061] 2. When dealing with complex tasks requiring long durations and multi-step logic, existing methods face serious error accumulation and "forgetting" problems. The current mainstream strategy is to break down long sequences of actions into short, fixed- or variable-length segments for prediction. However, this segmentation is often based on statistical data rather than the natural boundaries of the task's semantics. The model struggles to establish robust hierarchical connections between "high-level logical planning" (e.g., deciding to open the cabinet before taking the cup) and "low-level action execution" (e.g., specific joint trajectory control). Due to the lack of effective causal reasoning and long-term memory mechanisms, the model is prone to losing contextual information when executing long sequences, leading to action distortion or logical confusion in the later stages of the task, making it difficult to handle long-cycle operations in the real world that are full of interference and dynamic changes.

[0062] To address the shortcomings of existing embodied intelligent policy learning methods in long-range complex tasks, such as low utilization of expert demonstrations, weak cross-task generalization ability, instability in the early training stage, and poor adaptability of inference and decision-making, this application first introduces a zero-initialization auxiliary network module to transform the retrieved reference video features into residual correction signals and inject them into the backbone network, thereby explicitly guiding and correcting the main policy network. Simultaneously, it combines differentiable expectation sampling technology to optimize the consistency between the discrete latent space and the continuous reference video feature space, enhancing the model's semantic understanding of reference actions. Furthermore, to address the issue of confidence fluctuations in the generated model at different operational stages, this invention proposes an adaptive dynamic sampling mechanism based on the information entropy of the prediction distribution. This mechanism can adjust the sampling range in real time according to the uncertainty of the current decision, automatically expanding the search space to avoid modal collapse when the model is "hesitant" (high entropy) and automatically converging to the optimal solution to eliminate random noise when the model is "confident" (low entropy). Through these improvements, this application can significantly improve the robot's cross-task generalization ability in low-sample scenarios while ensuring training stability, and effectively enhance the accuracy and robustness of action generation in long-term complex interactive tasks.

[0063] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0064] like Figure 1 As shown, this embodiment provides an entropy-aware robot control method based on expert demonstration, including the following steps:

[0065] Step S110: Obtain the task instructions input by the user, as well as the robot's current observation data and body perception data;

[0066] Step S120: Clean and extract features from the task instructions, observation data, and ontology perception data to obtain task instruction features, observation features, and ontology perception features; retrieve reference videos from the expert reference video database based on the task instructions, observation data, and ontology perception data.

[0067] Step S130: Encode and concatenate the task instruction features, the observation features, and the ontology perception features to generate a multimodal feature representation; extract features from the reference video to obtain reference video features;

[0068] Step S140: Process the multimodal feature representation and the reference video features through the skill generation model to obtain the discrete skill codebook index probability;

[0069] Step S150: Calculate the information entropy of the discrete skill codebook index probability, and dynamically determine the number of sampling candidates for the current candidate skill based on the information entropy;

[0070] Step S160: Generate a skill token sequence based on the discrete skill codebook index probability and the number of sampling candidates;

[0071] Step S170: The skill token sequence is decoded by the decoder of the pre-trained skill variational autoencoder to obtain the robot action sequence.

[0072] This application's entropy-aware robot control method employs a dual-stream generation architecture based on Retrieval-Augmented Generation (RAG). It retrieves the reference video that best matches the semantics of the current task from an external knowledge base and utilizes a specially designed auxiliary network to extract high-level operational logic from the reference video. The auxiliary network performs deep analysis of the reference video and injects correction signals into the backbone network in the form of residuals. This decoupled "backbone-auxiliary" design allows the model to retain basic motion control capabilities while also acquiring crucial information from the reference video. Even in data-scarce scenarios, the model can infer the correct operational procedures based solely on the retrieved reference video, significantly reducing reliance on large-scale labeled data and greatly improving the robustness of the robot control method in real-world, complex scenarios.

[0073] In one embodiment, step S110 involves acquiring the task instructions input by the user, as well as the robot's current observation data and body perception data.

[0074] It is understood that the input stage of this embodiment consists of the robot's current observation, body perception data, and user-inputted task instructions; wherein, the current observation includes color red-green-blue images of the robot's current third-person perspective (panoramic view) and hand-eye perspective (end-effector view); the body perception data includes the current joint angles and end-effector coordinates; the task instructions include receiving natural language instructions (exemplary, such as placing the target object in the target position).

[0075] Specifically, the visual observation data includes three perspectives: color red-green-blue images, namely the third-person perspective and the hand-eye perspective, with the resolution uniformly adjusted to 128×128 pixels. The low-dimensional proprioception data includes the robot's joint states (7-dimensional joint angles), end effector positions (3-dimensional coordinates), and gripper states (2-dimensional opening and closing), totaling 12-dimensional vectors, used to accurately describe the robot's physical state.

[0076] In one embodiment, step S120 involves cleaning and extracting features from the task instructions, observation data, and ontology perception data to obtain task instruction features, observation features, and ontology perception features; and retrieving reference videos from the expert reference video database based on the task instructions, observation data, and ontology perception data.

[0077] In this embodiment, pre-trained visual encoders and language encoders are used to extract features, obtaining visual encoded features and task instruction features. Specifically, a multilayer perceptron is used to map the current 12-dimensional ontology perception data into ontology perception features; a pre-trained text encoder is used to encode task instructions into task features; and a visual encoder is used to extract current image features and project them into visual features.

[0078] In this embodiment, the system, based on the task instructions and the current observation field, retrieves a reference video (e.g., another video demonstrating the capture of an object and its placement at a target location) from the expert reference video database using a vector index, and extracts the feature sequence of this reference video as external memory. In this embodiment, the method for constructing the expert reference video database includes:

[0079] Step S121: Process the expert reference demonstration videos in the training set through the pre-trained video representation network to obtain spatiotemporal feature vectors;

[0080] Step S122: The spatiotemporal feature vector and its corresponding metadata are indexed and stored in the vector database to obtain the expert reference video database.

[0081] This embodiment performs feature extraction and index construction. It uses a pre-trained video representation network to process all expert reference demonstration videos in the training set, extracts spatiotemporal feature vectors containing rich semantic and motion information, and indexes these high-dimensional features and their corresponding metadata and stores them in an efficient vector database, thereby completing the construction of the expert reference video database.

[0082] Understandably, a real-time retrieval mechanism is established concurrently with the construction of the expert reference video database. Based on the natural language semantic description of the current task or the similarity of the current visual observation, the system quickly retrieves the most matching and relevant reference video from the vector database by calculating metrics such as cosine similarity. Then, feature caching and memory injection are performed to extract the feature sequences corresponding to the retrieved reference videos and encapsulate them as "reference memory." This memory is input into the cross-attention layer of the generative model as an external knowledge source, thereby providing the model with explicit reference and guidance on operation procedures, object interaction methods, and motion trajectories in long-term tasks.

[0083] The robot control method in this embodiment constructs an external reference video knowledge base based on retrieval enhancement (RAG), which provides specific correction signals and explicit guidance for the robot's motion generation process, enhancing the model's generalization ability in unknown scenarios.

[0084] In one embodiment, step S130 involves encoding and concatenating the task instruction features, the observation features, and the ontology perception features to generate a multimodal feature representation; and extracting features from the reference video to obtain reference video features.

[0085] In this embodiment, the task instruction features, ontology perception features, and visual observation features at the current moment are concatenated to form a basic contextual feature representation, which serves as the input to the skill generation model. Feature extraction is performed on the retrieved reference video to obtain reference video features. It is important to clarify that these reference video features include both valid and invalid features. In the backbone network of the skill generation model, a cross-attention mechanism is used to further filter valid features, preventing invalid features from affecting the results.

[0086] In one embodiment, step S140 involves processing the multimodal feature representation and the reference video features through a skill generation model to obtain discrete skill codebook index probabilities; the skill generation model includes a backbone network and an auxiliary network; the specific implementation of this embodiment includes the following steps:

[0087] The skill generation model includes a backbone network and an auxiliary network. The process of processing the multimodal feature representation and the reference video features through the skill generation model to obtain discrete skill codebook index probabilities includes:

[0088] In step S141, the backbone network uses the multimodal feature representation as a conditional input to obtain a predicted skill token sequence; the auxiliary network processes the reference video features to obtain a residual correction signal.

[0089] Step S142: The backbone network processes the predicted skill token sequence through a self-attention mechanism to obtain a temporally correlated optimized predicted skill token sequence.

[0090] Step S143: The backbone network queries relevant reference video features in the reference video features based on the temporal correlation optimized prediction skill token sequence through a cross-attention mechanism, and fuses the queried relevant reference video features with the temporal correlation optimized prediction skill token sequence to obtain the initial discrete skill codebook index probability.

[0091] In step S144, the backbone network adjusts the initial discrete skill codebook index probability by superimposing the residual correction signal to obtain the discrete skill codebook index probability.

[0092] The auxiliary network processes the reference video features to obtain the residual correction signal, including:

[0093] Step S1411: Extract high-level guidance signals from the reference video features through deep interaction;

[0094] It is important to clarify that deep interaction represents the extraction of feature information across multiple layers, time sequences, and segments. Deep representation can focus on the relationships between frames, the sequence of actions, and the connections between skills. High-level guidance signals are abstract, semantic, and task-level guidance information extracted after deep interaction. In contrast, low-level guidance signals can only simply express the color, outline, joint position, and simple motion trajectory of a frame, while high-level guidance signals can describe the overall task objective of the reference video, the process and sequence of skill execution, the methods and constraints of object interaction, and the intent and specifications of actions.

[0095] Step S1412: Process the higher-level guidance signal through a zero-initialized linear layer to obtain the residual correction signal.

[0096] The skill generation model in this embodiment processes multimodal feature representations and reference video features through a two-stream network. The backbone network of the skill generation model receives the multimodal feature representation of the concatenated sequence to generate a predicted skill token sequence. Then, it integrates contextual information into the generated predicted skill token sequence through a self-attention mechanism, while simultaneously querying and filtering effective features (i.e., relevant reference video features) from the reference video features through a cross-attention mechanism. Finally, it fuses the features to obtain the initial discrete skill codebook index probability. The auxiliary network receives the retrieved reference video features and analyzes the action patterns in the reference video (such as approaching an object, grabbing, moving, and releasing). It extracts high-level guidance signals from the reference video features through deep interaction, processes the high-level guidance signals through a zero-initialization linear layer, and obtains residual correction signals.

[0097] The auxiliary network outputs a residual correction vector (equivalent to the features extracted by the auxiliary network telling the backbone network: "In the reference video, this step is to raise the robotic arm, please correct your trajectory"). This vector is added to the output features of the backbone network, and finally the skill generation model outputs the discrete skill codebook index probability.

[0098] The backbone network in this embodiment adopts a standard Transformer decoder architecture. It uses a self-attention mechanism to process historically generated skill feature sequences and a cross-attention mechanism to query reference video memories, outputting preliminary skill prediction probabilities, i.e., initial discrete skill codebook index probabilities. This embodiment designs a lightweight parallel Transformer decoder branch as an auxiliary network. This branch is dedicated to receiving reference video features, extracting high-level guidance signals through deep interaction, and generating residual correction signals through a zero-initialization linear layer.

[0099] This embodiment superimposes the output of the auxiliary network onto the feature space of the backbone network and employs a zero-initialization design in the terminal projection layer where the features of the auxiliary and backbone networks are fused. The core mechanism of this design is as follows: In the initial stage of model training, by forcibly initializing the weights and biases of this terminal linear layer to zero, the initial features input from the auxiliary network to the backbone network are strictly zero. This ensures that no random, misaligned noise is injected into the backbone network, which already possesses strong prior knowledge, when the auxiliary network is added, guaranteeing absolute stability in the early stages of training. As backpropagation progresses, although the initial forward output of this layer is zero, the gradient signal it receives is non-zero. Under the action of the optimizer, the weights of this zero-initialized layer are smoothly and progressively updated from zero. This adaptive "wake-up" mechanism allows the backbone network to gradually accept and fuse the reference trajectory guidance information provided by the auxiliary network at a stable pace. Ultimately, without compromising the existing generalization ability of the backbone network, robust and precise intervention control of the reference trajectory for robot motion generation is achieved.

[0100] In this embodiment, the skill generation model is trained using a sequence of continuous actions extracted from a demonstration video. The steps for obtaining the continuous action sequence include:

[0101] Step S210: Collect robot demonstration data based on a high-fidelity robot simulation environment;

[0102] Step S220: Clean the robot demonstration data to obtain valid demonstration data;

[0103] In this embodiment, to ensure the robustness of skill learning, a strict cleaning process is performed on the collected demonstration data. Data will be removed or truncated if it meets any of the following conditions: (1) the task was not successfully completed and cannot be used as a positive demonstration sample; (2) the robot's motion trajectory has singular points or violent jitters, which does not meet the smooth kinematic constraints; (3) key visual observations are severely obstructed, making it impossible to infer the current state from the image; (4) operation steps are missing or there is redundant waiting time, resulting in an excessively high entropy value of the action sequence and no practical operation significance.

[0104] Step S230: Normalize the valid demonstration data and extract features to obtain a long-term action feature sequence;

[0105] For the retained valid data, normalization is used to map the action space to a standard range, and features are extracted using pre-trained visual encoders and language encoders to obtain long-term action feature sequences.

[0106] Step S240: Local feature extraction and temporal downsampling are performed on the long-term action feature sequence to capture long-distance action dependencies and generate low-dimensional latent variables;

[0107] Step S250: The low-dimensional latent variables are mapped using a finite scalar quantization mechanism to obtain a skill feature sequence;

[0108] Step S260: Decode and reconstruct the skill feature sequence to obtain the restored continuous action sequence.

[0109] To address the high-dimensionality and multimodal nature of continuous action spaces, this invention presents a skill extraction module based on a variational autoencoder. This module consists of an encoder, a quantizer, and a decoder. The encoder employs a hybrid convolutional-Transformer architecture. First, residual temporal convolutional blocks are used to extract local features and temporally downsample the input long-term action sequence. Then, a Transformer encoder is used to capture long-distance action dependencies, generating low-dimensional latent variables. The quantizer introduces a finite scalar quantization mechanism to map continuous latent variables to discrete codebook indices. These discrete indices are defined as "skill feature sequences," forming the robot's "skill vocabulary." The decoder uses a Transformer decoder to reconstruct the original continuous action sequence based on the discrete skill feature sequences. The training objective of the skill extraction module is to minimize the reconstruction error, ensuring that the discrete feature sequences retain the semantics and details of the original actions without loss.

[0110] In this embodiment, after obtaining the continuous action sequence, the skill generation model (autoregressive Transformer model) is trained using the continuous action sequence to obtain the trained skill generation model. Understandably, the skill extraction module converts all actions in the video into discrete sequences as real labels, and then trains the skill generation model (Autoregressive Transformer model) to predict these sequences.

[0111] In this embodiment, the training objective of the skill generation model includes minimizing consistency constraints, and the method for constructing the consistency constraints includes:

[0112] Step S310: Process the discrete skill codebook index probability using differentiable expected sampling technology to obtain the expected skill feature vector; extract the corresponding spatiotemporal feature vector from the expert demonstration reference video;

[0113] Step S320: Calculate the mean square error of the expected skill feature vector and the spatiotemporal feature vector, and use the mean square error as a consistency constraint for the skill generation model.

[0114] In this embodiment, the system extracts the spatiotemporal feature vectors of real reference videos retrieved from the expert reference video library and calculates the mean squared error between these vectors and the previously generated expected skill feature vectors. This mean squared error is added to the overall training objective of the model as an auxiliary constraint loss function. By minimizing this consistency constraint, the loss of semantic information due to discrete quantization is effectively avoided, forcing the generative model to maintain a high degree of consistency between the generated action intent and the explicit guidance signal of the external reference video during multimodal decision-making.

[0115] In the model training phase of this embodiment, a differentiable expectation sampling mechanism is introduced to construct a gradient path from discrete probability distributions to continuous video features, forcing the generated actions to remain semantically consistent with the reference video. This strategy significantly improves the robot's success rate in complex interactive environments.

[0116] In one embodiment, step S150 involves calculating the information entropy of the discrete skill codebook index probability and dynamically determining the number of sampling candidates for the current candidate skill based on the information entropy. The specific implementation of this embodiment includes the following steps:

[0117] Step S151: During the process of generating the skill feature sequence through autoregression, the information entropy of the predicted distribution at the current time step is calculated in real time.

[0118] Step S152: Establish a positive correlation mapping function between entropy value and sampling parameter k. When the entropy value is low (model is confident), decrease the value of k to utilize high-confidence actions; when the entropy value is high (model is hesitant / multimodal), increase the value of k to maintain the diversity of actions.

[0119] In the entropy adaptive sampling decision-making stage of this embodiment, the model first outputs the log-probability of the probability distribution of the next skill token and calculates the information entropy of the distribution in real time to quantify the uncertainty of the current decision. The system dynamically determines the current number of sampling candidates based on the preset mapping relationship and truncation function (such as the truncation value after the sampling parameter is equal to the entropy value multiplied by the coefficient). Specifically, when the robot is in the posture adjustment stage before grasping an object, due to the presence of multiple potential approach paths, the model predicts a relatively flat distribution, resulting in a high calculated entropy value (e.g., an entropy value of 3.5). At this time, the algorithm automatically calculates a larger sampling range parameter (e.g., 35) according to the formula, allowing the model to explore and sample the top 35 candidate skill features with the highest probability, thereby preserving the diversity of action paths. In contrast, if the robot is currently in the fixed straight-line path stage after grasping, the model predicts more confidently, resulting in a lower entropy value (e.g., an entropy value of 0.5). The sampling range parameter will be automatically reduced to a preset lower limit (e.g., 5). By forcibly narrowing the selection range, low-probability long-tail noise interference is eliminated, thereby significantly ensuring the accuracy and smoothness of the final action while ensuring the flexibility of the action.

[0120] This embodiment designs an algorithm for dynamic sampling during model inference, which effectively avoids the pattern collapse or long-tailed noise sampling problems caused by fixed k values, significantly improves the success rate of long-term tasks, and solves the problem of large uncertainty differences in the model at different decision stages.

[0121] In one embodiment, step S160 involves generating a skill token sequence based on the discrete skill codebook index probability and the number of sampled candidates;

[0122] In this embodiment, a skill token sequence of length L is generated through an autoregressive process.

[0123] In one embodiment, step S170 involves decoding the skill token sequence using a decoder of a pre-trained skill variational autoencoder to obtain a robot action sequence.

[0124] In this embodiment, the discrete sequence is input into the decoder of a pre-trained skill variational autoencoder; then, the decoder maps the discrete features back to a continuous 7-dimensional action sequence (including joint control signals for future T steps); finally, the action sequence is sent to the robot controller for execution to complete the task. It should be noted that the decoder is built on the Transformer architecture.

[0125] This invention also provides an entropy-aware robot control system based on expert demonstration, comprising:

[0126] The data acquisition module acquires the task instructions input by the user, as well as the robot's current observation data and proprioceptive perception data;

[0127] The first processing module cleans and extracts features from the task instructions, observation data, and ontology perception data to obtain task instruction features, observation features, and ontology perception features; and retrieves reference videos from the expert reference video database based on the task instructions, observation data, and ontology perception data.

[0128] The second processing module encodes and concatenates the task instruction features, the observation features, and the ontology perception features to generate a multimodal feature representation; and performs feature extraction on the reference video to obtain reference video features.

[0129] The probability generation module processes the multimodal feature representation and the reference video features through the skill generation model to obtain the discrete skill codebook index probability;

[0130] The sampling selection module calculates the information entropy of the discrete skill codebook index probability and dynamically determines the number of sampling candidates for the current candidate skill based on the information entropy.

[0131] The skill generation module generates a skill token sequence based on the discrete skill codebook index probability and the number of sampled candidates;

[0132] The skill decoding module decodes the skill token sequence using a decoder of a pre-trained skill variational autoencoder to obtain a robot action sequence.

[0133] It is understood that the entropy-sensing robot control system based on expert demonstration provided in this embodiment of the invention corresponds to the entropy-sensing robot control method based on expert demonstration described above. The explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding content in the entropy-sensing robot control method based on expert demonstration, and will not be repeated here.

[0134] This invention also provides a computer-readable storage medium storing a computer program for an entropy-aware robot control method based on an expert demonstration, wherein the computer program causes a computer to execute the entropy-aware robot control method based on the expert demonstration as described above.

[0135] This application also provides an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing the entropy-aware robot control method based on expert demonstration as described above.

[0136] In summary, compared with existing technologies, it has the following beneficial effects:

[0137] 1. This invention first uses a hybrid architecture based on variational autoencoders to compress and quantize high-dimensional, continuous robot action sequences into discrete "skill token" sequences, constructing a robot skill vocabulary. Then, it utilizes a Transformer-based autoregressive network to learn the temporal dependencies between skills. This mechanism effectively solves the problem of error accumulation in long-term tasks, enabling the robot to generate complex operational actions like generating language, significantly improving the robustness and expressiveness of the policy in multi-stage tasks.

[0138] 2. This invention overcomes the limitation of relying solely on current observations for decision-making by designing a parallel "backbone-auxiliary" dual-stream generation architecture. This architecture introduces an external knowledge base retrieval mechanism to retrieve relevant reference video features in real time based on task semantics. A dedicated auxiliary network is designed to deeply extract operational patterns from the reference videos, and a correction signal is injected into the backbone generation network as residuals through a zero-initialization layer. This design enables the model to significantly enhance its adaptability to new tasks and scenarios without retraining, utilizing reference videos.

[0139] 3. This invention proposes to calculate the information entropy of the discrete skill codebook index probability and dynamically determine the number of sampling candidates for the current candidate skill based on the information entropy. This entropy-based adaptive sampling mechanism dynamically adjusts the sampling range by calculating the information entropy of the predicted distribution in real time. When the model is "hesitant" (high entropy), it automatically expands the search space to explore feasible paths. When the model is "confident" (low entropy), it automatically converges to the optimal solution to ensure accuracy. Thus, a dynamic balance is achieved between "exploration" and "utilization", which improves both the accuracy and efficiency of robot action generation.

[0140] 4. The differentiable expected sampling and consistency loss introduced in the training phase of this invention successfully establish a differentiable gradient path between discrete action distribution and continuous video features, forcing the actions generated by the model to be strictly aligned with the reference video in semantic space. This mechanism not only theoretically solves the problem of discrete-continuous modal alignment, but also significantly reduces collisions or operation interruptions caused by sampling randomness in practice, making the robot's action decision-making more human-like, intelligent, safe, and reliable.

[0141] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An entropy-sensing robot control method based on expert demonstration, characterized in that, include: Acquire user-inputted task commands, as well as the robot's current observation and proprioceptive data; The task instructions, the observation data, and the ontology perception data are cleaned and feature extracted to obtain task instruction features, observation features, and ontology perception features. Reference videos are retrieved from the expert reference video database based on the task instructions, observation data, and ontology perception data. The task instruction features, observation features, and ontology perception features are encoded and concatenated to generate a multimodal feature representation; feature extraction is performed on the reference video to obtain reference video features; The discrete skill codebook index probability is obtained by processing the multimodal feature representation and the reference video features through a skill generation model. Calculate the information entropy of the discrete skill codebook index probability, and dynamically determine the number of sampling candidates for the current candidate skill based on the information entropy; A skill token sequence is generated based on the discrete skill codebook index probability and the number of sampled candidates; The skill token sequence is decoded by a decoder of a pre-trained skill variational autoencoder to obtain a robot action sequence.

2. The entropy-sensing robot control method according to claim 1, characterized in that, The method for constructing the expert reference video database includes: Spatiotemporal feature vectors are obtained by processing expert reference demonstration videos in the training set through a pre-trained video representation network. The spatiotemporal feature vectors and their corresponding metadata are indexed and stored in a vector database to obtain the expert reference video database.

3. The entropy-sensing robot control method according to claim 1, characterized in that, The skill generation model includes a backbone network and an auxiliary network. The process of processing the multimodal feature representation and the reference video features through the skill generation model to obtain discrete skill codebook index probabilities includes: The backbone network uses the multimodal feature representation as a conditional input to obtain a predicted skill token sequence; the auxiliary network processes the reference video features to obtain a residual correction signal. The backbone network processes the predicted skill token sequence through a self-attention mechanism to obtain a temporally correlated optimized predicted skill token sequence. The backbone network uses a cross-attention mechanism to query relevant reference video features from the reference video features based on the temporally correlated optimized predicted skill token sequence, and then merges the queried relevant reference video features with the temporally correlated optimized predicted skill token sequence to obtain the initial discrete skill codebook index probability. The backbone network adjusts the initial discrete skill codebook index probability by superimposing the residual correction signal, thereby obtaining the discrete skill codebook index probability.

4. The entropy-sensing robot control method according to claim 3, characterized in that, The auxiliary network processes the reference video features to obtain a residual correction signal, including: High-level guidance signals are extracted from the features of the reference video through deep interaction; The residual correction signal is obtained by processing the higher-level guidance signal through a linear layer with zero initialization.

5. The entropy-sensing robot control method according to claim 1, characterized in that, Also includes: Data for robot demonstrations was collected based on a high-fidelity robot simulation environment. The robot demonstration data is cleaned to obtain valid demonstration data; The effective demonstration data is normalized and features are extracted to obtain a long-term action feature sequence. Local feature extraction and temporal downsampling are performed on the long-term action feature sequence to capture long-distance action dependencies and generate low-dimensional latent variables; The low-dimensional latent variables are mapped using a finite scalar quantization mechanism to obtain a skill feature sequence; The skill feature sequence is decoded and reconstructed to obtain the restored continuous action sequence.

6. The entropy-sensing robot control method according to claim 5, characterized in that, include: The skill generation model is trained by the continuous action sequence to obtain the trained skill generation model.

7. The entropy-sensing robot control method according to claim 1, characterized in that, The training objective of the skill generation model includes minimizing consistency constraints, and the methods for constructing the consistency constraints include: The discrete skill codebook index probability is processed by differentiable expected sampling technique to obtain the expected skill feature vector; the corresponding spatiotemporal feature vector is extracted from the expert demonstration reference video; Calculate the mean square error of the expected skill feature vector and the spatiotemporal feature vector, and use the mean square error as a consistency constraint for the skill generation model.

8. An entropy-sensing robot control system based on expert demonstration, characterized in that, include: The data acquisition module acquires the task instructions input by the user, as well as the robot's current observation data and proprioceptive perception data; The first processing module cleans and extracts features from the task instructions, the observation data, and the ontology perception data to obtain task instruction features, observation features, and ontology perception features. Reference videos are retrieved from the expert reference video database based on the task instructions, observation data, and ontology perception data. The second processing module encodes and concatenates the task instruction features, the observation features, and the ontology perception features to generate a multimodal feature representation; and performs feature extraction on the reference video to obtain reference video features. The probability generation module processes the multimodal feature representation and the reference video features through the skill generation model to obtain the discrete skill codebook index probability; The sampling selection module calculates the information entropy of the discrete skill codebook index probability and dynamically determines the number of sampling candidates for the current candidate skill based on the information entropy. The skill generation module generates a skill token sequence based on the discrete skill codebook index probability and the number of sampled candidates; The skill decoding module decodes the skill token sequence using a decoder of a pre-trained skill variational autoencoder to obtain a robot action sequence.

9. A computer-readable storage medium, characterized in that, It stores a computer program for entropy-aware robot control based on expert demonstrations, wherein the computer program causes a computer to execute the entropy-aware robot control method based on expert demonstrations as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs including methods for performing the entropy-aware robot control method based on expert demonstration as described in any one of claims 1 to 7.

Images