Embodied robot action execution method and system based on confidence adaptive prior guidance

Abstract

The application relates to a method and system for embodied robot action execution based on confidence adaptive prior guidance. The method comprises: acquiring current observation information of an embodied robot at a current time for a to-be-executed task; generating a current search key based on the current observation information; acquiring a preset success memory library and a preset failure memory library; determining a target success action prior associated with the current observation information from the success memory library based on the current search key, and determining a target confidence corresponding to the target success action prior based on the current search key; determining a target failure exclusion item based on the current search key and the failure memory library; and performing prior guidance on an action generation process of an action generation model based on a OneModel architecture for the to-be-executed task based on the target success action prior, the target confidence and the target failure exclusion item, to obtain a target execution action.

Description

Technical Field

[0001] This application relates to the field of intelligent embodied robot technology, and in particular to a method and system for embodied robot motion execution based on confidence-adaptive prior guidance. Background Technology

[0002] Visual-Language-Motion (VLA) models can generate robot actions based on multi-view images, robot body states, and verbal commands. Generative motion models can also generate continuous action blocks through diffusion models or flow matching models. While these models have potential applications in general robot operations, they still suffer from closed-loop execution instability during real-world deployments. For example, due to factors such as visual noise, viewpoint shift, calibration errors, local state drift, and accumulated motion errors, robots may fail even in similar scenarios.

[0003] Currently, related technologies typically employ a "generate-after-select" paradigm, which involves first generating multiple candidate actions and then selecting the better action through scoring or a validator. Since similar observation states may correspond to different action requirements at different task stages, existing methods may still suffer from unstable action generation, repeated triggering of erroneous actions, and decreased task success rates, resulting in low action execution stability and success rates in long-duration, multi-stage tasks. Summary of the Invention

[0004] In view of the above, this application provides a method, system, device and storage medium for embodied robot action execution based on confidence-adaptive prior guidance, the purpose of which is to solve the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for executing actions of an embodied robot based on confidence-adaptive prior guidance, the method comprising: Obtain the current observation information of the embodied robot regarding the task to be performed at the current moment, and generate the current search key based on the current observation information; Obtain a preset success memory bank and a preset failure memory bank; the success memory bank includes multiple prior candidate successful actions that were executed successfully in the past, and the failure memory bank includes multiple prior candidate failure actions that were executed unsuccessfully in the past. Based on the current retrieval key, determine the target successful action prior associated with the current observation information from the successful memory, and determine the target confidence corresponding to the target successful action prior based on the current retrieval key; Based on the current search key and the failure memory, determine the target failure exclusion item; Based on the target confidence, action sampling parameters are determined regarding the prior of the successful action of the target. Random action noise is obtained, and the random action noise and the prior of the successful action of the target are fused based on the action sampling parameters to obtain the initial execution action. The action sampling parameters are used to characterize the guiding strength of the prior of the successful action of the target on the action generation process of the action generation model. Based on the initial execution action and the target failure exclusion term, the action generation model is given prior guidance for the action generation process of the task to be executed, so as to obtain the target execution action.

[0006] Secondly, this application provides a confidence-adaptive prior guidance-based embodied robot motion execution system, which includes: The first acquisition module is used to acquire the current observation information of the embodied robot for the task to be performed at the current moment, and generate the current search key based on the current observation information; The second acquisition module is used to acquire a preset success memory bank and a preset failure memory bank; the success memory bank includes multiple candidate successful actions prior to historical execution, and the failure memory bank includes multiple candidate failed actions prior to historical execution. The first determining module is used to determine, based on the current retrieval key, the target successful action prior associated with the current observation information from the successful memory, and to determine the target confidence level corresponding to the target successful action prior based on the current retrieval key; The second determining module is used to determine the target failure exclusion item based on the current search key and the failure memory. The generation module is used to determine action sampling parameters about the prior of the successful action of the target based on the target confidence, obtain random action noise, fuse the random action noise and the prior of the successful action of the target based on the action sampling parameters to obtain an initial execution action, and provide prior guidance to the action generation model for the action generation process of the task to be executed based on the initial execution action and the target failure exclusion term to obtain the target execution action; the action sampling parameters are used to characterize the guidance strength of the prior of the successful action of the target on the action generation process of the action generation model.

[0007] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the embodied robot action execution method based on confidence-adaptive prior guidance as described in any embodiment of the first aspect.

[0008] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the embodied robot action execution method based on confidence-adaptive prior guidance as described in any embodiment of the first aspect.

[0009] The technical solutions provided in this application have the following advantages compared with the prior art: (1) Successful experiences in the historical task execution process of the embodied robot are transformed into reusable prior guidance, while failure experiences are transformed into exclusionary guidance. Both of these improve the stability and success rate of action execution in long-term, multi-stage tasks. (2) Through confidence-based prior guidance, successful experiences are fully utilized in high-reliability retrieval, while the model's free generation capability is preserved in low-reliability retrieval. (3) Through the bidirectional constraint of successful priors bringing the model closer and failure exclusionary terms pushing it away, the generated actions are both advantageous and disadvantageous, effectively improving the closed-loop execution robustness of the embodied robot. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a preferred embodiment of the embodied robot action execution method based on confidence-adaptive prior guidance in this application; Figure 2 This is an exemplary flowchart illustrating the determination of prior successful actions for a target, according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the determination of target failure exclusion items according to some embodiments of this application; Figure 4 This is an exemplary flowchart illustrating the determination of target execution actions according to some embodiments of this application; Figure 5 This is an exemplary flowchart illustrating the determination of target execution actions according to other embodiments of this application; Figure 6This is an exemplary flowchart illustrating the generation of the current search key according to some embodiments of this application; Figure 7 This is an exemplary flowchart illustrating the successful construction of a memory bank according to some embodiments of this application; Figure 8 This is an exemplary flowchart illustrating the construction of a successful memory bank according to other embodiments of this application. Figure 9 This is a schematic diagram of a preferred embodiment of the embodied robot motion execution system based on confidence-adaptive prior guidance of this application; Figure 10 This is a schematic diagram of a preferred embodiment of the electronic device of this application; The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0015] Reference Figure 1 The diagram shown is a flowchart illustrating an embodiment of the embodied robot action execution method based on confidence-adaptive prior guidance according to this application. The method is executed by an electronic device, which can be implemented by a software system and / or a hardware system. This embodied robot action execution method based on confidence-adaptive prior guidance includes: Step 101: Obtain the current observation information of the embodied robot for the task to be performed at the current moment, and generate the current search key based on the current observation information.

[0016] Current observation information refers to the multimodal raw data acquired by the embodied robot at the current moment through its various onboard sensors as it perceives its physical environment. For example, current observation information may include current scene images captured by a camera, scene depth maps acquired by a depth camera, body state data such as the angles of each joint of the embodied robot and the pose of its end effector, the opening and closing degree of the gripper, and natural language task commands issued by the user. In some embodiments, current observation information includes a visual image of clothing to be folded on a clothes rack and the opening and closing state of the gripper at the end of the embodied robot's robotic arm.

[0017] The current search key refers to the feature vector representation used to retrieve similar historical states in the memory.

[0018] In some embodiments, the task to be performed is a clothing folding task. A clothing folding task refers to the task in which the embodied robot arranges clothing into a neat folded shape through a series of operations.

[0019] In some embodiments, the processor generates the current search key by: performing block embedding processing on the image of each viewpoint using the visual encoder of the visual language action model to obtain a set of block features carrying spatial location encoding; rearranging the block features of each viewpoint into a two-dimensional grid according to their spatial location; performing average pooling downsampling on the two-dimensional grid of each viewpoint at a certain step size to reduce spatial resolution and enhance the translation invariance of features; flattening the pooled features of each viewpoint into a one-dimensional vector; concatenating the one-dimensional vectors of all viewpoints according to a preset viewpoint order; and performing normalization on the concatenated multi-view fusion vector to obtain the current search key.

[0020] For more information on generating the current search key, please see the following text. Figure 6 And its related descriptions.

[0021] Step 102: Obtain the preset success memory bank and the preset failure memory bank.

[0022] A success memory is a dataset used to store key observational action segments that drive a robot's task progress through its historical successful trajectories. The success memory includes multiple candidate successful action priors from past successful executions. A candidate successful action prior is a single historical action segment stored in the success memory; each historical action segment contains a sequence of robot actions over multiple consecutive time steps, such as gripper displacement, rotation angle, and opening / closing control variables.

[0023] A failure memory is a dataset used to store observed action segments in a robot's historical trajectory that caused significant regressions in task progress. The failure memory includes multiple candidate failure action priors from past executions. Each candidate failure action prior refers to a single failed action segment stored in the failure memory and its corresponding risk score.

[0024] In some embodiments, obtaining preset success and failure memories includes loading pre-built memory files from an external storage medium.

[0025] For details on how the memory is constructed, please refer to the following text. Figure 7 And its related descriptions.

[0026] Step 103: Based on the current retrieval key, determine the target successful action prior associated with the current observation information from the success memory, and determine the target confidence corresponding to the target successful action prior based on the current retrieval key.

[0027] Prior successful actions refer to reference action templates retrieved from the success memory to guide the generation of the current action.

[0028] Target confidence is a quantitative score used to characterize the prior reliability of actions leading to the success of a target.

[0029] In some embodiments, the processor determines the target successful action prior and target confidence by: calculating the cosine similarity between the current search key and the historical search key corresponding to each candidate successful action prior in the success memory; selecting the top N similarity values ​​(N is a positive integer, such as 5, 10, 20, etc.) as an initial selection set, and discarding weakly related candidates with similarity values ​​below a first threshold; for candidate action blocks in the initial selection set, calculating the multivariate dynamic time-normalized distance between every two candidate action blocks; for each candidate action block, using the median distance between it and other candidate action blocks as the trajectory inconsistency score of that candidate, and discarding candidates with inconsistency scores greater than a preset outlier threshold; and for the remaining candidate action blocks, performing a weighted summation using the normalized value of the similarity corresponding to each candidate scaled by a temperature coefficient as the weight, to obtain the target successful action prior.

[0030] In other embodiments, the processor's determination of the target's successful action prior and target confidence may further include: performing batch matrix multiplication operations on the current search key and historical search keys in the success memory to obtain all similarities at once; using an approximate nearest neighbor index structure to accelerate retrieval and returning the top N candidates with the highest similarity, without calculating all candidates; using Euclidean distance instead of multivariate dynamic time warping distance to determine trajectory consistency for the returned candidates, thereby reducing computational complexity; and aligning the remaining candidate action blocks according to timestamps and taking the median of each dimension to obtain the target's successful action prior.

[0031] Further explanation regarding the determination of prior actions for successful target acquisition and target confidence levels can be found later in this text. Figure 2 And its related descriptions.

[0032] Step 104: Based on the current search key and the failure memory, determine the target failure exclusion item.

[0033] Target failure exclusion refers to guiding information used during action generation to push the generated trajectory away from historical failure patterns. Target failure exclusion can include a set of candidate failure exclusions, the exclusion weight of each candidate, and the failure relevance of the current state relative to the failure memory.

[0034] In some embodiments, the processor determines the target failure exclusion item by: calculating the cosine similarity between the current retrieval key and the failure retrieval key corresponding to each candidate failure action in the failure memory; selecting the top N candidates with the highest similarity as initial failure items in descending order of similarity; determining whether the similarity of each initial failure item is greater than a second threshold, and determining the initial failure items with similarity greater than the second threshold as target failure items, wherein the second threshold is used to filter failure memories that are weakly related to the current state to prevent false triggering; determining the historical failure action blocks corresponding to each target failure item and their corresponding failure risk scores as candidate failure exclusion items; determining the exclusion weight corresponding to each candidate based on the normalized value of the similarity corresponding to each candidate failure exclusion item after scaling by a temperature coefficient; determining the failure relevance of the current state relative to the failure memory based on the difference between the average similarity corresponding to each candidate failure exclusion item and the second threshold after scaling to the range of 0 to 1 by a risk normalization coefficient; and jointly determining the candidate failure exclusion items, the exclusion weights corresponding to each candidate failure exclusion item, and the failure relevance as target failure exclusion items.

[0035] Further explanation on determining target exclusion terms can be found below. Figure 3 And its related descriptions.

[0036] Step 105: Based on the prior knowledge of the target success action, the target confidence, and the target failure exclusion term, the action generation model is given prior guidance for the action generation process of the task to be executed, so as to obtain the target execution action.

[0037] A target execution action refers to a continuous sequence of actions output by the motion generation model to drive the robot to perform the current task. In some embodiments, the target execution action includes a continuous motion trajectory for controlling the end-effector of the robotic arm to grasp the sleeve of the garment to be folded and fold the sleeve to the center line of the garment body, and / or a continuous motion trajectory for controlling the end-effector of the robotic arm to fold the garment body in half.

[0038] In some embodiments, the processor obtains the target execution action by: determining action sampling parameters based on the target confidence level regarding the prior of a successful target action; the action sampling parameters are used to characterize the guiding strength of the prior of a successful target action on the action generation process of the action generation model; acquiring random action noise, fusing the random action noise and the prior of a successful target action based on the action sampling parameters to obtain an initial execution action; adaptively updating the initial execution action through the action generation model based on the current observation information, the task to be executed, and the action sampling parameters; and during the adaptive update of the initial execution action, guiding the update of the initial execution action based on a target failure rejection term to obtain the target execution action.

[0039] For further explanation on obtaining the target's execution action, please refer to the following text. Figure 4 And its related descriptions.

[0040] In some embodiments of this application, successful experiences from the historical task execution process of the embodied robot are transformed into reusable prior guidance, while failure experiences are transformed into exclusionary guidance. Both of these improve the stability and success rate of action execution in long-term, multi-stage tasks. At the same time, through confidence-based prior guidance, successful experiences are fully utilized in high-reliability retrieval, while the model's ability to generate freely is preserved in low-reliability retrieval. Furthermore, through the bidirectional constraint of bringing successful priors closer and pushing away failure exclusionary terms, the generated actions are both advantageous and disadvantageous, effectively improving the closed-loop execution robustness of the embodied robot.

[0041] Figure 2 This is an exemplary flowchart illustrating the determination of prior actions for a target success according to some embodiments of this application. In some embodiments, this process may be executed by an electronic device (such as its processor).

[0042] Step 201: Obtain the action retrieval key corresponding to the prior of each candidate successful action in the success memory, and the first similarity between it and the current retrieval key.

[0043] The first similarity refers to a quantitative value used to characterize the degree of similarity in feature space between the current search key and a certain historical search key in the successful search memory.

[0044] An action retrieval key refers to a historical observation feature vector stored in the success memory that is associated with the prior of a candidate successful action. The action retrieval key can be generated by performing the same feature extraction and encoding process on the observation information at the time the action prior is written.

[0045] In some embodiments, the processor obtains the first similarity by: organizing all action retrieval keys in the successful action memory into a two-dimensional feature matrix, wherein the number of rows in the matrix is ​​the number of action priors in the memory, and the number of columns is the feature dimension of each retrieval key; using the current retrieval key as a query vector, performing matrix multiplication on the feature matrix to obtain the dot product of the query vector and each row feature vector; dividing each dot product by the product of the query vector norm and the corresponding row vector norm to obtain the cosine similarity between each action retrieval key and the current retrieval key; and outputting all cosine similarities as the first similarity corresponding to each candidate successful action prior.

[0046] Step 202: Determine the search key with a first similarity greater than a first threshold to the current search key from the success memory as the matching search key, and obtain the candidate successful action prior corresponding to the matching search key from the success memory as the matching successful action prior.

[0047] The first threshold refers to a lower bound value of similarity used to filter historical memories that are sufficiently relevant to the current state. The specific value of the first threshold can be set empirically or adaptively determined based on the distribution of all first similarities in the successful memory bank.

[0048] A matching retrieval key refers to a historical retrieval key in the successful retrieval memory whose first similarity exceeds a first threshold. A successful matching action prior refers to a sequence of historical action blocks stored in pairs with the matching retrieval key.

[0049] In some embodiments, the processor determines the matching retrieval key and the matching successful action prior by: comparing all first similarities one by one with a preset first threshold; when the first similarity corresponding to a candidate successful action prior is greater than the first threshold, marking the action retrieval key corresponding to the candidate as the matching retrieval key. Further, based on the storage index of the matching retrieval key in the success memory, reading the candidate successful action prior stored associated with the matching retrieval key, and using the candidate successful action prior as the matching successful action prior.

[0050] Step 203: Perform trajectory consistency filtering on the successfully matched action priors to obtain the filtered successfully matched action priors.

[0051] Trajectory consistency filtering refers to the process of removing abnormal candidates from the prior of successfully matched actions that have significant deviations from the successful trajectory.

[0052] Successful action priors refer to the successful action priors that are retained after trajectory consistency filtering.

[0053] In some embodiments, the processor performs trajectory consistency filtering by: calculating the Euclidean distance between every two action block sequences in the prior of a successfully matched action; for each action block sequence, determining the average of its Euclidean distances with other action block sequences as the density score of the action block sequence, wherein the larger the density score, the more the action deviates from the successful trajectory; sorting the density scores in descending order, removing the top 20% of action block sequences by density score, and using the remaining action block sequences as priors for the filtered successful actions.

[0054] Step 204: Aggregate the filtered successful action priors to obtain the target successful action priors associated with the current observation information.

[0055] Aggregation processing refers to the calculation process of merging multiple successful filtering actions according to their respective contribution weights.

[0056] The target success action prior refers to the sequence of action blocks obtained after aggregation processing, which is used to generate subsequent guiding actions.

[0057] In some embodiments, the processor performs aggregation processing including: obtaining a first similarity between the action retrieval key corresponding to each action block sequence in the filtered successful action prior and the current retrieval key, and using the first similarity as the original weight of each action block sequence; applying softmax normalization to each original weight to obtain the aggregated weight of each action block sequence; aligning all action block sequences in the filtered successful action prior according to time steps, and for sequences with inconsistent time step lengths, linear interpolation can be used to stretch the shorter sequence to the same length as the longest sequence; at each time step, weighting and summing the action values ​​of all action block sequences at that time step according to the corresponding aggregated weights to obtain the aggregated action value at that time step; arranging the aggregated action values ​​at all time steps in chronological order to form a complete action block sequence, thereby obtaining the target successful action prior.

[0058] In some embodiments, the aggregation process of the filtered successful action priors to obtain the target successful action priors associated with the current observation information may include the following operations: S11, obtain the second similarity between the search key corresponding to the prior of the successful filtering action and the current search key.

[0059] The second similarity refers to the quantitative value used to characterize the closeness between the historical search key corresponding to the prior of the successful filtering action and the current search key in the feature space.

[0060] In some embodiments, the processor obtains the second similarity by: reading the action retrieval key associated with each action block sequence from the data structure corresponding to the filtered successful action priors; organizing all the read action retrieval keys into a filtering feature matrix, wherein the number of rows in the matrix is ​​the number of filtered successful action priors and the number of columns is the feature dimension of the retrieval key; using the current retrieval key as a query vector, performing matrix multiplication on the filtering feature matrix and dividing by its respective norm to obtain the cosine similarity between each row in the filtering feature matrix and the query vector; and outputting the cosine similarity as the second similarity corresponding to each filtered successful action prior.

[0061] S12, determine the aggregation weight of the prior of the successful filtering action based on the second similarity.

[0062] Aggregation weight refers to a numerical coefficient used to characterize the contribution of each successful filtering prior in the subsequent aggregation process. Among them, the more similar the historical successful prior is to the current state, the higher the weight is given during aggregation.

[0063] In some embodiments, the processor determines the aggregation weight by: linearly normalizing the second similarity, that is, subtracting the minimum value among all second similarities from each second similarity, and then dividing by the difference between the maximum and minimum values ​​among all second similarities to obtain the preliminary weight in the interval of 0 to 1; and performing secondary normalization on the preliminary weight using the softmax function to obtain the final aggregation weight.

[0064] S13, use the aggregation weights to perform a weighted summation of the filtered successful action priors to obtain the target successful action priors associated with the current observation information.

[0065] In some embodiments, the processor performs weighted summation by: aligning all action block sequences in the filtered successful action priors by time steps; for action sequences with different time step lengths, using linear interpolation to stretch shorter sequences to the same length as the longest sequence, so that all sequences have a consistent length in the time dimension; at each time step, multiplying the action value of each filtered successful action prior by its corresponding aggregation weight, and then summing the product results to obtain the weighted summation action value at that time step; and arranging the weighted summation action values ​​at all time steps in chronological order to form a complete action block sequence as the target successful action prior.

[0066] In this embodiment, by aligning multiple successful action priors in the time dimension and then performing a weighted summation, the common motion patterns of multiple successful trajectories are integrated, effectively suppressing the random errors in a single candidate. The generated target successful action prior has both excellent generalization and stability, providing a reliable action benchmark for subsequent action generation.

[0067] In some embodiments, the number of successful action priors to be filtered is multiple. Determining the target confidence level corresponding to the target successful action prior based on the current search key may include the following operations: S21, determine the third similarity between the retrieval key corresponding to the prior of each successful filtering action and the current retrieval key, and determine the comprehensive state confidence between the current observation information and the historical observation information corresponding to the prior of the successful filtering action based on the third similarity.

[0068] The third similarity refers to a quantitative value used to characterize the degree of similarity in feature space between the historical search key corresponding to the prior of the successful filtering action and the current search key.

[0069] The overall state confidence score is a composite score representing the overall similarity between the current observation information and all matching historical observation information, obtained by aggregating the third similarity scores corresponding to the priors of all successfully filtered actions. For example, in the folding clothes task, the third similarity scores between the search keys corresponding to the 10 folding action segments retained after filtering and the current search key are 0.93, 0.91, 0.89, 0.87, 0.85, 0.84, 0.82, 0.81, 0.79, and 0.78, respectively. The overall state confidence score is calculated as 0.849 based on the average of these similarities.

[0070] In some embodiments, the processor determines the overall state confidence by: calculating the cosine similarity between the historical search key and the current search key corresponding to each prior of a successful filtering action, and using all the calculation results as the third similarity corresponding to each prior of a successful filtering action; and using the average of all the third similarities as the overall state confidence.

[0071] In other embodiments, the processor determining the overall state confidence may further include: sorting all third similarities in descending order of value, removing the highest and lowest values, and taking the average of the remaining third similarities as the overall state confidence.

[0072] S22, determine the trajectory difference metric among multiple successful filtering action priors, and determine the action confidence corresponding to the multiple successful filtering action priors based on the trajectory difference metric.

[0073] Trajectory difference metric is a quantitative indicator used to characterize the degree of difference in the shape of action trajectories among multiple successful action priors.

[0074] Action confidence is a score calculated based on trajectory difference metric, which characterizes the consistency of action patterns among multiple successful filtering action priors.

[0075] There is a negative correlation between action confidence and trajectory difference measure. That is, the smaller the trajectory difference measure, the more consistent the priors of each action are, and the higher the action confidence. The larger the trajectory difference measure, the more dispersed the priors of each action are, and the lower the action confidence.

[0076] In some embodiments, the processor determines the action confidence by: calculating the multivariate time-normalized distance between every two successfully filtered action priors to obtain multiple distance values; for each successfully filtered action prior, taking the median of all multivariate time-normalized distances between it and other successfully filtered action priors as the outlier score of that action prior; taking the average of all outlier scores as the trajectory difference measure; and inputting the trajectory difference measure into a preset negative correlation mapping function, which can map the trajectory difference measure to the interval between 0 and 1, and the mapping result is the action confidence.

[0077] In some embodiments, determining the trajectory difference metric among the plurality of successful filtering action priors, and determining the action confidence corresponding to the plurality of successful filtering action priors based on the trajectory difference metric, may include the following operations: S221, obtain the multivariate dynamic time-warped distance between each filter success action prior and other filter success action priors.

[0078] Multivariate dynamic temporal warping distance is a distance metric used to measure the similarity of the overall motion shapes of two action block sequences in the time dimension. This distance is achieved by performing non-linear alignment of the two action sequences on the time axis, finding the alignment path that minimizes the cumulative distance between the two sequences, and using this minimum cumulative distance as the similarity metric between the two sequences.

[0079] In some embodiments, the processor obtains the multivariate dynamic time warping distance by: constructing a dynamic time warping cumulative distance matrix for each action block sequence corresponding to two successfully filtered action priors, wherein the number of rows in the matrix is ​​the time step length of the first action sequence and the number of columns is the time step length of the second action sequence; filling the matrix according to the recursive formula of the dynamic time warping algorithm, wherein the value at each position is equal to the Euclidean distance between the action vectors of the two sequences at that position plus the minimum of the cumulative distances of the three adjacent positions to the left, below, and below the position; determining the cumulative distance value at the lower right corner of the matrix as the multivariate dynamic time warping distance between the two action block sequences; and traversing all pairwise combinations of successfully filtered action priors to obtain the multivariate dynamic time warping distance.

[0080] S222, obtain the median distance of the multivariate dynamic time-warped distance corresponding to the prior of each successful filtering action.

[0081] The median distance refers to the value in the middle of the multivariate dynamic time-normalized distances between a given successfully filtered action prior and all other successfully filtered action priors, sorted from smallest to largest. The median distance reflects the overall deviation of this action prior relative to all other action priors. A smaller median distance indicates that the trajectory shape of this action prior is relatively similar to that of the other action priors, while a larger median distance indicates that the trajectory shape of this action prior differs significantly from that of the other action priors.

[0082] In some embodiments, the processor obtains the median distance by: extracting all distance values ​​in the i-th row corresponding to the prior of the i-th successful filtering action from the multivariate dynamic time-warped distance matrix, and sorting the remaining distance values ​​in ascending order of value; when the number of remaining distance values ​​is odd, taking the distance value in the middle position after sorting as the median distance of the prior of the i-th successful filtering action; when the number of remaining distance values ​​is even, taking the average of the two middle distance values ​​as the median distance.

[0083] S223, Based on the median distances corresponding to the priors of multiple successful filtering actions, determine the trajectory difference metric among the priors of multiple successful filtering actions.

[0084] Trajectory difference metric is a quantitative indicator that represents the overall dispersion of multiple action priors in the shape of action trajectory, obtained by aggregating the median distances corresponding to all successfully filtered action priors.

[0085] In some embodiments, the processor determines the trajectory difference metric by taking the arithmetic mean of the median distances corresponding to all successfully filtered action priors as the trajectory difference metric. This average value can reflect the central tendency of the deviation level of each action prior relative to other actions in the same group. The larger the average value, the higher the overall deviation and the greater the trajectory difference metric.

[0086] S224, determine the action confidence corresponding to multiple successful filtering action priors based on the trajectory difference metric; there is a negative correlation between action confidence and trajectory difference metric.

[0087] Action confidence is a score calculated based on trajectory difference metric, which characterizes the consistency of action patterns among multiple successful filtering action priors.

[0088] The negative correlation between action confidence and trajectory difference measure means that when the trajectory difference measure is small, the trajectory shapes of each action prior are highly consistent, and the action confidence value is high, indicating good cooperative stability among candidate actions. When the trajectory difference measure is large, the trajectory shapes of each action prior are dispersed, and the action confidence value is low, indicating that there is a large divergence in motion patterns among candidate actions, and the reliability of the target successful action prior in this dimension is low.

[0089] In some embodiments, the processor determines the action confidence by: inputting a trajectory difference metric into a preset negative correlation mapping function, which may be in the form of exponential decay. For example, the action confidence is equal to an exponential function with the natural constant as the base, where the exponent is the negative of the trajectory difference metric divided by a preset scale parameter; when the trajectory difference metric approaches zero, the value of the exponential function approaches one, indicating that the action confidence is close to its highest value; when the trajectory difference metric gradually increases, the value of the exponential function monotonically decreases and approaches zero, indicating that the action confidence gradually decreases to its lowest value.

[0090] In some embodiments of this application, by introducing a multivariate dynamic time warping distance metric to measure the trajectory shape similarity between action block sequences, the problem of time axis misalignment of action sequences under different execution durations can be effectively addressed. This makes it possible to make consistent comparisons of successful trajectories under different execution speeds, avoiding the misjudgment problem caused by the sensitivity of traditional Euclidean distance to time misalignment data. At the same time, based on the negative correlation between trajectory difference metric and action confidence, the degree of trajectory consistency between candidate actions is quantified, providing a reliable action dimension evaluation basis for the subsequent weighted fusion of target confidence.

[0091] S23, perform weighted fusion of state confidence and action confidence to obtain fused confidence, and determine the fused confidence as the target confidence corresponding to the prior of the target successful action.

[0092] Fusion confidence refers to the overall confidence score obtained by fusing state confidence and action confidence according to their respective weighting coefficients.

[0093] In some embodiments, the processor obtains the target confidence score by: acquiring preset state confidence weighting coefficients and action confidence weighting coefficients, for example, setting the state confidence weighting coefficient to 0.6, the action confidence weighting coefficient to 0.4, and the sum of the two weighting coefficients equals 1; multiplying the state confidence score by the state confidence weighting coefficient to obtain a weighted state confidence score, and multiplying the action confidence score by the action confidence weighting coefficient to obtain a weighted action confidence score; and adding the weighted state confidence score and the weighted action confidence score to obtain a fused confidence score.

[0094] In this embodiment, the target confidence is obtained by weighted fusion of state confidence and action confidence. This allows the target confidence to reflect information on both state similarity and action consistency, avoiding the confidence evaluation bias caused by relying on only a single similarity index and effectively improving the robustness and adaptability of the action generation system.

[0095] Figure 3This is an exemplary flowchart illustrating the determination of a target failure exclusion item according to some embodiments of this application. In some embodiments, this process may be performed by an electronic device (such as its processor).

[0096] Step 301: Obtain the failure retrieval key, rollback action fragment, and failure risk score corresponding to the prior of each candidate failure action in the failure memory.

[0097] A failure retrieval key refers to a historical observation feature vector stored in the failure memory that is prior to the candidate failed action. The failure retrieval key can be generated during failure memory writing by performing the same feature extraction and encoding process on the observation information that caused the task progress rollback.

[0098] A rollback action fragment refers to a sequence of action blocks stored in the failure memory that causes a significant rollback in task progress. This action fragment includes robot motion data across multiple consecutive time steps.

[0099] The failure risk score is a quantitative value used to characterize the severity of task progress regression caused by a failed action segment. This score is positively correlated with the magnitude of the progress regression; the greater the regression, the higher the failure risk score.

[0100] In some embodiments, the processor may read a pre-built failure memory file from an external storage medium, the failure memory including a stored failure retrieval key field, a rollback action fragment field, and a failure risk score field.

[0101] In some embodiments, a memory copy of the failure memory can be maintained in real time during robot deployment. This memory copy can be stored in a hash table structure, where the key is a unique identifier of the candidate failed action prior, and the value is a structure containing three fields: failure retrieval key, fallback action fragment, and failure risk score. When a new failure fragment is written to the failure memory, the hash table is updated synchronously. The processor can directly traverse all action priors from the hash table and read the contents of the three fields to obtain the failure retrieval key, fallback action fragment, and failure risk score corresponding to the candidate failed action prior.

[0102] Step 302: Based on the failure retrieval key and the current retrieval key, calculate the state similarity between the current sampling state corresponding to the current observation information and the historical failure state corresponding to the prior of each candidate failure action.

[0103] State similarity refers to the quantitative value used to characterize the degree of similarity in feature space between the current sampled state corresponding to the current observation information and the historical failure state corresponding to the failure retrieval key of a candidate failure action prior.

[0104] In some embodiments, the processor calculates state similarity by: performing a dot product operation on the current search key and the failed search key corresponding to the candidate failed action prior currently being processed, obtaining the inner product of the two vectors; calculating the L2 norm of the current search key and the L2 norm of the failed search key, multiplying them to obtain a norm product; and dividing the inner product by the norm product to obtain a cosine similarity as the state similarity. The processor iterates through each candidate failed action prior in the failure memory, calculating the corresponding state similarity for each in turn.

[0105] Step 303: Select candidate failed action priors with state similarity greater than the second threshold from the failure memory bank as candidate failure exclusion terms.

[0106] The second threshold refers to the lower bound value of similarity used to filter out failure actions that are sufficiently relevant to the current state to trigger failure rejection guidance.

[0107] Candidate failure exclusion terms refer to prior candidate failed actions in the failure memory whose state similarity exceeds a second threshold. The value of the second threshold can be determined based on prior experience.

[0108] In some embodiments, the processor filters candidate failure exclusion items by: comparing the state similarity with a preset second threshold, the second threshold being 0.65; when the state similarity corresponding to a certain candidate failure action prior is greater than 0.65, the candidate failure action prior is marked as a candidate failure exclusion item; after traversing all candidate failure action priors, all candidates with state similarity greater than the second threshold are collected to form a candidate failure exclusion item set.

[0109] Step 304: Based on the state similarity corresponding to the candidate failure rejection item, determine the failure candidate weight corresponding to the candidate failure rejection item, and determine the failure correlation between the current sampled state and the historical failure state.

[0110] The failure candidate weight is a normalized numerical coefficient used to characterize the contribution of each candidate failure rejection term to subsequent failure rejection guidance. This weight is determined based on the state similarity of each candidate; the higher the state similarity, the greater the corresponding failure candidate weight.

[0111] Failure relevance is a comprehensive quantitative score that characterizes the degree of correlation between the current sampled state and historical failure states in the failure memory. This score is obtained by aggregating the state similarity of candidate failure exclusion items, and its value ranges from 0 to 1. The higher the failure relevance, the closer the current state is to the historical failure state, and the stronger the necessity to trigger failure exclusion guidance.

[0112] In some embodiments, the processor determining the failure candidate weights and failure relevance may include: processing the state similarity corresponding to the candidate failure exclusion item using a linear normalization method, that is, subtracting the minimum value among all state similarities from each state similarity and dividing by the difference between the maximum value and the minimum value to obtain the preliminary weight of each candidate, and then summing and normalizing the preliminary weights to obtain the failure candidate weights; and taking the maximum value of the state similarity corresponding to all candidate failure exclusion items as the failure relevance.

[0113] In some embodiments, the process of obtaining the failure exclusion term described above can be as shown in the following examples: From the failure memory bank When searching for failed segments similar to the current state, given the current search key... For the first in the failure memory bank A priori failure action Calculate state similarity: (1) Select the Top- with the highest similarity There are several failed candidates, and the threshold is used to determine the failure candidates. Filtering weakly correlated samples: (2) For the retained candidate failed actions, their weights are defined as follows: (3) in, This is the temperature coefficient.

[0114] To measure the correlation between the current state and historical failure states, we define failure correlation: (4) in: (5) When the current state is highly similar to a historical failure state Larger values ​​lead to enhanced negative sample guidance; when the current state is dissimilar to the memory of failure, When the value approaches 0, the failure boot process will automatically weaken or shut down.

[0115] Step 305: Based on the candidate failure exclusion items, failure candidate weights, failure relevance, and the failure risk score and fallback action fragments corresponding to the candidate failure exclusion items, determine the target failure exclusion item.

[0116] In some embodiments, a target failure exclusion term can be used to construct a failure energy value during the sampling update process of the action generation model. Specifically, the higher the similarity between the current action estimate and the backtracking action segment in the action semantic space, the higher the failure risk score, the higher the failure candidate weight, and / or the higher the failure relevance, the larger the failure energy value. The processor can then correct the sampling update direction of the action generation model based on the failure energy value, so that the generated action is far away from historical failed action patterns.

[0117] In some embodiments, the processor determines the target failure exclusion item by: organizing the candidate failure exclusion items into a candidate list, wherein each candidate action prior includes the failure retrieval key, fallback action fragment and failure risk score corresponding to the exclusion item; associating and storing the corresponding failure candidate weight with the candidate action prior to form a weight-labeled candidate list; appending the failure relevance as a global control parameter to the candidate list; and determining the candidate list containing all the above information as the target failure exclusion item and outputting it.

[0118] In other embodiments, the processor determining the target failure exclusion term may further include: when the number of candidate failure exclusion terms is zero, that is, when no candidate failure action prior state similarity exceeds the second threshold, the target failure exclusion term is set to an empty identifier; when the target failure exclusion term is an empty identifier, failure exclusion guidance will be automatically skipped in subsequent steps, and the sampling process depends on successful prior guidance and the original model generation.

[0119] The empty identifier handling mechanism ensures that the system will not generate anomalies due to missing failure rejection information when no relevant failure memory is retrieved, thus guaranteeing the normal operation of the method when the failure memory bank is sparse or the current state is dissimilar to all historical failures.

[0120] In this embodiment, by calculating state similarity and filtering with a second threshold, it is ensured that only historical failure memories highly related to the current state are selected as candidate failure exclusion items. This avoids interference from failure experiences unrelated to the current scene on action generation, achieving accurate retrieval and on-demand triggering of failure memories. At the same time, by jointly determining the candidate failure exclusion item, failure candidate weight, and failure relevance as the target failure exclusion item, complete input information is provided for the construction of the failure energy function and the calculation of the exclusion gradient in the subsequent sampling process.

[0121] In some embodiments, the processor may further determine the action semantic similarity between the target successful action prior and the backoff action fragment corresponding to the candidate failure rejection term; if the action semantic similarity is greater than a preset conflict threshold, the action sampling parameter corresponding to the target successful action prior shall be reduced, and / or the failure rejection strength corresponding to the target failure rejection term shall be increased.

[0122] Specifically, to avoid conflicts between the target successful action prior and historical failed action patterns that could mislead the action generation process, the processor can also perform conflict detection on the fallback action segments corresponding to the target successful action prior and candidate failure rejection terms. The processor can map the target successful action prior to a preset action semantic space to obtain successful action semantic features, and map the fallback action segments corresponding to candidate failure rejection terms to the action semantic space to obtain failed action semantic features. The processor can calculate the action semantic similarity between the successful action semantic features and the failed action semantic features. If the action semantic similarity is greater than a preset conflict threshold, it indicates that the target successful action prior and historical failed action patterns have a high similarity at the action semantic level, and the currently retrieved target successful action prior may be misleading. In this case, the processor can reduce the action sampling parameters corresponding to the target successful action prior to weaken the guiding strength of the target successful action prior on the action generation process of the action generation model; and / or, the processor can increase the failure rejection strength corresponding to the target failure rejection term to enhance the rejection effect on historical failed action patterns during the action generation process. Therefore, when there is a conflict between the prior knowledge of successful actions and the prior knowledge of failed actions, the influence of successful prior knowledge can be adaptively reduced and the guidance of failure rejection can be enhanced, thereby reducing the probability of generating repeated historical failure patterns of actions.

[0123] In some embodiments, the failure risk score is determined based on the task progress rollback caused by the corresponding rollback action segment, and updated according to the number of times the rollback action segment is repeatedly retrieved within a preset historical window. Specifically, the processor can obtain the task progress evaluation value before and after the execution of the rollback action segment, and determine the task progress rollback range based on the difference between the task progress evaluation value before and after execution; the larger the task progress rollback range, the higher the failure risk score. The processor can also update the failure risk score according to the number of times the rollback action segment is repeatedly retrieved within the preset historical window. The preset historical window can be a preset number of historical task execution rounds, or a preset time period of historical task execution records. If a rollback action segment is repeatedly retrieved more times within the preset historical window, it indicates that the failure mode corresponding to the rollback action segment is more likely to be repeatedly triggered in the current deployment environment, and the processor can increase the failure risk score corresponding to the rollback action segment; if a rollback action segment is not retrieved or has a low number of hits within the preset historical window, the processor can maintain or decrease the failure risk score corresponding to the rollback action segment.

[0124] Figure 4 This is an exemplary flowchart illustrating the determination of a target to perform an action according to some embodiments of this application. In some embodiments, this process may be performed by an electronic device (such as its processor).

[0125] Step 401: Determine the action sampling parameters for the prior of the successful action of the target based on the target confidence.

[0126] Action sampling parameters are numerical coefficients calculated based on target confidence levels, used to control the fusion ratio of successful target action priors in the initial stage of action generation. Action sampling parameters characterize the guiding strength of successful target action priors on the action generation process of the action generation model. The specific values ​​of the action sampling parameters determine the relative proportion between the successful target action prior components and the random exploration components in the initial execution action.

[0127] Guidance strength refers to the degree to which the prior knowledge of the target's successful action influences the action generation process. The higher the guidance strength, the closer the initial action is to the prior knowledge of the target's successful action; the lower the guidance strength, the more randomness is retained in the initial action.

[0128] In some embodiments, the processor determines the action sampling parameters by: directly mapping the target confidence level to the value of the action sampling parameters, that is, the action sampling parameters are equal to the target confidence level and the value range is 0 to 1.

[0129] Step 402: Obtain random action noise, and fuse the random action noise and the prior of the target successful action based on the action sampling parameters to obtain the initial execution action.

[0130] Random action noise refers to a Gaussian random vector with the same dimension as the prior of the successful action of the target, which is sampled from a standard normal distribution.

[0131] The initial execution action refers to the action sequence obtained by fusing random action noise and prior successful action of the target through action sampling parameters.

[0132] In some embodiments, the processor samples a random vector from a standard normal distribution with the same dimension as the prior dimension of the target successful action as random action noise; the random action noise and the prior dimension of the target successful action are linearly interpolated and fused according to the values ​​of the action sampling parameters, and the initial action is executed.

[0133] Step 403: Using the action generation model, the initial execution action is adaptively updated based on the current observation information, the task to be executed, and the action sampling parameters.

[0134] Adaptive update refers to the process by which an action generation model takes the initial action as the sampling starting point and, through the iterative denoising process of a flow matching or diffusion model, combines current observation information and task instructions to gradually optimize the initial action into an action sequence that is adapted to the current scene.

[0135] Action generation model refers to generative action decoder in visual language action model. It receives observation information, task instructions and intermediate action state of the current sampling step as input, and outputs predicted velocity field or noise prediction to guide the iterative update of intermediate action state.

[0136] In some embodiments, the processor performs adaptive updates by: encoding the multi-view images and robot body state in the current observation information and concatenating them with the feature representation of the task instructions, as conditional inputs to the action generation model; inputting the intermediate action state of the current sampling step and the current sampling time step into the action generation model, and the model outputs the predicted velocity field; performing Eulerian updates on the intermediate action state using the predicted velocity field, i.e., the intermediate action state of the next sampling step is equal to the intermediate action state of the current sampling step minus the sampling step length multiplied by the predicted velocity field; repeating the above process until a preset total number of sampling steps is reached.

[0137] Step 404: During the adaptive update of the initial execution action, the update of the initial execution action is guided by failure rejection based on the target failure rejection term to obtain the target execution action.

[0138] Failure exclusion guidance refers to the process of modifying the sampling update direction by using candidate failure exclusion terms, failure candidate weights, and failure correlation in the target failure exclusion terms during the adaptive update of intermediate action states, so that the update trajectory of intermediate action states is far away from the historical failure action patterns.

[0139] For detailed instructions on performing failure rejection boots, please see below. Figure 5 And its related descriptions.

[0140] In this embodiment, the action sampling parameters are adaptively determined by the target confidence level, so that the guidance strength of the prior of the successful action of the target is automatically adjusted as the retrieval reliability changes. When the retrieval result is reliable, the prior guidance is strengthened to make full use of the successful experience, and when the retrieval result is unreliable, the prior guidance is reduced or canceled to retain the original generation ability of the model, thus realizing the adaptive adjustment of the successful experience guidance.

[0141] Figure 5 This is an exemplary flowchart illustrating the determination of a target execution action according to other embodiments of this application. In some embodiments, this process may be executed by an electronic device (such as its processor).

[0142] Step 501: Obtain the intermediate action state of the action generation model in the current sampling step, and predict the predicted velocity field corresponding to the intermediate action state through the action generation model.

[0143] The intermediate action state corresponding to the current sample refers to the action sequence state of the action generation model at the current iteration step during the flow matching sampling process. This state gradually evolves from the initial action to the final target action over time.

[0144] The predicted velocity field refers to the velocity vector field in the action space that the action generation model outputs, given the intermediate action state, the current sampling time step, observation information, and task instructions as inputs.

[0145] In some embodiments, the processor obtains the intermediate action state of the action generation model at the current sampling step by: determining the initial execution action as the intermediate action state corresponding to the first sampling step, and determining the intermediate action state corresponding to the current sampling step in sequence according to the sampling step order.

[0146] In some embodiments, the processor acquires intermediate action states and predicts velocity fields by: reading an intermediate action state tensor corresponding to the current sampling step index from the memory buffer of the sampling process, wherein the dimension of the tensor is the action block length multiplied by the action dimension; concatenating the intermediate action state tensor with the normalized time code of the current sampling time step as the first part of the input to the action generation model; fusing and encoding the multi-view image features and robot body state features in the current observation information, and performing cross-attention fusion with the text feature code of the task instructions to obtain a conditional feature tensor as the second part of the input to the action generation model; and inputting the above two parts of the input into the action generation model, wherein the model outputs a predicted velocity field tensor corresponding to the intermediate action state.

[0147] Step 502: Determine the current action estimate corresponding to the current sampling step based on the intermediate action state and the predicted velocity field.

[0148] Current action estimation refers to the action sequence estimated corresponding to the current sampling state, which is obtained by inverse calculation based on the intermediate action state of the current sampling step and the predicted velocity field.

[0149] In some embodiments, the processor determines the current action estimate by: obtaining the normalized time parameter of the current sampling step; subtracting the current time parameter from the intermediate action state and multiplying it by the predicted velocity field to obtain the current action estimate.

[0150] In other embodiments, the processor determining the current action estimate may further include: subtracting the square of the current time parameter from the intermediate action state and multiplying it by the predicted velocity field to obtain the current action estimate.

[0151] After determining the current action estimate corresponding to the current sampling step, the failure rejection strength corresponding to the current sampling step can be determined based on the current action estimate and the target failure rejection term. For example, the semantic similarity between the current action estimate and the prior of candidate failed actions in the target failure rejection term can be determined first. Then, the failure energy value of the intermediate action state can be determined by combining the failure relevance, failure candidate weight, and failure risk score. Finally, the failure rejection strength can be obtained by calculating the gradient of the failure energy value. For details on how to determine the failure rejection strength corresponding to the current sampling step, please refer to steps 503 to 506.

[0152] Step 503: Map the prior of the candidate failed action in the current action estimate and the target failure rejection term to the preset action semantic space respectively to obtain the semantic features of the current action estimate and the semantic features of the failed action corresponding to the target failure rejection term.

[0153] Action semantic space refers to the high-dimensional semantic representation space to which the original action sequence is mapped by the action encoder. Action vectors that are close in location in this space have similar motion semantic meanings. For example, folding actions are close to each other in semantic space, while folding actions and pulling actions are far apart in semantic space.

[0154] The semantic features of the current action estimate refer to the feature vectors in the action semantic space after the current action estimate is mapped by the action encoder.

[0155] The semantic features of failed actions refer to the feature vectors in the action semantic space corresponding to the backtracking action segments corresponding to each candidate failed action in the target failure exclusion term after being mapped by the same action encoder.

[0156] In some embodiments, the processor inputs the current action estimate into a pre-trained action encoder network, which may consist of multiple layers of one-dimensional convolutional neural networks and global pooling layers, and outputs a semantic feature vector of fixed dimensions; the processor inputs the backtracking action fragment corresponding to the prior of each candidate failed action in the target failure exclusion term into the same action encoder network to obtain the semantic feature vector of each candidate failed action.

[0157] Step 504: Determine the semantic similarity between the semantic features of the current action estimation and the semantic features of the failed action.

[0158] Semantic similarity is a quantitative value used to characterize the degree of closeness between the current action estimation semantic feature vector and the failed action semantic feature vector corresponding to a candidate failure exclusion term in the action semantic space.

[0159] In some embodiments, the processor calculates the dot product between the current action estimation semantic feature vector and the failed action semantic feature vector corresponding to each candidate failure exclusion term to obtain the inner product value; calculates the L2 norm of the current action estimation semantic feature vector and the L2 norm of each failed action semantic feature vector respectively; divides each inner product value by the product of the norm of the current action estimation semantic feature vector and the norm of the corresponding failed action semantic feature vector to obtain the cosine similarity corresponding to each candidate, which is used as the semantic similarity.

[0160] Step 505: Determine the failure energy value corresponding to the intermediate action state of the current sampling step based on the failure relevance, failure candidate weight, failure risk score and semantic similarity corresponding to the target failure exclusion item.

[0161] Failure energy is a scalar value used to quantify how close the current intermediate action state is to the historical failed action pattern. The higher the failure energy value, the closer the currently generated action is to the historical failed pattern, and the stronger the rejection guidance needs to be applied.

[0162] In some embodiments, the processor determines the failure energy value by: for each candidate failure exclusion item, calculating the difference between its corresponding semantic similarity and the preset negative sample interval; inputting the difference into the activation function, multiplying the activation result by the corresponding failure risk score and failure candidate weight to obtain the single energy contribution of the candidate; summing the single energy contributions of all candidates and multiplying by the global failure relevance to obtain the failure energy value.

[0163] Step 506: Calculate the gradient information of the intermediate action state of the current sampling step based on the failure energy value to obtain the failure rejection strength corresponding to the current sampling step.

[0164] Failure repulsion strength refers to the gradient vector of the failure energy value relative to the intermediate action state. This gradient vector indicates the direction in which the failure energy value increases the fastest in the intermediate action state space. Updating the intermediate action state along the negative direction of this gradient can reduce the failure energy value, even if the currently generated action is far from the historical failure pattern. The dimension of failure repulsion strength is the same as that of the intermediate action state, and each element corresponds to the correction direction of the corresponding action parameter in the intermediate action state at the current sampling step.

[0165] In some embodiments, the processor uses the failure energy value as a scalar loss function, takes the partial derivative of each element in the intermediate action state tensor, and obtains a gradient tensor with the same dimension as the intermediate action state. The gradient tensor can be used as the failure rejection strength.

[0166] Step 507: Based on the failure repulsion strength, the predicted velocity field is corrected to obtain the corrected velocity field.

[0167] The corrected velocity field can refer to the velocity field obtained by superimposing the failure repulsion intensity on the original predicted velocity field of the action generation model.

[0168] In some embodiments, the processor obtains a preset repulsion strength coefficient, which is used to control the overall effect strength of the failure repulsion guidance; multiplies the failure repulsion strength by the repulsion strength coefficient to obtain a scaled repulsion term; and superimposes the scaled repulsion term onto the original predicted velocity field predicted by the action generation model to obtain a corrected velocity field, that is, the corrected velocity field is equal to the predicted velocity field plus the product of the repulsion strength coefficient and the failure repulsion strength.

[0169] In some embodiments, the processor can further calculate the difference vector between the intermediate action state and the prior of the candidate failed action, normalize the difference vector, and determine the normalized difference vector as the failure rejection direction vector. The failure rejection intensity is multiplied by the failure rejection direction vector as a scalar coefficient to obtain the failure rejection velocity increment, where the failure rejection velocity increment represents the offset required to move away from the failure trajectory in the action space. Then, the failure rejection velocity increment is scaled by a preset rejection fusion coefficient and vector-added with the predicted velocity field to obtain a preliminary corrected velocity field. For example, the scaled failure rejection velocity increment can be added element-wise to the predicted velocity field to obtain the preliminary corrected velocity field. The velocity amplitude is then obtained by calculating the square root of the sum of the squares of each action dimension in the preliminary corrected velocity field. If the velocity amplitude is greater than the maximum allowable velocity amplitude, the preliminary corrected velocity field is multiplied by the ratio of the maximum allowable velocity amplitude to the velocity amplitude to obtain the corrected velocity field.

[0170] Step 508: Update the intermediate action state corresponding to the current sampling step according to the corrected velocity field to obtain the intermediate action state of the next sampling step. Take the next sampling step as the new current sampling step and iterate the above process until the sampling ends. Determine the intermediate action state obtained by the final sampling as the target action to be executed.

[0171] The intermediate action state for the next sampling refers to the intermediate action state obtained for the next iteration after applying the modified velocity field to the current intermediate action state and performing one Euler update.

[0172] In some embodiments, the processor subtracts the product of the sampling step size and the corrected velocity field from the intermediate action state of the current sampling step to obtain the intermediate action state of the next sampling step.

[0173] In this embodiment, by mapping the current action estimate and historical failed actions to the action semantic space for similarity comparison, the similarity judgment of action patterns transcends the numerical differences in the original action space. It can capture the essential similarity between the current generated action and historical failed actions from the semantic level of motion intention and trajectory shape, thereby improving the generalization ability and accuracy of failure pattern recognition. At the same time, by calculating the gradient of the intermediate action state through the failure energy value to obtain the failure rejection strength, the contribution of each dimension of the intermediate action state to the failure risk is accurately quantified, so that the rejection correction can be applied to the intermediate action state in the manner of gradient descent, thereby achieving fine and accurate correction of the generated trajectory.

[0174] In some embodiments, successful action priors can be utilized. Initialize the generative sampling process: (6) In this embodiment, the successful prior initialization method remains unchanged, and only failed negative samples are added to guide the sampling process.

[0175] During the sampling process, the current intermediate state is The model predicts the velocity field as follows: .

[0176] To determine whether the currently generated action is close to a historical failed action, the corresponding clean action is estimated based on the current intermediate state: (7) in, This represents the action estimate corresponding to the current sampling state. The current action estimate and the failed action fragment are then mapped to the action semantic space: (8) in, This represents an action encoder, which can be implemented by an action MLP, a trajectory encoder, or a pre-trained action representation network.

[0177] Based on the similarity between the current action estimate and the failed action, a failure energy function is defined: (9) in, Indicates the energy intensity of failure. Indicates the degree of failure relevance. Indicates the weight of the failed candidate. Indicates the risk score of failure. This indicates the negative sample interval.

[0178] When the current generated action is estimated With historical failures The more similar they are in the action semantic space, the better. The larger the value, the greater the corresponding failure energy. The larger the value, the better. Therefore, this energy function can characterize how closely the currently generated action resembles a historical failed action pattern.

[0179] For failed negative samples during the sampling process, the original samples are updated as follows: (10) After adding failed negative samples as a guide, the sampling is updated as follows: (11) Equivalent land can be written as: (12) in, The original VLA generation velocity field remains unchanged; This is a newly added failure memory exclusion term, used to push the sampled state away from the historical failure action region.

[0180] The induction strength of failed negative samples can be adaptively determined based on the failure correlation: (13) in, Indicates the maximum failure rejection strength. Indicates the risk trigger threshold. Control the sensitivity of the mapping function.

[0181] When the current state is highly correlated with failed memories Larger The larger the value, the stronger the system's rejection of historical failed actions; when no reliable failure samples are retrieved, Smaller When the value approaches 0, the sampling process degenerates into the original method.

[0182] Finally, the action block is obtained after sampling. .

[0183] Therefore, the newly added failure memory branch in this application does not change the original successful memory guidance process, but rather adds an additional negative sample rejection term to the original sampling process. Successful memories are used to provide reliable action priors, while failure memories are used to avoid repeated action patterns that cause task progress to regress. Together, we can improve the closed-loop stability of generative VLA in continuous deployment scenarios.

[0184] Figure 6 This is an exemplary flowchart illustrating the generation of the current search key according to some embodiments of this application. In some embodiments, this process may be performed by an electronic device (such as its processor).

[0185] Step 601: Obtain image perception data corresponding to multiple viewpoints in the current observation information.

[0186] Image perception data refers to two-dimensional color or depth images captured by an embodied robot at the current moment of decision-making using multiple cameras mounted at different locations on its body. These multiple perspectives include a forward-facing camera positioned on the robot's head, a wrist-facing camera positioned on the robotic arm's wrist, and lateral-facing cameras positioned on either side of the robot's torso. For example, in a clothes-folding task, the image perception data includes a top-down image of the entire garment captured by the robot's head camera and a close-up image of the contact area between the gripper and the cuff captured by the wrist camera.

[0187] In some embodiments, the processor acquires image perception data by reading image perception data from multiple camera drivers through an image acquisition interface provided by the robot operating system.

[0188] Step 602: The visual encoder of the action generation model performs image encoding on the image perception data corresponding to each viewpoint to obtain the spatial block feature set corresponding to each viewpoint.

[0189] A visual encoder is a neural network module in an action generation model used to extract visual features from an image.

[0190] A spatial block feature set refers to the set of multiple spatial block feature vectors output by a visual encoder after performing block embedding processing on a single image. Each spatial block feature corresponds to the content encoding of a local region in the image.

[0191] Spatial position relationship refers to the row and column position coordinates of the image block corresponding to each spatial block feature in the original image.

[0192] In some embodiments, the processor inputs the image into a visual encoder. The encoder divides the image into fixed-size image blocks, for example, each image block is 16 by 16 pixels, resulting in 14 by 14, or 196 image blocks. A linear projection is performed on each image block to obtain a block embedding vector, and a corresponding learnable positional code is superimposed on each block embedding vector. This positional code contains the row and column index information of the image block in the grid, so that the features retain spatial positional relationships. All block embedding vectors after superimposing positional codes are input into the multi-head self-attention transformer layer of the visual encoder for processing. Each transformer layer interacts with features across the entire image through a self-attention mechanism. The last layer of the visual encoder outputs the spatial block features corresponding to each image block. Multiple spatial block features constitute the spatial block feature set of this viewpoint.

[0193] Step 603: For each viewpoint, rearrange the spatial block features in the spatial block feature set corresponding to the viewpoint into a two-dimensional feature grid according to their corresponding spatial position relationship to obtain the viewpoint feature grid corresponding to the viewpoint.

[0194] A two-dimensional feature grid refers to reorganizing a set of spatial block features in one-dimensional sequence form into a two-dimensional matrix structure according to the row and column numbers corresponding to each feature. The number of rows in the grid is equal to the number of rows in the image block, and the number of columns is equal to the number of columns in the image block. The element at each position in the grid is the feature vector of the corresponding image block.

[0195] A viewpoint feature grid is a two-dimensional feature structure obtained by rearranging spatial block features for a certain viewpoint, which preserves the spatial topological relationships of the original image.

[0196] In some embodiments, the processor performs secondary interpolation on the spatial block features during the rearrangement process. When the number of image blocks is insufficient to form a complete rectangular grid, a bilinear interpolation method is used to supplement the virtual features at the missing positions, so that the view feature grid always maintains a rectangular structure; or the spatial block features are directly filled into the two-dimensional grid row by row in sequence to complete the rearrangement.

[0197] Step 604: Perform pooling downsampling on the view feature mesh corresponding to the viewpoint to obtain the view compressed feature corresponding to the viewpoint.

[0198] Pooling downsampling refers to the process of applying local pooling operations to a viewpoint feature grid to reduce its spatial resolution. This process reduces the number of features by aggregating features from adjacent spatial locations into a single feature value, while simultaneously enhancing the translation invariance and local smoothness of the features.

[0199] Viewpoint compression features refer to feature meshes with reduced spatial resolution but unchanged feature channels obtained after pooling downsampling.

[0200] In some embodiments, the processor slides a pooling window of size 2 by 2 with a step size of 2 on the two-dimensional spatial plane of the view feature grid. The average value of the feature vectors of the four adjacent positions covered by the pooling window each time is calculated element by element to obtain an output feature vector with the same dimension as the input feature vector. After the pooling window slides through the entire view feature grid, all output feature vectors are reorganized into a grid structure according to the window position. The spatial size of this grid is half of the original size, which is the view compression feature.

[0201] Step 605: Flatten the view compression features corresponding to each viewpoint to obtain the one-dimensional flattened compression features corresponding to each viewpoint.

[0202] Flattening refers to the operation of unfolding the viewpoint compression features of a two-dimensional mesh structure into a one-dimensional sequence along the spatial dimension. That is, it involves arranging feature vectors from multiple spatial locations sequentially to form a long one-dimensional vector. A one-dimensional flattened compressed feature is a single-size feature vector obtained after flattening.

[0203] In some embodiments, the processor traverses each spatial location of the view compression feature in row-major order, that is, it extracts the feature vector of each spatial location row by row and column by column starting from the first row and first column of the grid; and concatenates each extracted feature vector end to end to form a complete one-dimensional vector, that is, a one-dimensional flattened compression feature.

[0204] Step 606: The one-dimensional flattened and compressed features corresponding to multiple viewpoints are spliced ​​together according to a preset viewpoint order to obtain multi-view fusion features.

[0205] Preset view order refers to the order in which multiple viewpoints are arranged during stitching. For example, a fixed order from head-mounted camera to wrist camera to side camera.

[0206] Multi-view fusion features refer to a single feature vector obtained by sequentially splicing together the one-dimensional flattened and compressed features from all perspectives in a preset order.

[0207] In some embodiments, the processor obtains a preset viewpoint order list, for example, the list sequentially records the number identifiers of the head-mounted camera viewpoint, wrist camera viewpoint, left camera viewpoint, and right camera viewpoint; according to the order of the list, the processor sequentially extracts the one-dimensional flattened and compressed features corresponding to each viewpoint; the flattened feature of the first extracted viewpoint is used as the starting point for stitching, and the flattened features of each subsequent viewpoint are sequentially appended to the end of the stitched features to form an accumulated stitch; after all viewpoints are processed, the resulting long vector is the multi-view fusion feature.

[0208] Step 607: Normalize the multi-view fusion features to obtain the current search key.

[0209] In some embodiments, the processor calculates the L2 norm of the multi-view fused feature vector, that is, sums the squares of each element in the feature vector and takes the square root; divides each element in the multi-view fused feature vector by the L2 norm to make the normalized vector magnitude 1; and determines the normalized vector as the current search key.

[0210] In this embodiment, image feature extraction is performed by reusing the visual encoder of the action generation model, eliminating the need to train an additional independent retrieval encoder network. This avoids additional training data requirements and an increase in the number of model parameters, achieving zero-cost construction of the retrieval key generation module. The method can be directly applied to existing pre-trained visual language action models without modifying the model structure.

[0211] Figure 7 This is an exemplary flowchart illustrating the successful construction of a memory bank according to some embodiments of this application. In some embodiments, this process may be performed by an electronic device (such as its processor).

[0212] Step 701: Obtain historical observation information and historical action blocks corresponding to multiple historical moments in the historical successful action trajectory of the embodied robot, and obtain the progress evaluation value corresponding to each historical moment in the historical successful action trajectory.

[0213] A historical successful motion trajectory refers to the complete sequence of actions performed by an embodied robot in the past when it ultimately succeeded in performing a certain task. This trajectory can extend from the start of the task to the completion of the task.

[0214] Historical observation information refers to the multimodal perception data collected by the robot through sensors at a certain moment in the historical successful action trajectory, which may include multi-view images, robot body state and gripper state, etc.

[0215] A historical action block refers to a sequence of action instructions executed by the robot at a certain moment in a historical successful action trajectory. It may include control quantities such as joint angles and gripper opening and closing degrees at multiple consecutive time steps.

[0216] The progress evaluation score is a numerical score used to quantify the degree of task completion at a given moment. This score is output after processing the current observation information and task instructions by a pre-trained progress evaluator, and the value ranges from 0 to 1, with a higher value indicating a higher degree of task completion.

[0217] In some embodiments, the processor can obtain historical observation information and historical action blocks corresponding to multiple historical moments in the historical successful action trajectory of the embodied robot stored in the storage device by reading from the storage device or calling the data interface, and obtain the progress evaluation value corresponding to each historical moment in the historical successful action trajectory.

[0218] In some embodiments, obtaining the progress evaluation value at each moment in the historical successful action trajectory may include the following operations: S31, select multiple evaluation moments from each historical moment of the historical successful action trajectory at preset time intervals.

[0219] The multiple evaluation moments include the first evaluation moment, the second evaluation moment, and the third evaluation moment arranged in sequence.

[0220] The preset time interval refers to the fixed time step used to extract evaluation moments from consecutive moments of historical successful action trajectories.

[0221] The evaluation moment refers to the historical moment selected for progress evaluation. The first evaluation moment, the second evaluation moment, and the third evaluation moment are three adjacent evaluation moments arranged sequentially on the timeline. The first evaluation moment is the starting reference moment, the second evaluation moment is the moment after the first evaluation moment with a preset time interval, and the third evaluation moment is the moment after the second evaluation moment with a preset time interval.

[0222] In some embodiments, the processor starts from the start time 0 of the historical successful action trajectory and selects evaluation times sequentially in steps of a preset time interval Δ until the total duration T of the trajectory is exceeded; the selected evaluation time sequence is 0, Δ, 2Δ, 3Δ up to the largest integer multiple Δ less than or equal to T; any three consecutive times in the sequence are respectively determined as the first evaluation time, the second evaluation time and the third evaluation time.

[0223] S32. Using a pre-trained progress evaluation model, based on historical observation information corresponding to the first evaluation time, historical observation information corresponding to the second evaluation time, task instructions, and task reference process, determine the change in the first task progress between the first evaluation time and the second evaluation time.

[0224] A pre-trained progress evaluation model refers to a neural network model that has been trained in advance. This model can take the observed image sequence at two time points, the task instructions, and the task reference process as input, and output the numerical value of the change in task progress between those two time points.

[0225] Task instructions refer to abstract descriptions of the tasks the robot is currently performing, such as natural language instructions like folding clothes or tidying up a table.

[0226] A task reference process refers to a standard demonstration procedure or reference trajectory for completing a task, used to provide a benchmark for progress evaluation.

[0227] The first task progress change refers to the change in the task completion progress between the first assessment time and the second assessment time. A positive value indicates that the task is moving forward, a negative value indicates that the task has regressed, and a zero value indicates that the task status remains unchanged.

[0228] In some embodiments, the processor reads historical observation information corresponding to the first and second evaluation times from the historical successful action trajectories; stacks the multi-view images of the first and second evaluation times in a time dimension to form an image sequence tensor with a time span of a preset time interval; concatenates the image sequence tensor with the text encoding features of the task instructions and the feature representation of the task reference process as input to the progress evaluation model; the progress evaluation model processes the input features through a multi-layer transformer network and outputs a scalar value, which is the first task progress change between the first and second evaluation times.

[0229] S33, based on the progress assessment value at the first assessment time and the change in the progress of the first task, determine the progress assessment value at the second assessment time.

[0230] The progress assessment value corresponding to the first assessment moment refers to the known quantitative score of the degree of task completion at that moment.

[0231] The progress assessment value at the second assessment time refers to the task completion score at the second assessment time, which is obtained by adding the progress assessment value at the first assessment time to the change in the progress of the first task.

[0232] In some embodiments, the processor reads the progress evaluation value corresponding to the first evaluation time from the memory cache. If the first evaluation time is the start time of the trajectory, the value is the preset initial progress value of 0. The progress evaluation value corresponding to the first evaluation time is added to the first task progress change amount, and the sum is the progress evaluation value corresponding to the second evaluation time.

[0233] S34. Using a pre-trained progress evaluation model, based on historical observation information corresponding to the second evaluation time, historical observation information corresponding to the third evaluation time, task instructions, and task reference process, determine the change in the second task progress between the second and third evaluation times.

[0234] The second task progress change refers to the change in the task completion progress between the second assessment time and the third assessment time.

[0235] The calculation method for this step is the same as the method described above for calculating the change in the progress of the first task, but the input data is changed to the historical observation information corresponding to the second and third evaluation times.

[0236] S35, based on the progress assessment value at the second assessment time and the change in the progress of the second task, determine the progress assessment value at the third assessment time.

[0237] The progress assessment value at the third assessment time is the task completion score obtained by adding the progress assessment value at the second assessment time to the progress change of the second task. Through the recursive calculations in S31-S35 above, the progress assessment values ​​at all assessment times can be obtained sequentially from the progress assessment value at the initial time.

[0238] In this embodiment, a pre-trained progress evaluation model automatically calculates the change in task progress between adjacent evaluation moments, eliminating the need for manual annotation of progress values ​​frame by frame. This significantly reduces the human and time costs of data annotation during the construction of the successful memory bank, making it possible to quickly build a high-quality successful memory bank in large-scale deployment scenarios. Simultaneously, the positive or negative value of the task progress change is used to determine whether the task is progressing forward or regressing within adjacent time intervals. This provides a unified progress measurement benchmark for peak progress filtering in subsequent successful memory bank construction and failure interval identification in failed memory bank construction. This ensures that the distinction between effective progress segments and regressive segments during memory writing is based on the same progress evaluation system, guaranteeing consistency and comparability of successful and failed memories in terms of selection criteria.

[0239] Step 702: Based on the progress evaluation value corresponding to each historical moment in the historical successful action trajectory, determine the progress peak moment of the historical successful action trajectory.

[0240] The peak progress moment refers to the historical moment when the progress assessment value reaches its maximum value in the historical successful action trajectory.

[0241] In some embodiments, the processor iterates through all progress evaluation values ​​to determine the peak progress time.

[0242] In some embodiments, the processor performs smoothing filtering on the progress evaluation value sequence to eliminate local jitter in the progress values ​​caused by observation noise, and finds the moment corresponding to the maximum value in the smoothed sequence as the progress peak moment.

[0243] Step 703: Extract historical observation information and historical action blocks from the historical successful action trajectories for each historical moment before the progress peak moment.

[0244] In some embodiments, the processor reads historical observation information and historical action blocks for each historical moment before the progress peak moment from the historical successful action trajectory based on the progress peak moment index.

[0245] In some embodiments, when the processor extracts historical moments before the progress peak moment, it sets a minimum progress threshold and extracts only the data at moments when the progress evaluation value is greater than the minimum threshold, so as to filter out invalid frames that have not yet started effective operation in the initial stage of the task; or it extracts the data in the interval before the progress peak moment where the progress evaluation value shows a monotonically increasing trend, excludes the data in the interval where the progress evaluation value decreases, and retains only the action segments that actually drive the task forward.

[0246] Step 704: Generate corresponding historical success retrieval keys based on the extracted historical observation information, and associate the historical success retrieval keys with the corresponding historical action blocks and store them in the success memory bank; the historical action blocks are used as priors for candidate success actions.

[0247] Historical successful retrieval keys refer to standardized feature vectors generated from extracted historical observation information through feature extraction and encoding processes, used for memory retrieval. Their generation method can be the same as that described earlier for generating the current retrieval key.

[0248] Related storage refers to writing historical successful retrieval keys and historical action blocks as key-value pairs into the data structure of the success memory, where the historical successful retrieval key serves as the index key during the query, and the historical action block serves as the value returned after the query is successful.

[0249] Candidate successful action priors refer to the historical action blocks stored in the success memory.

[0250] In some embodiments, the processor can process each historical moment in the extracted prefix trajectory sequence according to the multi-view images in its historical observation information. Figure 6 The process involves visual encoder encoding, spatial block feature rearrangement, pooling downsampling, flattening, multi-view stitching, and normalization processing in sequence to generate the historical successful retrieval key corresponding to that moment. The historical successful retrieval key and the historical action block at that moment are written as key-value pairs into the hash table structure of the success memory, where the key is the historical successful retrieval key and the value is the historical action block.

[0251] In this embodiment, the peak progress moment in the historical successful action trajectory is determined by the progress evaluation value, and only the historical observation information and historical action blocks before the peak moment are extracted. This ensures that the actions written into the success memory are effective action segments that truly drive the task forward. It avoids storing redundant actions after the peak moment, irrelevant actions after the task has been completed, or extra actions caused by the delay in the task end judgment into the success memory, thus ensuring the purity of the samples in the memory from the source.

[0252] Figure 8 This is an exemplary flowchart illustrating the successful construction of a memory bank according to other embodiments of this application. In some embodiments, this process may be performed by an electronic device, such as its processor.

[0253] Step 801: After executing the target action, obtain the task execution result corresponding to the task to be executed.

[0254] The task execution result refers to the information used to determine whether the task to be executed has been successfully completed after the robot has performed the target action. The task execution result can include two states: success and failure.

[0255] The execution result can be determined through visual verification, progress assessment, or manual confirmation. For example, in the task of folding clothes, after the robot completes the folding action, it uses a head-mounted camera to capture an image of the folded clothes and determines whether the clothes have been folded into a neat rectangular shape. If so, the task execution result is successful; otherwise, it is considered a failure.

[0256] Step 802: Calculate the action deviation metric between the target's executed action and the target's successful action prior, and calculate the scene difference metric based on the current observation information and the historical observation information corresponding to the target's successful action prior.

[0257] Action deviation metric is a quantitative value used to characterize the degree of difference in action space between the target's executed action and the prior successful action of the target. This metric can be obtained by calculating the Euclidean distance or multivariate dynamic time warped distance between two action sequences.

[0258] Scene difference metric is a quantitative value used to characterize the degree of difference in state space between current observation information and historical observation information corresponding to the prior successful action of the target. This metric can be obtained by calculating the cosine distance or Euclidean distance between two observation feature vectors.

[0259] Step 803: If the task execution result is successful and the action deviation metric is less than the preset deviation threshold, then the current working scenario is determined to be stable. The current observation information and the corresponding target execution action are used as new candidate successful action priors and stored in the success memory bank with the first memory weight.

[0260] The preset deviation threshold is a critical value used to determine whether the action deviation is within an acceptable range. When the action deviation metric is less than this threshold, it means that the current action is highly consistent with the historical successful prior.

[0261] The current working environment is stable, meaning that the robot's working environment is basically consistent with the environmental conditions corresponding to the historical successful experiences in the memory bank, and no significant changes have occurred.

[0262] The first memory weight refers to the baseline weight coefficient used to control the degree of influence of newly added action priors in the memory bank, for example, set to 1.0.

[0263] Adding a new candidate successful action prior refers to writing the observation information and action information corresponding to the current successful execution as a new successful experience into the memory.

[0264] In some embodiments, the processor compares the action deviation metric with a preset deviation threshold. If the action deviation metric is less than the preset deviation threshold, it enters the stable scene storage branch; the current observation information at the current execution moment is then processed according to the preceding text. Figure 6The method generates the current search key; the current search key and the current target action are used as a key-value pair, and the initial value of the first memory weight can be 1.0.

[0265] Step 804: If the task execution result is successful and the action deviation metric is greater than or equal to the preset deviation threshold, it is determined that the working scene of the embodied robot has experienced environmental drift. The current observation information and the corresponding target execution action are stored in the success memory bank with a second memory weight greater than the first memory weight. The retention priority of the historical candidate successful action prior in the success memory bank is reduced according to the scene difference metric.

[0266] Environmental drift refers to a significant change in the robot's working environment relative to the environmental conditions corresponding to historical successful experiences stored in its memory bank. Examples of changes include changes in lighting conditions, shifts in the position of the folding table, and changes in the material of clothing.

[0267] The second memory weight is a weighting coefficient greater than the first memory weight. It can be used to enhance the impact of new experiences on memory retrieval in context-shifting scenarios, for example, it can be set to 2.0.

[0268] Retention priority refers to the priority of each action prior in the success memory being hit during retrieval. The higher the priority, the greater the bonus obtained in similarity calculation. Lowering retention priority means that after environmental drift, the system is more inclined to match newly accumulated experience rather than historical experience.

[0269] In some embodiments, the processor compares the action deviation metric with a preset deviation threshold. If the action deviation metric is greater than or equal to the preset deviation threshold, it enters the environment drift storage branch. The current search key and the current target action are written into the success memory with a second memory weight greater than the first memory weight, such as 2.0. The priority decay coefficient is calculated based on the scene difference metric. The decay coefficient is equal to 1 minus the product of the scene difference metric and the minimum value of the preset decay factor, ensuring that the decay coefficient is kept in the range of 0 to 1. All historical candidate successful action priors in the success memory are traversed, and their priority is multiplied by the decay coefficient.

[0270] Step 805: If the task execution result is failure, generate a failure retrieval key based on the current observation information, generate candidate failure action priors based on the target execution action, determine the failure risk score based on the degree of failure of the task execution result, and associate and store the failure retrieval key, candidate failure action priors and failure risk score in the failure memory.

[0271] In some embodiments, when the task execution result is determined to be a failure, the failure memory writing process is initiated; a failure retrieval key is generated based on the current observation information; the action sequence corresponding to the current target execution action is used as a candidate failure action prior; a failure risk score is determined according to the degree of failure in the task execution result, the degree of failure can be determined by the progress evaluation value, and the greater the difference between the final progress evaluation value and the target progress 1, the higher the failure risk score; the failure retrieval key, the candidate failure action prior, and the failure risk score are written as a failure action prior into the storage structure of the failure memory bank.

[0272] In some embodiments, the processor may write the failure action prior to the failure memory in the manner described in the following embodiments: For the i-th test trajectory, a temporary cache is maintained during the action execution process, as shown in the following formula: (14) in, This indicates the search key extracted based on the current observation information. This indicates the action block that is executed at that moment.

[0273] Let the progress evaluation interval be . , No. The set of evaluation times for each trajectory is: (15) Let two adjacent evaluation times be... and The schedule variation within a given interval is given by the schedule estimator: (16) in, This represents the progress estimator for pre-training. Indicates task instructions. This indicates the task reference process.

[0274] when When, it indicates that the interval propels the task towards its objective; when When this occurs, it indicates that the time interval caused a rollback in task progress. To avoid recording minor fluctuations or irrelevant actions into the failure memory, this application sets a negative progress threshold. ,in .

[0275] When a certain interval satisfies If so, then the interval is considered a failure-inducing interval, that is, from arrive The execution of the action resulted in a significant rollback in task progress. At this point, the state-action fragment within that interval is written to the failure memory. (17) in, This represents the failure risk score, used to measure the severity of the failed segment. It is defined as follows: Normalized form can also be used: (18) in, This is the risk normalization coefficient.

[0276] The failure memory bank has been updated to: (19) If there are multiple intervals that satisfy the negative progress condition within the same trajectory, then all corresponding intervals are written to the failure memory: (20) Therefore, the failure memory can be represented as: (twenty one) in, The search key indicating a failure status. This indicates the action segment that caused the task progress to regress. This indicates the corresponding failure risk score.

[0277] Unlike directly saving the entire failure trajectory, this application only saves the ones that meet the requirements. The local negative progress interval. This avoids the failure memory being contaminated by a large number of irrelevant actions, allowing the failure memory to more centrally represent the key states – action patterns – that lead to task degradation.

[0278] In this embodiment, by comparing the action deviation measurement with the preset deviation threshold, the consistency between the current action and the historical successful prior can be automatically identified. When the action deviation is small, the scene is determined to be stable and the experience is stored with normal weight. When the action deviation is large, the environment drift is determined to have occurred and the new experience is stored with higher weight while the priority of the historical experience is reduced. This allows the content of the memory to be updated and adjusted adaptively with the changes in the environment, avoiding the problem of decreased retrieval accuracy of the static memory after environmental changes.

[0279] The following embodiment is an exemplary embodiment of the embodied robot action execution method described above: During the i-th task execution, the embodied robot receives the current observation features of the i-th task. This current observation information includes the current scene (e.g., the current camera image), the robot's body state, the gripper state, and task instructions. For the i-th task execution, the system can create a corresponding temporary cache and generate a retrieval key at time t in this temporary cache based on the current observation information. This retrieval key is used to represent the current observation state and to search for "successful action segments similar to the current state" in the online success memory. Because this round of the task has not yet ended, the system does not know whether this execution was successful or not, nor which segments are worth saving, so it temporarily stores them. After this task ends, the system uses a progress estimator to determine the outcome.

[0280] For details, please refer to the following formula (22): (twenty two) in, For example, the first time folding clothes, the second time folding clothes; This is the nth time step in this mission; For the first Temporary cache for this test task; For the first The retrieval key at time t in the test task, which is the feature extracted from the current observation; For the first The action fragment generated or executed at time t in the test task; For the first Total duration of each test task.

[0281] The method for generating search keys can be found in the preceding text. Figure 6 Related descriptions.

[0282] Taking folding clothes as an example, when the robot performs the folding task for the third time, it will record the process in a temporary cache as it performs the task: t=0: Observe the initial state of the clothes → Action: Approach the clothes; t=1: The gripper approaches the cuff → Action: Adjust the gripper position; t=2: Clamp the cuff → Action: Close the clamp; t=3: Begin folding towards the center line → Action: Move and rotate the gripper; t=4: Sleeve folded into place → Action: Release gripper; t=5: Unnecessary dragging or reversal occurs; The success memory includes the historical search key and historical action block for each successful entry. The matching degree between the current search key and historical search keys is calculated, and a set of candidate action blocks is determined based on the matching degree. This is essentially a similarity-gated successful action retrieval. Given the current search key... For each memory entry (action prior) The cosine similarity is used to calculate its correlation with the current state: Select the top ones with the highest similarity. We have 10 candidate options, and we use a threshold to remove weakly correlated results to obtain an initial candidate set.

[0283] Based on the current search key, retrieve historical search keys that match the current search key, i.e., find historical observation features similar to the current observation information. Retrieve the historical action block corresponding to the historical search key from the successful memory bank. For example, if the robot currently sees "the gripper is close to the edge of the bowl, and the bowl is slightly to the left of the center of the table", and there is a similar visual state in the memory bank, and at that time "the gripper retracted inward, slightly lifted, and clamped", that historical action block will be retrieved.

[0284] Furthermore, trajectory consistency filtering based on multivariate dynamic time warping (see above) Figure 2 (Related description). Since similar states do not necessarily mean suitable actions, as the task stage, contact state, and object posture may differ, the multivariate dynamic time warping (DTW) distance between action blocks in the candidate set can be obtained, as shown in formula (23): (twenty three) in, To calculate the multivariate dynamic time-warped distance, and For different candidate action blocks.

[0285] For each candidate action block The median distance between it and other candidate action blocks is calculated based on multivariate dynamic time warping (DTW) distance and used as the trajectory inconsistency score, as shown in formula (24). (twenty four) Trajectory Inconsistency Score The larger the inconsistency score, the less the action block resembles other candidate action blocks, and the more likely it is to be an abnormal action block that deviates from the mainstream success trajectory pattern. Therefore, by removing candidates with excessively high inconsistency scores, we obtain the final set of target action blocks (the prior set of target success actions).

[0286] Furthermore, the prior aggregation of target successful actions, i.e. the prior set of target successful actions: after obtaining the set of target action blocks, soft aggregation is performed on multiple action blocks in the target action block set.

[0287] First, the weight of each action block is determined based on the similarity, as shown in formula (25): (25) in Temperature parameters used to control the sharpness of the weight distribution. The smaller the value, the more the weight is concentrated on the most similar candidate. The larger the value, the closer the weight distribution is to a uniform distribution.

[0288] Furthermore, based on the weights corresponding to each action block, a weighted sum is performed on each action block to obtain the prior knowledge of the target successful action. As shown in formula (26): (26) in, As weight, This is an action block.

[0289] Since prior knowledge of successful target actions is not necessarily entirely reliable, this application proposes a confidence-based adaptive prior guidance approach. The retrieved prior knowledge of successful target actions is injected into the stream matching sampler as a soft generation constraint, and its strength is adjusted based on the confidence level of the prior knowledge. To ensure the guidance strength adapts to retrieval reliability, confidence scores are estimated from both state-level similarity and action-level consistency perspectives. The similarity between current observation information and matched historical observation features is calculated; higher similarity equates to higher confidence. Simultaneously, the consistency between elite action blocks is calculated; greater consistency among action blocks results in higher confidence. Finally, the target execution action of the embodied robot is generated based on the confidence-based adaptive guidance.

[0290] After completing a task, instead of adding all action blocks to the online success memory, a progress estimator determines which action blocks in the trajectory represent task progress, and then only retains the action blocks with the best progress from the successful trajectory. When the embodied robot performs a task, it generates current observation information and current action blocks at each moment, storing these in a temporary cache. After the task is completed, the progress estimator evaluates the task completion status of two adjacent moments. If the task completion status at the later moment is closer to the goal, then that moment is marked with "positive progress." If it deviates from the goal, regresses, or fails, it is marked with negative progress or low progress. These progresses are then accumulated to obtain a "task progress curve."

[0291] For example: Time 0: Progress 0; Time 1: Progress 20; Time 2: Progress 45; Time 3: Progress 80; Time 4: Progress 95; Time 5: Progress 70; Find the maximum progress and its time. If the maximum progress exceeds a threshold: save actions before or not exceeding the peak progress time; if the maximum progress does not exceed the threshold: discard the entire trajectory. The embodied robot automatically records the execution process during deployment, uses a task progress estimator to filter out successful observation-action segments before reaching a higher task progress, and writes them into an online memory for subsequent retrieval of similar states.

[0292] Based on the method described above, taking the task of folding clothes as an example, the embodied robot performs the task of folding clothes multiple times. When it successfully folds the sleeve to the center line of the clothes, the system does not simply discard this trajectory. Instead, it uses a progress estimator to determine which segments truly drive the task forward, such as "approaching the cuff," "grabbing the cuff," and "folding to the center line." The observation features and action segments of these successful segments are stored in the online success memory.

[0293] When the robot encounters a similar clothing state again, the system can first use the current image features to retrieve similar successful fragments from its memory. If multiple similar "folding sleeves" actions are retrieved, the system further checks whether the trajectories of these action fragments are consistent, removes candidates that deviate significantly, and then aggregates the remaining candidates into a "target successful action prior".

[0294] Generative VLA does not directly copy historical actions. Instead, it starts from an intermediate state that is close to the prior of a successful action for the target during sampling and continues to generate by combining the current observations and instructions. This utilizes the folding actions that have been successfully verified in the current environment while retaining the ability to adapt to subtle differences in the current clothing state.

[0295] The technical solution of this application can bring at least the following technical effects: (1) It can achieve adaptive testing without updating the frozen VLA parameters; (2) It can transform successful experiences in the deployment process into reusable online memories; (3) It can avoid storage failures, rollbacks and redundant actions by selecting the progress peak prefix; (4) It can improve the reliability of action priors by similarity retrieval and DTW consistency filtering; (5) It can avoid random errors caused by single nearest neighbor actions by generating target action priors through soft aggregation; (6) It can strengthen successful experiences in high-reliability retrieval and retain the original model generation capability in low-reliability retrieval by confidence-adaptive guidance; (7) It is suitable for robot operation tasks with long time, multiple stages and repeated deployment.

[0296] Reference Figure 9 The diagram shown is a functional module schematic of the embodied robot motion execution system 100 based on confidence-adaptive prior guidance of this application.

[0297] The embodied robot motion execution system 100 based on confidence-adaptive prior guidance described in this application is installed in an electronic device. Depending on the functions implemented, the embodied robot motion execution system 100 includes a first acquisition module 110, a second acquisition module 120, a first determination module 130, a second determination module 140, and a generation module 150. These modules can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0298] In this embodiment, the functions of each module / unit are as follows: The first acquisition module 110 is used to acquire the current observation information of the embodied robot for the task to be performed at the current moment, and generate the current search key based on the current observation information; The second acquisition module 120 is used to acquire a preset success memory bank and a preset failure memory bank; the success memory bank includes multiple candidate successful actions prior to historical execution, and the failure memory bank includes multiple candidate failed actions prior to historical execution. The first determining module 130 is used to determine, based on the current retrieval key, the target successful action prior associated with the current observation information from the successful memory, and to determine the target confidence level corresponding to the target successful action prior based on the current retrieval key; The second determining module 140 is used to determine the target failure exclusion item based on the current search key and the failure memory; The generation module 150 is used to determine action sampling parameters about the prior of the successful action of the target based on the target confidence, obtain random action noise, fuse the random action noise and the prior of the successful action of the target based on the action sampling parameters to obtain an initial execution action, and provide prior guidance to the action generation model for the action generation process of the task to be executed based on the initial execution action and the target failure exclusion term to obtain the target execution action; the action sampling parameters are used to characterize the guidance strength of the prior of the successful action of the target on the action generation process of the action generation model.

[0299] The specific implementation of the embodied robot motion execution system based on confidence-adaptive prior guidance in this application is largely the same as the specific implementation of the embodied robot motion execution method based on confidence-adaptive prior guidance described above, and will not be repeated here.

[0300] Reference Figure 10 The diagram shown is a schematic representation of a preferred embodiment of the electronic device of this application.

[0301] The electronic device includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. The memory 113 is used to store computer programs, such as a confidence-adaptive prior guidance program for executing embodied robot actions; In some embodiments, the processor 111 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 111 is typically used to control the overall operation of the electronic device, such as performing data interaction or communication-related control and processing. In this embodiment, the processor 111 is used to run program code stored in the memory 113 or process data.

[0302] The communication interface 112 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The communication interface 112 may also be used to establish a communication connection between the electronic device and other electronic devices.

[0303] The memory 113 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 113 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. In other embodiments, the memory 113 may also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. of the electronic device. Of course, the memory 113 may include both internal storage units and external storage devices of the electronic device. In this embodiment, the memory 113 is typically used to store the operating system and various computer programs installed on the electronic device, such as the program code of a embodied robot action execution program based on confidence-adaptive prior guidance. In addition, the memory 113 can also be used to temporarily store various types of data that have been output or will be output.

[0304] Figure 10Only an electronic device having a processor 111, a communication interface 112, a memory 113 and a communication bus 114 is shown. However, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0305] In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the embodied robot action execution method based on confidence-adaptive prior guidance provided in any of the foregoing method embodiments, including: Obtain the current observation information of the embodied robot regarding the task to be performed at the current moment, and generate the current search key based on the current observation information; Obtain a preset success memory bank and a preset failure memory bank; the success memory bank includes multiple prior candidate successful actions that were executed successfully in the past, and the failure memory bank includes multiple prior candidate failure actions that were executed unsuccessfully in the past. Based on the current retrieval key, determine the target successful action prior associated with the current observation information from the successful memory, and determine the target confidence corresponding to the target successful action prior based on the current retrieval key; Based on the current search key and the failure memory, determine the target failure exclusion item; Based on the prior knowledge of the target success action, the target confidence level, and the target failure exclusion term, the action generation model is given prior guidance for the action generation process of the task to be executed, so as to obtain the target execution action.

[0306] For a detailed explanation of the above steps, please refer to the above. Figure 1 A flowchart illustrating an embodiment of a method for executing embodied robot actions based on confidence-adaptive prior guidance.

[0307] Furthermore, this application also proposes a computer-readable storage medium that is both non-volatile and volatile. This computer-readable storage medium is any one or any combination of several of the following: hard disk, multimedia card, SD card, flash memory card, SMC, read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area. The program storage area stores a embodied robot action execution program based on confidence-adaptive prior guidance. When executed by a processor, the embodied robot action execution program based on confidence-adaptive prior guidance performs the following operations: Obtain the current observation information of the embodied robot regarding the task to be performed at the current moment, and generate the current search key based on the current observation information; Obtain a preset success memory bank and a preset failure memory bank; the success memory bank includes multiple prior candidate successful actions that were executed successfully in the past, and the failure memory bank includes multiple prior candidate failure actions that were executed unsuccessfully in the past. Based on the current search key, determine the target successful action prior associated with the current observation information from the successful memory, and determine the target confidence corresponding to the target successful action prior based on the current search key; Based on the current search key and the failure memory, determine the target failure exclusion item; Based on the prior knowledge of the target success action, the target confidence level, and the target failure exclusion term, the action generation model is given prior guidance for the action generation process of the task to be executed, so as to obtain the target execution action.

[0308] The specific implementation of the computer-readable storage medium in this application is largely the same as the specific implementation of the embodied robot action execution method based on confidence-adaptive prior guidance described above, and will not be repeated here.

[0309] It should be noted that the sequence numbers of the embodiments in this application are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof in this application are intended to cover non-exclusive inclusion, such that a process, system, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, system, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, system, article, or method that includes that element.

[0310] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware simulation platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0311] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for executing embodied robot actions based on confidence-adaptive prior guidance, characterized in that, The method includes: Obtain the current observation information of the embodied robot regarding the task to be performed at the current moment, and generate the current search key based on the current observation information; Obtain a preset success memory and a preset failure memory; the success memory includes multiple candidate successful action priors that have been executed successfully in the past and an action retrieval key corresponding to each candidate successful action prior; the failure memory includes multiple candidate failed action priors that have been executed unsuccessfully in the past. Obtain the action retrieval key corresponding to each candidate successful action prior in the success memory, and the first similarity between it and the current retrieval key; determine retrieval keys from the success memory whose first similarity to the current retrieval key is greater than a first threshold, as matching retrieval keys, and obtain the candidate successful action prior corresponding to the matching retrieval key from the success memory, as matching successful action prior; perform trajectory consistency filtering on the matching successful action prior to obtain filtered successful action prior; perform aggregation processing on the filtered successful action prior to obtain the target successful action prior associated with the current observation information, and the filtered successful action prior... The method involves multiple tests, determining a third similarity between the search key corresponding to each successful filtering action prior and the current search key, and determining a comprehensive state confidence between the current observation information and the historical observation information corresponding to the successful filtering action prior based on the third similarity; determining a trajectory difference metric among the multiple successful filtering action priors, and determining an action confidence corresponding to the multiple successful filtering action priors based on the trajectory difference metric; weighted fusing the state confidence and the action confidence to obtain a fused confidence, and determining the fused confidence as the target confidence corresponding to the target successful action prior; Based on the current search key and the failure memory, determine the target failure exclusion item; Based on the target confidence, action sampling parameters are determined regarding the prior of the successful action of the target. Random action noise is obtained, and the random action noise and the prior of the successful action of the target are fused based on the action sampling parameters to obtain the initial execution action. The action sampling parameters are used to characterize the guiding strength of the prior of the successful action of the target on the action generation process of the action generation model. Based on the initial execution action and the target failure exclusion term, the action generation model is given prior guidance for the action generation process of the task to be executed, so as to obtain the target execution action.

2. The method for executing actions of a hymenoided robot as described in claim 1, characterized in that, The aggregation process of the filtered successful action priors to obtain the target successful action priors associated with the current observation information includes: Obtain the second similarity between the search key corresponding to the prior of the successful filtering action and the current search key; The aggregation weight of the prior of the successful filtering action is determined based on the second similarity. The successful action priors of the filtered actions are weighted and summed using the aggregation weights to obtain the target successful action priors associated with the current observation information.

3. The method for executing actions of a hymenoided robot as described in claim 1, characterized in that, The step of determining the trajectory difference metric among the plurality of successful filtering action priors, and determining the action confidence corresponding to the plurality of successful filtering action priors based on the trajectory difference metric, includes: Obtain the multivariate dynamic time-warped distance between each of the filter success action priors and other filter success action priors; Obtain the median distance of the multivariate dynamic time-warped distance corresponding to the prior of each successful filtering action; Based on the median distances corresponding to the multiple successful filtering action priors, a trajectory difference metric is determined among the multiple successful filtering action priors; The action confidence level corresponding to the prior of the multiple successful filtered actions is determined based on the trajectory difference metric; the action confidence level and the trajectory difference metric have a negative correlation.

4. The method for executing actions of a hymenoided robot as described in claim 1, characterized in that, The step of determining the target failure exclusion item based on the current search key and the failure memory includes: Obtain the failure retrieval key, rollback action fragment, and failure risk score corresponding to each candidate failure action in the failure memory; Based on the failure retrieval key and the current retrieval key, calculate the state similarity between the current sampling state corresponding to the current observation information and the historical failure state corresponding to the candidate failure action prior. Candidate failed action priors with a state similarity greater than a second threshold are selected from the failure memory and used as candidate failure exclusion terms. Based on the state similarity corresponding to the candidate failure rejection item, the failure candidate weight corresponding to the candidate failure rejection item is determined, and the failure correlation between the current sampled state and the historical failure state is determined. The target failure exclusion item is determined based on the candidate failure exclusion item, the failure candidate weight, the failure relevance, and the failure risk score and rollback action fragment corresponding to the candidate failure exclusion item.

5. The method for executing actions of a hymenoided robot as described in claim 1, characterized in that, Based on the initial execution action and the target failure exclusion term, the action generation model is given prior guidance for the action generation process of the task to be executed, to obtain the target execution action, including: The initial execution action is determined as the intermediate action state corresponding to the first sampling step, and the intermediate action states corresponding to the current sampling step are determined sequentially according to the sampling step order; The motion generation model predicts the predicted velocity field corresponding to the intermediate motion state, and the current motion estimate corresponding to the current sampling step is determined based on the intermediate motion state and the predicted velocity field. Based on the current action estimate and the target failure rejection term, determine the failure rejection strength corresponding to the current sampling step; Based on the failure rejection strength, the predicted velocity field is corrected to obtain the corrected velocity field; The intermediate action state corresponding to the current sampling step is updated according to the corrected velocity field to obtain the intermediate action state of the next sampling step. The next sampling step is taken as the new current sampling step, and the above process is iteratively executed until the sampling ends. The intermediate action state obtained by the final sampling is determined as the target action to be executed.

6. The method for executing actions of a hymenoided robot as described in claim 5, characterized in that, The step of determining the failure rejection strength corresponding to the current sampling step based on the current action estimate and the target failure rejection term includes: The current action estimate and the candidate failed action priors in the target failure rejection term are respectively mapped to a preset action semantic space to obtain the semantic features of the current action estimate and the semantic features of the failed action corresponding to the target failure rejection term. Determine the semantic similarity between the estimated semantic features of the current action and the semantic features of the failed action; Based on the failure relevance, failure candidate weight, failure risk score, and semantic similarity corresponding to the target failure exclusion term, the failure energy value corresponding to the intermediate action state of the current sampling step is determined. The gradient information of the intermediate action state of the current sampling step is obtained based on the failure energy value, and the failure rejection strength corresponding to the current sampling step is obtained.

7. The method for executing actions of a hymenoided robot as described in claim 1, characterized in that, The step of generating the current search key based on the current observation information includes: Obtain image perception data corresponding to multiple viewpoints from the current observation information; The visual encoder of the action generation model encodes the image perception data corresponding to each viewpoint to obtain the spatial block feature set corresponding to each viewpoint. For each viewpoint, the spatial block features in the spatial block feature set corresponding to the viewpoint are rearranged into a two-dimensional feature grid according to their corresponding spatial positional relationship to obtain the viewpoint feature grid corresponding to the viewpoint. Pooling downsampling is performed on the view feature grid corresponding to the view to obtain the view compressed feature corresponding to the view; The view compression features corresponding to each view are flattened to obtain the one-dimensional flattened compression features corresponding to each view. The one-dimensional flattened and compressed features corresponding to the multiple viewpoints are spliced ​​together according to a preset viewpoint order to obtain multi-view fusion features. The multi-view fusion features are normalized to obtain the current search key.

8. The method for executing actions of a hymenoided robot as described in claim 1, characterized in that, The success memory bank is pre-built in the following manner: Obtain historical observation information and historical action blocks corresponding to multiple historical moments in the historical successful action trajectory of the android, and obtain the progress evaluation value corresponding to each historical moment in the historical successful action trajectory; Based on the progress evaluation value corresponding to each historical moment in the historical successful action trajectory, the peak progress moment of the historical successful action trajectory is determined. From the historical successful action trajectory, extract the historical observation information and historical action blocks for each historical moment before the progress peak moment; Based on the extracted historical observation information, a corresponding historical success retrieval key is generated, and the historical success retrieval key is associated with the corresponding historical action block and stored in the success memory bank; the historical action block is used as a priori for the candidate success action.

9. The method for executing actions of a hymenoided robot as described in claim 8, characterized in that, Obtain the progress evaluation value at each moment in the historical successful action trajectory, including: Multiple evaluation moments are selected from each historical moment of the historical successful action trajectory at preset time intervals; the multiple evaluation moments include a first evaluation moment, a second evaluation moment, and a third evaluation moment arranged in sequence; Using a pre-trained progress evaluation model, based on historical observation information corresponding to the first evaluation time, historical observation information corresponding to the second evaluation time, task instructions, and task reference process, the first task progress change between the first evaluation time and the second evaluation time is determined. Based on the progress assessment value at the first assessment time and the change in the progress of the first task, determine the progress assessment value at the second assessment time; Using a pre-trained progress evaluation model, based on historical observation information corresponding to the second evaluation time, historical observation information corresponding to the third evaluation time, the task instructions, and the task reference process, the second task progress change between the second evaluation time and the third evaluation time is determined; Based on the progress assessment value at the second assessment time and the change in the progress of the second task, the progress assessment value at the third assessment time is determined.

10. The method for executing actions of a hymenoided robot as described in claim 8, characterized in that, The method further includes: After executing the target action, obtain the task execution result corresponding to the task to be executed; Calculate the action deviation metric between the target's executed action and the target's successful action prior, and calculate the scene difference metric based on the current observation information and the historical observation information corresponding to the target's successful action prior; If the task execution result is successful and the action deviation metric is less than the preset deviation threshold, then the current working scenario is determined to be stable, and the current observation information and the corresponding target execution action are used as new candidate successful action priors and stored in the success memory bank with the first memory weight. If the task execution result is successful and the action deviation metric is greater than or equal to the preset deviation threshold, it is determined that the working scene of the embodied robot has experienced environmental drift. The current observation information and the corresponding target execution action are stored in the success memory bank with a second memory weight greater than the first memory weight. The retention priority of the historical candidate successful action prior in the success memory bank is reduced according to the scene difference metric. If the task execution result is a failure, a failure retrieval key is generated based on the current observation information, a candidate failure action prior is generated based on the target execution action, and a failure risk score is determined based on the degree of failure of the task execution result. The failure retrieval key, the candidate failure action prior, and the failure risk score are then associated and stored in the failure memory.

11. The method for executing actions of a hymenoided robot as described in any one of claims 1-10, characterized in that, The task to be performed is a clothing folding task; The current observation information includes visual images of the clothes to be folded on the folding table and the opening and closing status of the gripper at the end of the robotic arm of the embodied robot. The target execution action includes a continuous motion trajectory for controlling the end gripper of the robotic arm to grasp the sleeve of the garment to be folded and fold the sleeve to the center line of the garment body, and / or a continuous motion trajectory for controlling the end gripper of the robotic arm to fold the garment body in half.

12. A body robot motion execution system based on confidence-adaptive prior guidance, characterized in that, The system includes: The first acquisition module is used to acquire the current observation information of the embodied robot for the task to be performed at the current moment, and generate the current search key based on the current observation information; The second acquisition module is used to acquire a preset success memory bank and a preset failure memory bank; the success memory bank includes multiple candidate successful action priors that have been executed successfully in the past and an action retrieval key corresponding to each candidate successful action prior; the failure memory bank includes multiple candidate failed action priors that have been executed unsuccessfully in the past. The first determining module is configured to: obtain the action retrieval key corresponding to each candidate successful action prior in the successful memory, and the first similarity between it and the current retrieval key; determine retrieval keys from the successful memory that have a first similarity greater than a first threshold with the current retrieval key as matching retrieval keys, and obtain candidate successful action priors corresponding to the matching retrieval keys from the successful memory as matching successful action priors; perform trajectory consistency filtering on the matching successful action priors to obtain filtered successful action priors; and perform aggregation processing on the filtered successful action priors to obtain target successful action priors associated with the current observation information. The system includes multiple successful action priors, a third similarity between the search key corresponding to each filtered successful action prior and the current search key, and a comprehensive state confidence between the current observation information and the historical observation information corresponding to the filtered successful action prior based on the third similarity. It also determines a trajectory difference metric among the multiple filtered successful action priors and an action confidence corresponding to each of the multiple filtered successful action priors based on the trajectory difference metric. Finally, it performs a weighted fusion of the state confidence and the action confidence to obtain a fused confidence, which is then determined as the target confidence corresponding to the target successful action prior. The second determining module is used to determine the target failure exclusion item based on the current search key and the failure memory. The generation module is used to determine action sampling parameters about the prior of the successful action of the target based on the target confidence, obtain random action noise, fuse the random action noise and the prior of the successful action of the target based on the action sampling parameters to obtain an initial execution action, and provide prior guidance to the action generation model for the action generation process of the task to be executed based on the initial execution action and the target failure exclusion term to obtain the target execution action; the action sampling parameters are used to characterize the guidance strength of the prior of the successful action of the target on the action generation process of the action generation model.

Images