Action blocking method and system, electronic equipment and storage medium

Abstract

The invention provides an action blocking method and system, electronic equipment and a storage medium. A first action block is generated according to a first current observation; generating a second action block according to the second current observation and the first action block under the condition that the execution length of the first action block is equal to the reasoning delay step number; iteratively performing multiple times of generation processing according to the second action block to obtain a plurality of action blocks; the Nth generation processing comprises the following steps: performing dynamic weighting on each action in the (M-1) th action block, and freezing the head end part of the (M-1) th action block according to the weight of each action in the (M-1) th action block; repairing an unfrozen part in the (M-1) th action block by utilizing the (M-1) th action block and the (M-1) th action block after freezing; and executing the action in the unfrozen part in the repaired (M-1) th action block, and generating an Mth action block according to the repaired (M-1) th action block and the Mth current observation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and more particularly to an action block method and system, an electronic device and a storage medium. BACKGROUND

[0002] With the continuous development of science and technology, robots are increasingly widely used. In order to improve the working efficiency of the robot, the action blocks performed by the robot can be processed in blocks to predict the subsequent action blocks performed by the robot in real time, thereby improving the working efficiency of the robot.

[0003] In the prior art, action blocks are generally processed in blocks using synchronous action block and interpolation, naive asynchronous block, bidirectional decoding, diffusion model acceleration, etc. However, using the existing action block processing method to process action blocks not only causes obvious pauses and throughput decreases in the robot during work, but also causes acceleration mutations, severe reaction delays, and waste of computing resources. SUMMARY

[0004] Therefore, the present application provides an action block method, system, electronic device and storage medium to avoid the problems of obvious pauses, throughput decreases, acceleration mutations, severe reaction delays and waste of computing resources in real-time control of the robot.

[0005] The first aspect of the present application provides an action block method suitable for a robot, the method comprising:

[0006] obtaining a first current observation, and generating a first action block based on the first current observation;

[0007] In a case where the execution length of the first action block is equal to the inference delay step number, a second current observation is obtained, and a second action block is generated based on the second current observation and the first action block; wherein the end portion of the first action block overlaps with the beginning portion of the second action block;

[0008] iterative generation processing is performed based on the second action block to obtain a plurality of action blocks;

[0009] wherein the Nth generation processing comprises:

[0010] each action in the M-1th action block is dynamically weighted, and the beginning portion of the M-1th action block is frozen based on the weight of each action in the M-1th action block; wherein M is greater than or equal to 3, and N is greater than or equal to 1;

[0011] the unfrozen portion in the M-1th action block is repaired using the frozen M-1th action block and the M-1th action block;

[0012] Perform the action in the unfrozen part in the M-1 action block after repair, and generate the M action block according to the M-1 action block after repair and the M current observation; wherein, the end part of the M-1 is overlapped with the head part of the M action block.

[0013] Optionally, the first current observation is obtained, and the first action block is generated according to the first current observation, including:

[0014] When receiving the task instruction, the first current observation is obtained, wherein the first current observation includes the first current environment state, the first historical action block and the task target;

[0015] The first action block is generated by the policy model according to the first current environment state, the first historical action block and the task target, wherein the first action block includes a plurality of actions arranged in sequence.

[0016] Optionally, in the case that the execution length of the first action block is equal to the inference delay step number, the second current observation is obtained, and the second action block is generated according to the second current observation and the first action block, including:

[0017] In the case that the execution length of the first action block is equal to the inference delay step number, the second current observation is obtained, wherein the second current observation includes the second current environment state, the second historical action block and the task target;

[0018] According to the inference delay step number, a plurality of actions are selected from the last action of the first action block to obtain the end part of the first action block;

[0019] According to the second current environment state, the second historical action block and the task target, an initial second action block is generated;

[0020] The end part of the first action block is spliced as the head part of the second action block with the initial second action block to generate the second action block.

[0021] Optionally, each action in the M-1 action block is dynamically weighted, and the frozen part and the unfrozen part in the M-1 action block are determined according to the weight of each action in the M-1 action block, including:

[0022] For each action in the M-1 action block, it is judged whether the action length of the action is less than the inference delay step number;

[0023] If the action length of the action is less than the inference delay step number, the weight of the action is assigned as 1;

[0024] if the action length of the action is not less than the reasoning delay steps, determining whether the action length of the action is greater than a first length, wherein the first length is determined according to an action block length of the M-1 action block and a second length, and the second length is a difference between the action block length of the M-1 action block and a length of a tail end part thereof;

[0025] if the action length of the action is not greater than the first length, determining a distance coefficient corresponding to the action, and determining a weight corresponding to the action according to the distance coefficient; wherein the distance coefficient is determined according to the action block length of the M-1 action block, the second length, the reasoning delay steps and the action length of the action;

[0026] if the action length of the action is greater than the first length, assigning the weight of the action as 0;

[0027] freezing the action with a weight of 1 in the M-1 action block to complete freezing of the head end part of the M-1 action block.

[0028] Optionally, repairing the unfrozen part in the M-1 action block by using the frozen M-1 action block and the M-1 action block, comprises:

[0029] determining an M-1 velocity direction output by a diffusion model through the M-1 current observation, the M-1 action block and a generation time thereof;

[0030] determining an M-1 estimated action block before repair according to the M-1 velocity direction, the M-1 current observation, the M-1 action block and the generation time thereof;

[0031] determining a guiding correction value of each action of the frozen part in the frozen M-1 action block through the diffusion model according to the M-1 current observation, the M-1 action block and the generation time thereof, the frozen part in the frozen M-1 action block and the weight thereof, the M-1 velocity direction and the M-1 estimated action block;

[0032] repairing the corresponding action according to the guiding correction value of each action of the frozen part in the frozen M-1 action block.

[0033] Optionally, the determining of the guiding correction value of each action of the frozen part in the frozen M-1 action block through the diffusion model according to the M-1 current observation, the M-1 action block and the generation time thereof, the frozen part in the frozen M-1 action block and the weight thereof, the M-1 velocity direction and the M-1 estimated action block, comprises:

[0034] constructing, by the diffusion model, a gradient guide item of each action of the frozen part in the M-1th action block according to the weight of each action of the frozen part in the M-1th action block after freezing;

[0035] determining, by the diffusion model, an M-1th diffusion time weight item according to the generation time of the M-1th action block, and determining an M-1th guide intensity according to the M-1th diffusion time weight item, the generation time of the M-1th action block and a preset guide item intensity upper limit;

[0036] generating, by the diffusion model, a guide correction value of each action of the frozen part in the M-1th action block after freezing according to the M-1th velocity direction, the M-1th guide intensity, the frozen part in the M-1th action block after freezing, the M-1th estimated action block, the gradient guide item of each action of the frozen part in the M-1th action block after freezing and the M-1th action block.

[0037] The second aspect of the present application provides an action block system suitable for a robot, the system comprising:

[0038] a first generation module configured to obtain a first current observation and generate a first action block according to the first current observation;

[0039] a second generation module configured to obtain a second current observation when the execution length of the first action block is equal to the reasoning delay step number, and generate a second action block according to the second current observation and the first action block; wherein the end part of the first action block overlaps with the start part of the second action block;

[0040] a third generation module configured to iteratively perform multiple generation processes according to the second action block to obtain multiple action blocks;

[0041] wherein the third generation module of the Nth generation process comprises:

[0042] a freezing unit configured to dynamically weight each action in the M-1th action block, and freeze the start part of the M-1th action block according to the weight of each action in the M-1th action block; wherein M is greater than or equal to 3, and N is greater than or equal to 1;

[0043] a repairing unit configured to repair the unfrozen part in the M-1th action block using the M-1th action block after freezing and the M-1th action block;

[0044] a first generation unit configured to execute the actions in the unfrozen part of the M-1th action block after repairing, and generate an Mth action block according to the M-1th action block after repairing and an Mth current observation; wherein the end part of the M-1th action block overlaps with the start part of the Mth action block.

[0045] Optionally, the first generating module comprises:

[0046] a first obtaining unit, configured to obtain a first current observation when a task instruction is received, wherein the first current observation comprises a first current environment state, a first historical action block and a task target;

[0047] a second generating unit, configured to generate a first action block according to the first current environment state, the first historical action block and the task target by using a policy model, wherein the first action block comprises a plurality of actions arranged in sequence.

[0048] The third aspect of the present application provides an electronic device, comprising a processor and a memory connected through a bus; wherein the processor is used to call and execute a program stored in the memory; and the memory is used to store a program, which is used to implement the action block method provided in the first aspect of the present application.

[0049] The fourth aspect of the present application provides a computer readable storage medium, wherein computer executable instructions are stored in the computer readable storage medium, and the computer executable instructions are used to execute the action block method provided in the first aspect of the present application.

[0050] The application provides an action block method, a system, an electronic device and a storage medium. A first current observation is obtained, and a first action block is generated according to the first current observation. In the case where the execution length of the first action block is equal to the inference delay step number, a second current observation is obtained, and a second action block is generated according to the second current observation and the first action block. The end part of the first action block overlaps with the start part of the second action block. The second action block is iteratively processed multiple times to obtain multiple action blocks. The Nth processing includes dynamically weighting each action in the M-1th action block, and freezing the start part of the M-1th action block according to the weight of each action in the M-1th action block. M is greater than or equal to 3, and N is greater than or equal to 1. The unfrozen part in the M-1th action block is repaired by using the frozen M-1th action block and the M-1th action block. The action in the unfrozen part of the repaired M-1th action block is executed, and the Mth action block is generated according to the repaired M-1th action block and the Mth current observation. The end part of the M-1th action block overlaps with the start part of the Mth action block. In the process of iteratively generating corresponding action blocks, the start part of the M-1th action block is frozen, and the unfrozen part in the M-1th action block is repaired by using the frozen M-1th action block and the M-1th action block, so that the continuity and policy stability between two action blocks connected in sequence can be effectively improved, and problems such as pause, throughput reduction, acceleration mutation, serious reaction delay and waste of computing resources of a robot in real-time control can be avoided. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0052] Figure 1 A flowchart of an action block method provided by an embodiment of the present application;

[0053] Figure 2 A structural diagram of an action block system provided by an embodiment of the present application;

[0054] Figure 3 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0057] However, using synchronous action segmentation and interpolation introduces visible pauses during the segmentation process, which not only slows down the robot's execution speed but also reduces task throughput. Furthermore, these pauses alter the robot's dynamics, leading to inconsistencies between training and evaluation distributions. Naive asynchronous segmentation results in abrupt acceleration changes, including extremely high and out-of-distribution accelerations. Using bidirectional decoding for action segmentation is computationally intensive, making it difficult to use for high-frequency control, and it fails to improve continuity.

[0058] See Figure 1 The diagram illustrates a flowchart of an action segmentation method provided in an embodiment of this application. This action segmentation method is applicable to robots and specifically includes the following steps:

[0059] S101: Obtain the first current observation and generate the first action block based on the first current observation.

[0060] During the specific execution of step S101, it is possible to detect in real time whether the robot's task instruction has been received. When the robot's task instruction is received, the robot's current observation can be obtained (for ease of distinction, the first current observation obtained is called the first current observation). The first current observation includes at least the first current environment state, the first historical action block, and the task target. The robot's first action block is generated using the first current environment state, the first historical action block, and the task target.

[0061] It should be noted that the task instruction specifies a corresponding task objective; the first current environment state refers to the environment state related to the robot's current environment when the robot's current observation is first acquired; the first historical action block can include at least one historical action block that the robot has executed most recently.

[0062] Optionally, the process of obtaining the first current observation and generating the first action block based on the first current observation can be as follows: when a task instruction is received, obtain the first current observation, wherein the first current observation includes the first current environment state, the first historical action block and the task objective; generate the first action block through the policy model based on the first current environment state, the first historical action block and the task objective, wherein the first action block includes multiple actions arranged in sequence.

[0063] It should be noted that the policy model is the core decision-making component. Its function is to generate a coherent and reasonable sequence of actions or behavioral instructions based on the environmental state, historical action blocks, and task objectives. Its core function is to establish a mapping relationship between "environmental state" and actions.

[0064] In other words, this application uses the first current environmental state, the first historical action block, and the task objective obtained by the robot through sensors as inputs to the policy model. The policy model then generates and outputs specific action instructions (including action blocks of multiple actions) based on the current input. For example, the generated first action block could be... .

[0065] S102: If the execution length of the first action block is equal to the inference delay step, obtain the second current observation and generate the second action block based on the second current observation and the first action block; wherein the end part of the first action block overlaps with the beginning part of the second action block.

[0066] In this embodiment of the application, the sampling period can be controlled in advance. (e.g., 20ms) and inference time (e.g., 60ms) so that the corresponding inference delay steps can be calculated based on the control sampling period and inference time. The calculation method of the inference delay steps is shown in formula (1).

[0067] (1)

[0068] in, For the reasoning delay steps, To control the sampling period, This is the reasoning time.

[0069] In the specific execution step S102, after the first action block is generated, the first action block can be executed, and the execution length of the first action block is detected in real time to see if it is equal to the predetermined inference delay steps. When the execution length of the first action block is detected to be equal to the inference delay steps, the robot's second current observation can be obtained, wherein the second current observation includes at least the second current environment state, the second historical action block and the task objective. The second action block is generated based on the first action block, the second current environment state, the second historical action block and the task objective.

[0070] It should be noted that the execution length can be the number of execution steps. Each action in an action block can be considered as one step. For example, if an action block contains 8 actions, then the number of steps in that action block can be 8.

[0071] In other words, if the inference delay step is 4, then it is possible to detect in real time whether the execution step of the first action block is equal to 4 (inference delay step).

[0072] It should also be noted that after generating the first action block, it is possible to first determine whether the robot has completed the task objective after executing the first action block. If it has, the current process can be ended. If it has not, the execution length of the first action block can be checked in real time while executing the first action block. If the execution length of the first action block is equal to the predetermined inference delay steps, the process of generating the second action block can be executed when the execution length of the first action block is detected to be equal to the inference delay steps.

[0073] Optionally, if the execution length of the first action block is equal to the inference delay steps, the process of obtaining the second current observation and generating the second action block based on the second current observation and the first action block can be as follows: if the execution length of the first action block is equal to the inference delay steps, obtain the second current observation, where the second current observation includes the second current environment state, the second historical action block, and the task objective; select multiple actions starting from the last action of the first action block according to the inference delay steps to obtain the end part of the first action block; generate the initial second action block based on the second current environment state, the second historical action block, and the task objective; and concatenate the end part of the first action block as the beginning part of the second action block with the initial second action block to generate the second action block.

[0074] In practical applications, when the execution length of the first action block is detected to be equal to the inference delay steps, the robot's second current environment state, second history block, and task objective are acquired. Starting from the last action in the action sequence of the first action block, actions with an inference delay step count are selected from back to front to obtain the end part of the first action block. The second current environment state, second history block, and task objective are input into the policy model, causing the policy model to generate an initial second action block based on the second current environment state, second history block, and task objective. The end part of the first action block is used as the beginning part of the second action block. Finally, the beginning part of the second action block is concatenated with the initial second action block to obtain the second action block. The end part of the first action block overlaps with the beginning part of the second action block.

[0075] It should be noted that, in order to ensure that the newly generated second action block is connected with the "actions that have not yet been executed" in the first action block and to avoid action breakage, the end part of the current action block (the first action block) can be used as the beginning part of the new action block (the second action block). This ensures that the number of actions in the new action block before the inference delay step is exactly the same as the number of actions in the current action block after the inference delay step, thereby ensuring that there are no jumps in the action sequence when switching and achieving a smooth transition.

[0076] For example, the generated action sequence of the first action block includes 8 actions, such as the first action block A0 = [a0, a1, a2, a3, a4, a5, a6, a7]; assuming the calculated inference delay step d = 4, when the execution length of the first action block A0 is detected to be equal to 4, that is, the first action block A0 has already executed 4 actions (i.e., (It has been actually issued and executed); the number of inference delay steps from back to front in the first action block can be taken as the end part of the first action block, and the end part of the first action block can be taken as the beginning part of the second action block, that is, the end part of the first action block is equal to the beginning part of the second action block, as shown in formula (2).

[0077] (2)

[0078] in, This is the end part of the first action block. This represents the beginning of the second action block, where s is the step number corresponding to the nth inference delay step from the end in the first action block; correspondingly, This is the starting index of the actions already executed in the first action block, meaning it directly reuses the d actions executed during the inference period corresponding to the inference delay step. The actions within the second action block are designed to ensure that the first d actions within the second action block are consistent with the d actions in the first action block that were not executed during reasoning (the last d actions in the first action block), thereby ensuring behavioral coherence and causal consistency.

[0079] For example, if the first action block A0 = [a0, a1, a2, a3, a4, a5, a6, a7], then s can be 4, which means... That is, the first four actions (b0, b1, b2, b3) of the second action block A1 are equal to those of the first action block. ).

[0080] In practical applications, s can be determined from the action sequence of the first action block based on the inference delay steps, and the corresponding s can be sent to the controller so that the controller can start from the action corresponding to the step number s in the action sequence of the first action block and extract the inference delay steps as the end part of the first action block.

[0081] It should be noted that the number of steps s should satisfy greater than or equal to d and less than or equal to Hd, i.e., d ≤ s ≤ Hd. Here, d is the inference delay step, and H is the action length of the action block.

[0082] S103: Iterate through the second action block multiple times to obtain multiple action blocks; wherein, the Nth generation process includes: dynamically weighting each action in the (M-1)th action block, and freezing the beginning part of the (M-1)th action block according to the weight of each action in the (M-1)th action block; wherein, M is greater than or equal to 3, and N is greater than or equal to 1; repairing the unfrozen part of the (M-1)th action block using the frozen (M-1)th action block and the (M-1)th action block; executing the actions in the unfrozen part of the repaired (M-1)th action block, and generating the Mth action block according to the repaired (M-1)th action block and the current observation of the Mth action block; wherein, the end part of the (M-1)th action block overlaps with the beginning part of the Mth action block.

[0083] In the specific execution step S103, after generating the second action block, it can be determined whether the robot has completed the task objective after executing the second action block. If it has, the current process can be ended. If it has not, while executing the second action block, it can detect in real time whether the execution length of the second action block is equal to the predetermined inference delay steps. When it is detected that the execution length of the second action block is equal to the inference delay steps, the corresponding generation process is executed to continue generating subsequent action blocks. That is, the second action block is iteratively generated multiple times to generate multiple tasks related to the task objective until the task objective is completed.

[0084] In this embodiment, the Nth generation process can be as follows: based on the action length (action steps) and inference delay steps corresponding to each action block in the (M-1)th action block, dynamically weight each action in the (M-1)th action block, and freeze the actions with a weight of 1 in the (M-1)th action block to complete the freezing of the beginning part of the (M-1)th action block; wherein, M is greater than or equal to 3, and N is greater than or equal to 1; using the generation time of the frozen (M-1)th action block, the generation time of the frozen (M-1)th action block, the frozen part in the frozen (M-1)th action block and its weight, repair the unfrozen part in the (M-1)th action block; execute the actions in the unfrozen part in the repaired (M-1)th action block, and generate the Mth action block based on the repaired (M-1)th action block and the current observation of the Mth action block; wherein, the end part of the (M-1)th action block overlaps with the beginning part of the Mth action block.

[0085] It should be noted that during the execution of the actions within the repaired (M-1)th action block, it is possible to detect in real time whether the execution length of the repaired (M-1)th action block is equal to the predetermined inference delay steps. When it is detected that the execution length of the repaired (M-1)th action block is equal to the inference delay steps, the robot's Mth current observation can be obtained. The Mth current observation includes at least the Mth current environmental state, the Mth historical action block, and the task objective. The Mth action block is generated based on the repaired (M-1)th action block, the Mth current environmental state, the Mth historical action block, and the task objective.

[0086] Furthermore, in this embodiment, before executing the repaired (M-1)th action block, the completion time of the repaired (M-1)th action block can be determined, and the required generation time of the (M-1)th action block can be determined according to the task objective; it can be determined whether the completion time of the repaired (M-1)th action block is equal to the required generation time of the (M-1)th action block, whether the actions within the repaired (M-1)th action block are continuous, and whether the repaired (M-1)th action block conforms to the current environment state of (M-1); if it is determined that the completion time of the repaired (M-1)th action block is equal to the required generation time of the (M-1)th action block, whether the actions within the repaired (M-1)th action block are continuous, and whether the repaired (M-1)th action block conforms to the current environment state of (M-1); If the (M-1)th action block conforms to the current environment state of the (M-1)th action block, it is determined that the repaired (M-1)th action block is available, and each action within the repaired (M-1)th action block is executed; if it is determined that the completion time of the repaired (M-1)th action block is not equal to the required generation time of the (M-1)th action block, and / or, each action within the repaired (M-1)th action block is continuous, and / or, the repaired (M-1)th action block conforms to the current environment state of the (M-1)th action block, it is determined that the repaired (M-1)th action block is unavailable, the (M-2)th action block can be updated to the (M-1)th action block, and the step of dynamically weighting each action within the (M-1)th action block is returned to be executed, so as to freeze and repair the (M-1)th action block again.

[0087] Optionally, the process of dynamically weighting each action within the (M-1)th action block and determining the frozen and unfrozen portions of the (M-1)th action block based on the weight of each action can be as follows: For each action within the (M-1)th action block, determine whether the action length is less than the inference delay steps; if the action length is less than the inference delay steps, assign the action weight to 1; if the action length is not less than the inference delay steps, determine whether the action length is greater than a first length, where the first length is determined based on the action block length and the second length of the (M-1)th action block, and the second length is the difference between the action block length of the (M-1)th action block and the length of its end portion; if the action length is not greater than the first length, determine the distance coefficient corresponding to the action, and determine the weight corresponding to the action based on the distance coefficient; where the distance coefficient is determined based on the action block length of the (M-1)th action block, the second length, the inference delay steps, and the action length; if the action length is greater than the first length, assign the action weight to 0; freeze the actions with a weight of 1 within the (M-1)th action block to complete the freezing of the beginning portion of the (M-1)th action block.

[0088] In some embodiments, the difference between the action length of the (M-1)th action block and the length of its end portion can be calculated, i.e., the difference between the action length and the inference delay steps can be calculated to obtain the second length; the difference between the action length and the second length can be calculated to obtain the first length. For example, assuming the action length is H and the inference delay steps are d, then the number of steps s corresponding to the second length can be Hd, and thus the first length can be Hs.

[0089] It should be noted that for each action block generated after the first action block, i.e., for the (M-1)th action block, in order to improve cross-block continuity, reduce jerky behavior, enhance policy stability, and better determine which parts are repairable and which are not (frozen parts) in the subsequent repair process, the beginning part of the (M-1)th action block can be frozen after its generation. Specifically, the weight of each action in the (M-1)th action block can be determined so that the beginning part of the (M-1)th action block can be frozen precisely based on the weight of each action in the (M-1)th action block.

[0090] In practical applications, after generating the (M-1)th action block, for each action within the (M-1)th action block, the corresponding action length (action steps) can be obtained, and it can be determined whether the action steps are less than the inference delay steps. If they are less, it means that this action is the action of the previous action block, that is, the action that has been executed in the previous stage, and must be frozen and unchanged for reference only. At this time, a hard mask operation can be directly performed on the action to assign a weight of 1 to the action.

[0091] If it is not less than, we can further determine whether the action length of the action is greater than the first length, that is, whether the action length of the action is within the preset interval ([d, Hs]); if it is not greater than the first length, it means that the action length of the action is within the preset interval, that is, this action needs to refer to the previous action, and the action can be assigned the corresponding weight by using a soft mask, that is, the action can be assigned the corresponding weight by using an exponential decay method. Specifically, the corresponding distance coefficient can be calculated based on the action block length of the M-1th action block, the second length, the inference delay steps and the action length of the action, and the weight corresponding to the action can be calculated based on the distance coefficient, as shown in formula (3).

[0092] If the length is greater than the first length, it means that the action is a completely new action and does not need to refer to the previous actions. At this time, a hard mask operation can be performed on the action to assign a weight of 0 to the action. The way to assign weights to each action can be as shown in formula (4).

[0093] (3)

[0094] in, is the distance coefficient of the action, i represents the action length corresponding to the action, d is the inference delay step, H is the action block length of the M-1th action block, and s is the second length.

[0095] (4)

[0096] in, The weights corresponding to the actions, Here, is the distance coefficient for the action, i represents the action length corresponding to the action, d is the inference delay step, H is the action block length of the (M-1)th action block, and s is the second length. This is the first length.

[0097] It should be noted that, as can be seen from formulas (3) and (4), the larger the length of the action, the smaller the distance coefficient of the action, and the smaller the corresponding weight, and the smaller the need to refer to the previous action; correspondingly, the larger the weight of the action, the greater the need to refer to the previous action.

[0098] It's also worth noting that a hard mask uses a binary mask of 0 or 1 to indicate whether an action is frozen. A soft mask uses real-valued weights between [0,1] to assign a degree of "partial freezing" to each action, typically with exponential decay. Advantages of a soft mask include: smooth transitions between blocks: gradual guidance improves the naturalness of transitions between blocks and avoids abrupt switching; reduced jitter: alleviates jerky behavior and reduces acceleration abruptness; more stable policies: reduces distribution offset and improves deployment robustness; latency tolerance: maintains continuity even when d is small; flexible adaptation: adjustable weight decay adapts to different tasks and models.

[0099] In this embodiment of the application, in order to further achieve the continuity of action block stepping, the application uses the action block repair method based on Pseudoinverse Guidance to repair the action in the unfrozen part of the M-1th action block, and guides the denoising process to make the repaired action consistent with the frozen prefix.

[0100] Optionally, the process of repairing the unfrozen portion of the (M-1)th action block using the frozen (M-1)th action block and the (M-1)th action block can be as follows: determine the (M-1)th velocity direction output by the diffusion model using the current (M-1)th observation, the (M-1)th action block and its generation time; determine the estimated (M-1)th action block before repair based on the (M-1)th velocity direction, the current (M-1)th observation, the (M-1)th action block and its generation time; determine the guided correction value for each action of the frozen portion of the frozen portion in the frozen (M-1)th action block using the diffusion model based on the current (M-1)th observation, the (M-1)th action block and its generation time, the frozen portion and its weights in the frozen (M-1)th action block, the (M-1)th velocity direction, and the estimated (M-1)th action block; and repair the corresponding actions based on the guided correction value for each action of the frozen portion in the frozen (M-1)th action block.

[0101] As one implementation of this application, a gradient guidance term for each action of the frozen part within the frozen (M-1)th action block can be constructed using a diffusion model based on the weight of each action of the frozen part within the frozen (M-1)th action block; a diffusion model can determine a diffusion time weight term based on the generation time of the (M-1)th action block, and determine a guidance intensity based on the diffusion time weight term, the generation time of the (M-1)th action block, and a preset guidance term intensity upper limit; a guidance correction value for each action of the frozen part within the frozen (M-1)th action block can be generated using a diffusion model based on the velocity direction of the (M-1)th action block, the guidance intensity of the (M-1)th action block, the frozen part within the frozen (M-1)th action block, the estimated (M-1)th action block, the gradient guidance term for each action of the frozen part within the frozen (M-1)th action block, and the (M-1)th action block.

[0102] In some embodiments, after freezing the beginning part of the M-1th action, the M-1th velocity direction output by the diffusion model can be determined by the current observation of the M-1th action, the M-1th action block and its generation time, so as to determine the estimated M-1th action block before repair based on the M-1th velocity direction, the current observation of the M-1th action, the M-1th action block and its generation time. The specific determination method is shown in formula (5).

[0103] (5)

[0104] in, For the (M-1)th action block, Let M be the generation time of the (M-1)th action block. For the current observation of the M-1th time, The direction of the (M-1)th velocity (indicating the condition and the current observation of the (M-1)th velocity). and the generation time of the (M-1)th action block Below, the diffusion model predicts the update direction of the action block. The estimated action block before repair is the M-1th action block (meaning the M-1th action block plus the diffusion direction (velocity direction) forms the estimated action block before repair).

[0105] In some embodiments, after determining the (M-1)th velocity direction and the (M-1)th estimated action block, for each action in the unfrozen portion of the (M-1)th action block, the diffusion model constructs a gradient guidance term for that action based on its weight (for...). (A diagonal matrix constructed using the weights of the actions, used to construct the gradient guidance term); the (M-1)th diffusion time weight term is calculated based on the generation time of the (M-1)th action block, so that the (M-1)th guidance strength is determined based on the (M-1)th diffusion time weight term, the generation time of the (M-1)th action block, and the preset upper limit of guidance term strength (limited to a maximum of not exceeding). ), where the calculation method of the M-1 diffusion time weight term is as shown in formula (6); finally, based on the M-1 velocity direction, the M-1 guidance intensity, the frozen part in the M-1 action block after freezing, the M-1 estimated action block, the gradient guidance term of the action and the M-1 action block, the guidance correction value of the action is generated so as to repair the action based on the guidance correction value of the action, where the calculation method of the M-1 guidance correction value of the action is as shown in formula (7).

[0106] (6)

[0107] in, For the (M-1)th diffusion time weighting term, The generation time of the (M-1)th action block.

[0108] It should be noted that, It is a diffusion time dependent The weighting term controls how the guiding strength changes over time.

[0109] (7)

[0110] in, This is the guide correction value for the frozen portion of the action within the (M-1)th action block after freezing. For the (M-1)th action block, Let M be the generation time of the (M-1)th action block. For the current observation of the M-1th time, The direction of the (M-1)th velocity. For the estimated action block M-1 before repair, The strength of the (M-1)th leading term; The target is frozen (the frozen portion within the M-1th action block after freezing). For the gradient guiding term of the frozen portion of the action within the (M-1)th action block after freezing, These are preset parameters.

[0111] This application provides an action block segmentation method, which involves obtaining a first current observation and generating a first action block based on the first current observation; if the execution length of the first action block is equal to the inference delay steps, obtaining a second current observation and generating a second action block based on the second current observation and the first action block; wherein the end portion of the first action block overlaps with the beginning portion of the second action block; and performing multiple generation processes based on the second action block to obtain multiple action blocks; wherein the Nth generation process includes: performing a process on each action within the (M-1)th action block... The method involves dynamically weighting the actions and freezing the beginning portion of the (M-1)th action block based on the weight of each action within the (M-1)th action block, where M is greater than or equal to 3 and N is greater than or equal to 1. The frozen (M-1)th action block and its corresponding action block are then used to repair the unfrozen portions within the (M-1)th action block. Actions within the unfrozen portions of the repaired (M-1)th action block are then executed, and the (M)th action block is generated based on the repaired (M-1)th action block and the current observations. The ending portion of the (M-1)th action block overlaps with the beginning portion of the (M)th action block. The technical solution provided in this application, during the iterative generation of corresponding action blocks, effectively improves the continuity and policy stability between consecutive action blocks by freezing the beginning portion of the (M-1)th action block and repairing the unfrozen portions using the frozen (M-1)th action block and its corresponding action block. This avoids problems such as pauses, decreased throughput, sudden acceleration changes, severe reaction delays, and wasted computational resources during real-time robot control.

[0112] To better understand the action segmentation method provided in the above embodiments of this application, the following explanation is based on specific implementation methods.

[0113] In practical applications, this application provides a control thread, an inference thread, and a boot repair thread.

[0114] A1: Obtain the robot's first current observation through the control thread and write the first current observation into the observation buffer;

[0115] A2: The inference thread retrieves the first current observation from the observation buffer and generates the corresponding first action block based on the first current observation; the first action block is pushed into the action buffer, and the control thread is notified that the action is ready;

[0116] A3: Retrieve the first action block from the action buffer through the control thread and execute the first action block;

[0117] A4: During the execution of the first action block by the control thread, the inference thread checks in real time whether the execution length of the first action block is equal to the inference delay steps; whereby the inference delay steps are pre-calculated by the inference thread.

[0118] A4: When the inference thread detects that the execution length of the first action block is equal to the inference delay steps, it can notify the control thread.

[0119] A5: Obtain the robot's second current observation through the control thread and write the second current observation into the observation buffer;

[0120] A6: The inference thread retrieves the second current observation from the observation buffer and generates the corresponding second action block based on the second current observation and the first action block;

[0121] A7: Multiple action blocks are obtained by iterating through the control thread, inference thread, and bootstrap repair thread based on the second action block multiple times.

[0122] The Nth generation process includes:

[0123] Each action within the (M-1)th action block is dynamically weighted using an inference thread, and the beginning portion of the (M-1)th action block is frozen based on the weight of each action within the (M-1)th action block; where M is greater than or equal to 3 and N is greater than or equal to 1.

[0124] The inference thread calls the guide repair thread to repair the unfrozen part of the M-1 action block using the frozen M-1 action block and the M-1 action block.

[0125] The inference thread determines the required generation time of the M-1 action block based on the task objective, and judges whether the completion time of the repaired M-1 action block is equal to the required generation time of the M-1 action block, whether the actions in the repaired M-1 action block are connected and continuous, and whether the repaired M-1 action block conforms to the current environment state of the M-1 action block.

[0126] If the inference thread determines that the completion time of the repaired M-1 action block is not equal to the required generation time of the M-1 action block, and / or, the actions within the repaired M-1 action block are continuous, and / or, the repaired M-1 action block conforms to the current environment state of the M-1 action block, and determines that the repaired M-1 action block is unavailable, the M-2 action block is updated to the M-1 action block, and the thread returns to execute the step of dynamically weighting each action within the M-1 action block in order to refreeze the M-1 action block;

[0127] If the inference thread determines that the completion time of the repaired M-1 action block is equal to the required generation time of the M-1 action block, that the actions within the repaired M-1 action block are continuous, and that the repaired M-1 action block conforms to the current environment state of the M-1 action block, then the repaired M-1 action block is determined to be available. The repaired M-1 action block is then pushed into the action buffer and the control thread is notified that the action is ready.

[0128] The repaired (M-1)th action block is retrieved from the action buffer by the control thread and then executed.

[0129] During the execution of the repaired (M-1)th action block by the control thread, the inference thread checks in real time whether the execution length of the repaired (M-1)th action block is equal to the inference delay steps.

[0130] When the inference thread detects that the execution length of the repaired (M-1)th action block is equal to the inference delay steps, it can notify the control thread.

[0131] The control thread obtains the robot's Mth current observation and writes the Mth current observation into the observation buffer.

[0132] The inference thread retrieves the Mth current observation from the observation cache and generates the corresponding Mth action block based on the Mth current observation and the repaired M-1th action block.

[0133] Based on the action segmentation method provided in the above embodiments of this application, correspondingly, the embodiments of this application also provide an action segmentation system, such as... Figure 2 As shown, the system includes:

[0134] The first generation module 21 is used to obtain the first current observation and generate the first action block based on the first current observation;

[0135] The second generation module 22 is used to obtain the second current observation when the execution length of the first action block is equal to the inference delay step, and generate the second action block based on the second current observation and the first action block; wherein the end part of the first action block overlaps with the beginning part of the second action block;

[0136] The third generation module 23 is used to perform multiple generation processes based on the second action block to obtain multiple action blocks;

[0137] The third generation module of the Nth generation process includes:

[0138] The freeze unit is used to dynamically weight each action within the (M-1)th action block and freeze the beginning portion of the (M-1)th action block according to the weight of each action within the (M-1)th action block; where M is greater than or equal to 3 and N is greater than or equal to 1.

[0139] The repair unit is used to repair the unfrozen part of the M-1 action block using the frozen M-1 action block and the M-1 action block;

[0140] The first generation unit is used to execute the actions within the unfrozen portion of the repaired M-1 action block, and simultaneously generate the M-1 action block based on the repaired M-1 action block and the current observation of the M-1 action block; wherein the end portion of the M-1 action block overlaps with the beginning portion of the M-1 action block.

[0141] The specific principles and execution processes of each unit in the action block system disclosed in the above embodiments of this application are the same as those of the action block method disclosed in the above embodiments of this application. Please refer to the corresponding parts of the action block method disclosed in the above embodiments of this application, and they will not be repeated here.

[0142] This application provides an action block segmentation method, which involves obtaining a first current observation and generating a first action block based on the first current observation; if the execution length of the first action block is equal to the inference delay steps, obtaining a second current observation and generating a second action block based on the second current observation and the first action block; wherein the end portion of the first action block overlaps with the beginning portion of the second action block; and performing multiple generation processes based on the second action block to obtain multiple action blocks; wherein the Nth generation process includes: performing a process on each action within the (M-1)th action block... The method involves dynamically weighting the actions and freezing the beginning portion of the (M-1)th action block based on the weight of each action within the (M-1)th action block, where M is greater than or equal to 3 and N is greater than or equal to 1. The frozen (M-1)th action block and its corresponding action block are then used to repair the unfrozen portions within the (M-1)th action block. Actions within the unfrozen portions of the repaired (M-1)th action block are then executed, and the (M)th action block is generated based on the repaired (M-1)th action block and the current observations. The ending portion of the (M-1)th action block overlaps with the beginning portion of the (M)th action block. The technical solution provided in this application, during the iterative generation of corresponding action blocks, effectively improves the continuity and policy stability between consecutive action blocks by freezing the beginning portion of the (M-1)th action block and repairing the unfrozen portions using the frozen (M-1)th action block and its corresponding action block. This avoids problems such as pauses, decreased throughput, sudden acceleration changes, severe reaction delays, and wasted computational resources during real-time robot control.

[0143] Optionally, the first generation module includes:

[0144] The first acquisition unit is used to acquire 1 current observation when a task instruction is received, wherein the 1 current observation includes the 1 current environment state, the 1 historical action block and the task target;

[0145] The second generation unit is used to generate the first action block based on the first current environment state, the first historical action block and the task objective through the strategy model. The first action block includes multiple actions arranged in sequence.

[0146] Optionally, the second generation module includes:

[0147] The second acquisition unit is used to acquire the second current observation when the execution length of the first action block is equal to the inference delay steps. The second current observation includes the second current environment state, the second historical action block, and the task objective.

[0148] The selection unit is used to select multiple actions starting from the last action of the action block 1 based on the inference delay step number, so as to obtain the end part of the action block 1.

[0149] The third generation unit is used to generate the initial second action block based on the second current environment state, the second historical action block, and the task objective;

[0150] The fourth generation unit is used to splice the end part of the first action block as the beginning part of the second action block with the initial second action block to generate the second action block.

[0151] Optional, freeze unit, specifically used for:

[0152] For each action within the (M-1)th action block, determine whether the action length is less than the inference delay steps;

[0153] If the action length is less than the inference delay steps, assign the action a weight of 1.

[0154] If the action length is not less than the inference delay steps, determine whether the action length is greater than the first length. The first length is determined based on the action block length of the M-1 action block and the second length. The second length is the difference between the action block length of the M-1 action block and the length of its end part.

[0155] If the action length is not greater than the first length, determine the distance coefficient corresponding to the action, and determine the weight corresponding to the action based on the distance coefficient; wherein, the distance coefficient is determined based on the action block length of the M-1 action block, the second length, the inference delay steps, and the action length of the action;

[0156] If the action length is greater than the first length, the action weight is assigned a value of 0;

[0157] Freeze the actions with a weight of 1 within the M-1 action block to complete the freezing of the beginning part of the M-1 action block.

[0158] Optional, repair unit, specifically used for:

[0159] The direction of the M-1 velocity output by the diffusion model is determined by the current observation of the M-1th action block, the M-1th action block and its generation time.

[0160] Based on the M-1 velocity direction, the M-1 current observation, the M-1 action block and its generation time, determine the M-1 estimated action block before repair;

[0161] The diffusion model determines the guidance correction value for each action of the frozen part in the frozen part of the M-1 action block after freezing, based on the current observation of the M-1th action block, the M-1th action block and its generation time, the frozen part and its weight in the frozen M-1th action block, the M-1th velocity direction, and the M-1th estimated action block.

[0162] Repair the corresponding actions based on the guided correction values ​​of each action in the frozen part of the frozen action block M-1 after freezing.

[0163] Optionally, a repair unit is used to determine the guided correction value of each action in the frozen portion of the frozen portion within the frozen portion of the (M-1)th action block based on the diffusion model, the (M-1)th action block and its generation time, the frozen portion and its weight within the frozen (M-1)th action block, the (M-1)th velocity direction, and the (M-1)th estimated action block. Specifically, this is used for:

[0164] The gradient guidance term for each action in the frozen part of the frozen part within the frozen (M-1)th action block is constructed using the diffusion model based on the weight of each action in the frozen part.

[0165] The diffusion model determines the M-1 diffusion time weight term based on the generation time of the M-1 action block, and determines the M-1 guidance strength based on the M-1 diffusion time weight term, the generation time of the M-1 action block, and the preset guidance term strength upper limit.

[0166] The diffusion model generates the guidance correction value for each action of the frozen part in the frozen part of the (M-1)th action block based on the (M-1)th velocity direction, the (M-1)th guidance intensity, the frozen part in the (M-1)th action block after freezing, the (M-1)th estimated action block, the gradient guidance term of each action of the frozen part in the (M-1)th action block after freezing, and the (M-1)th action block.

[0167] This application also provides a storage medium storing program instructions, which, when loaded and executed by a processor, implement any of the above-described action block method embodiments.

[0168] This application also provides an electronic device, such as Figure 3As shown, the device includes a processor 301 and a memory 302, which are connected via a bus; the memory stores program instructions; the processor calls the program instructions in the memory to execute any of the above-described action block method embodiments.

[0169] The processor mentioned in this article can be the terminal's CPU, an integrated MCU within the terminal, or a combination of a CPU and an MCU. Furthermore, the processor contains a kernel that retrieves the corresponding program from memory; one or more kernels can be configured.

[0170] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0171] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0172] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0173] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0174] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for segmenting actions, characterized in that, Applicable to robots, the method includes: Obtain the first current observation, and generate the first action block based on the first current observation; If the execution length of the first action block is equal to the inference delay steps, a second current observation is obtained, and a second action block is generated based on the second current observation and the first action block; wherein the end portion of the first action block overlaps with the beginning portion of the second action block; The second action block is iteratively generated multiple times to obtain multiple action blocks. The Nth generation process includes: Each action within the (M-1)th action block is dynamically weighted, and the beginning portion of the (M-1)th action block is frozen according to the weight of each action within the (M-1)th action block; wherein, M is greater than or equal to 3, and N is greater than or equal to 1; The unfrozen portion within the (M-1)th action block is repaired using the frozen (M-1)th action block and the (M-1)th action block itself. Execute the actions within the unfrozen portion of the repaired M-1 action block, and simultaneously generate the M-1 action block based on the repaired M-1 action block and the current observation of the M-1 action block; wherein the end portion of the M-1 action block overlaps with the beginning portion of the M-1 action block.

2. The method according to claim 1, characterized in that, The step of obtaining the first current observation and generating the first action block based on the first current observation includes: When a task instruction is received, 1 current observation is obtained, wherein the 1st current observation includes the 1st current environment state, the 1st historical action block, and the task objective; The strategy model generates a first action block based on the first current environment state, the first historical action block, and the task objective. The first action block includes multiple actions arranged in sequence.

3. The method according to claim 1, characterized in that, The step of obtaining the second current observation when the execution length of the first action block is equal to the inference delay steps, and generating the second action block based on the second current observation and the first action block, includes: If the execution length of the first action block is equal to the inference delay steps, a second current observation is obtained, wherein the second current observation includes the second current environment state, the second historical action block, and the task objective; Based on the inference delay steps, multiple actions are selected starting from the last action of the 1 action block to obtain the end portion of the 1 action block; Based on the second current environment state, the second historical action block, and the task objective, generate an initial second action block; The end portion of the first action block is used as the beginning portion of the second action block and spliced ​​with the initial second action block to generate the second action block.

4. The method according to claim 1, characterized in that, The step of dynamically weighting each action within the (M-1)th action block and determining the frozen and unfrozen portions of the (M-1)th action block based on the weight of each action within the (M-1)th action block includes: For each action within the (M-1)th action block, determine whether the action length is less than the inference delay step number; If the action length is less than the inference delay step, the weight of the action is assigned a value of 1; If the action length is not less than the inference delay step, determine whether the action length is greater than the first length, wherein the first length is determined based on the action block length of the M-1 action block and the second length, and the second length is the difference between the action block length of the M-1 action block and the length of its end portion; If the action length is not greater than the first length, determine the distance coefficient corresponding to the action, and determine the weight corresponding to the action based on the distance coefficient; wherein, the distance coefficient is determined based on the action block length of the M-1 action block, the second length, the inference delay steps, and the action length of the action; If the action length is greater than the first length, the weight of the action is assigned to 0; Freeze the actions with a weight of 1 within the M-1 action block to complete the freezing of the beginning portion of the M-1 action block.

5. The method according to claim 4, characterized in that, The repair of the unfrozen portion within the (M-1)th action block using the frozen (M-1)th action block and the (M-1)th action block itself includes: The direction of the M-1 velocity output by the diffusion model is determined by the current observation of the M-1th action block, the M-1th action block and its generation time. Based on the M-1 velocity direction, the M-1 current observation, the M-1 action block and its generation time, determine the M-1 estimated action block before repair; The diffusion model determines the guidance correction value for each action of the frozen part within the frozen part of the (M-1)th action block based on the current observation of the (M-1)th action block, the (M-1)th action block and its generation time, the frozen part and its weight within the frozen (M-1)th action block, the (M-1)th velocity direction, and the (M-1)th estimated action block. Repair the corresponding actions based on the guided correction values ​​of each action in the frozen part of the frozen action block M-1 after freezing.

6. The method according to claim 5, characterized in that, The step of determining the guidance correction value for each action of the frozen portion within the frozen portion of the (M-1)th action block using the diffusion model based on the (M-1)th current observation, the (M-1)th action block and its generation time, the frozen portion within the frozen (M-1)th action block and its weights, the (M-1)th velocity direction, and the (M-1)th estimated action block includes: The gradient guidance term for each action in the frozen part of the frozen part within the frozen (M-1)th action block is constructed using the diffusion model based on the weight of each action in the frozen part after the freeze. The diffusion model determines the M-1 diffusion time weight term based on the generation time of the M-1 action block, and determines the M-1 guidance intensity based on the M-1 diffusion time weight term, the generation time of the M-1 action block, and the preset guidance term intensity upper limit. The diffusion model generates guidance correction values ​​for each action of the frozen portion within the frozen portion of the (M-1)th action block based on the (M-1)th velocity direction, the (M-1)th guidance intensity, the frozen portion within the frozen (M-1)th action block, the (M-1)th estimated action block, the gradient guidance term for each action of the frozen portion within the frozen (M-1)th action block, and the (M-1)th action block.

7. An action segmentation system, characterized in that, Suitable for robots, the system includes: The first generation module is used to obtain the first current observation and generate the first action block based on the first current observation; The second generation module is used to obtain a second current observation when the execution length of the first action block is equal to the inference delay steps, and generate a second action block based on the second current observation and the first action block; wherein the end portion of the first action block overlaps with the beginning portion of the second action block; The third generation module is used to iteratively generate multiple action blocks based on the second action block to obtain multiple action blocks. The third generation module of the Nth generation process includes: A freezing unit is used to dynamically weight each action within the (M-1)th action block and freeze the beginning portion of the (M-1)th action block according to the weight of each action within the (M-1)th action block; wherein, M is greater than or equal to 3 and N is greater than or equal to 1; A repair unit is used to repair the unfrozen portion of the (M-1)th action block using the frozen (M-1)th action block and the (M-1)th action block; The first generation unit is used to execute the actions within the unfrozen portion of the repaired M-1 action block, and simultaneously generate the M-1 action block based on the repaired M-1 action block and the current observation of the M-1 action block; wherein the end portion of the M-1 action block overlaps with the beginning portion of the M-1 action block.

8. The system according to claim 7, characterized in that, The first generation module includes: The first acquisition unit is used to acquire a first current observation when a task instruction is received, wherein the first current observation includes a first current environmental state, a first historical action block, and a task target; The second generation unit is used to generate a first action block based on the first current environment state, the first historical action block, and the task objective through a strategy model, wherein the first action block includes multiple actions arranged in sequence.

9. An electronic device, characterized in that, include: A processor and a memory are connected via a bus; wherein the processor is used to call and execute a program stored in the memory; The memory is used to store a program for implementing an action block method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for performing an action block method as described in any one of claims 1-6.

Images