Robot learning method and system based on action cutout and rolling time domain control

Abstract

The application relates to the technical field of robot control, and discloses a robot learning method and system based on action cutting and rolling time domain control. The method comprises the following steps: encoding multiple observation frames into an observation token sequence and encoding the tail of an unexecuted action in the last period into a planning draft token sequence; generating a head-level gating coefficient based on a visual frame difference; modulating a cross-attention score matrix between an observation hidden state and a planning draft by using the head-level gating coefficient, and outputting a rolling time domain action cutting sequence through a residual connection; extracting a first-step action execution and writing the remaining steps into a draft cache for the next period. The application improves the trajectory smoothness and disturbance recovery capability of the robot in continuous motion control in a dynamic industrial scene.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a robot learning method and system based on motion segmentation and rolling time-domain control. Background Technology

[0002] In the field of robot imitation learning, the mainstream paradigm for end-to-end visual motion policies adopts a combination of motion segmentation and rolling temporal control to generate robot motion commands. This paradigm was established by the Action Segmentation Transformer (ACT) proposed by Zhao et al. in 2023 and the Diffusion Policy proposed by Chi et al. in 2023. Its core idea is that the policy network predicts a sequence of K future actions in each control cycle, forming an action segment. However, it only executes the first step before re-acquiring observations and predicting again, thus maintaining the continuity of action sequence while preserving closed-loop response to environmental changes through a rolling temporal approach. This paradigm has demonstrated significantly better success rates and trajectory smoothness than traditional stepwise prediction methods in various robot tasks, such as fine dual-arm manipulation, industrial assembly, and weld seam tracking, becoming the technological foundation of current robot learning.

[0003] However, in the above paradigm, the policy reasoning process of each control cycle is completely independent of the prediction results of the previous cycle. The actions predicted in the 2nd to Kth steps of the previous cycle are discarded after only the 1st step is executed. The prediction of the new cycle is generated from scratch, and there is no information transfer relationship between the prediction of the previous cycle and the prediction of the previous cycle. This characteristic leads to a lack of continuity constraints between the prediction outputs of adjacent control cycles, and in actual deployments in industrial scenarios, inter-block trajectory jitter and action jump problems frequently occur. Existing technologies have made improvements to this problem, including the temporal integration method proposed by ACT, which performs a weighted average of overlapping predictions at the policy output end; the hot-start method proposed by Diffusion Policy, which uses the unexecuted predictions of the previous cycle as noise initialization for the denoising diffusion process; and the real-time block method proposed by Black et al. in 2025, which uses the prefix of the executed actions as hard constraints input to the policy network. However, all of the above solutions perform post-processing or pre-processing operations at the input or output end of the policy network, and do not establish a cross-cycle information transfer path at the attention calculation level inside the network. Moreover, none of them have the ability to adaptively adjust the reuse degree of old prediction information according to the degree of change in the visual scene. Summary of the Invention

[0004] This application provides a robot learning method and system based on motion segmentation and rolling time-domain control, which solves the problem of inter-block trajectory jitter and delayed response to environmental changes caused by the lack of selective information transmission at the network internal level in the prediction output of adjacent control cycles in existing motion segmentation and rolling time-domain control methods. This improves the trajectory smoothness and disturbance recovery capability of robots in continuous motion control in dynamic industrial scenarios.

[0005] In a first aspect, this application provides a robot learning method based on action segmentation and scrolling time-domain control, the robot learning method based on action segmentation and scrolling time-domain control comprising: Step S1: Encode multiple consecutive frames of images and their corresponding robot states into observation token sequences, and project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence. Step S2: Calculate the difference between the visual features of the current frame and the visual features of the previous frame to obtain the observation difference vector, and map the observation difference vector to a number of head-level gating coefficients through linear projection and activation function; Step S3: The observation token sequence is self-attention encoded to obtain the observation hidden state and used as the query matrix. The planning draft token sequence is mapped to the key matrix and the value matrix respectively. The attention score matrix is ​​calculated by the query matrix and the key matrix. The attention score matrix is ​​scaled head-by-head with the head-level gating coefficient to obtain the gated attention score matrix. The gated attention score matrix and the value matrix are weighted and summed and connected by residuals to obtain the revised fusion feature. The revised fusion feature is output as a rolling temporal action block sequence through the action prediction head. Step S4: In each rolling time-domain control cycle, the first action is extracted from the rolling time-domain action block sequence, executed after security arbitration, and the remaining steps in the rolling time-domain action block sequence are written into the draft cache for the next control cycle to generate the planning draft token sequence.

[0006] Optionally, step S1 includes: H consecutive frames of images are input frame by frame into a pre-trained visual encoder for global pooling feature extraction, resulting in a visual feature sequence composed of H visual feature vectors. The joint angle, end pose, joint angular velocity and gripper state corresponding to each frame are concatenated into a robot state vector. The robot state vector is then input into a two-layer state encoder for nonlinear mapping to obtain a state feature sequence aligned with the frame number of the visual feature sequence. The visual feature sequence and the state feature sequence are concatenated along the feature dimension and then transformed by a linear projection layer to obtain the observation token sequence. The unexecuted action tail, consisting of steps 2 to 3 from the action block output in the previous control cycle, is read from the draft cache. The unexecuted action tail is then input into the draft projection layer for dimension mapping to obtain the planning draft token sequence. In the first control cycle, when the draft cache is empty, each element of the planning draft token sequence is set to a zero vector.

[0007] Optionally, step S2 includes: The visual feature vectors of the current frame and the previous frame are extracted from the visual feature sequence, and the visual feature vectors of the current frame and the previous frame are subtracted element by element to obtain the observation difference vector. The observed difference vector is input into the first linear layer of the gated projection module for dimensionality reduction mapping. The intermediate features after dimensionality reduction mapping are then processed by a normalization layer and a nonlinear activation function to obtain the gated intermediate features. The gated intermediate features are input into the second linear layer of the gated projection module for projection, resulting in a gated original score vector with a dimension equal to the number of cross attention heads M. The Sigmoid activation function is applied element by element to the original gating score vector to obtain M first-level gating coefficients with values ​​ranging from zero to one. The bias parameter of the second linear layer is initialized to a negative value so that each first-level gating coefficient is less than 0.5 when the observation difference vector is zero.

[0008] Optionally, in step S3, the observed token sequence is self-attention encoded to obtain the observed hidden state and used as the query matrix. The planning draft token sequence is mapped into a key matrix and a value matrix, respectively, including: The observation token sequence is input into a multilayer self-attention encoder, and the observation hidden state is obtained after layer-by-layer self-attention calculation and feedforward network processing. Based on the M query projection matrices, the observed hidden state is linearly transformed to obtain M sets of query sub-matrices; Based on the M key projection matrices, the planning draft token sequence is linearly transformed to obtain M sets of key sub-matrices; Based on the M value projection matrices, the planning draft token sequence is linearly transformed to obtain M sets of value sub-matrices.

[0009] Optionally, in step S3, the attention score matrix is ​​scaled head-by-head using the head-level gating coefficients to obtain a gated attention score matrix. The revised fusion feature is obtained by weighted summation of the gated attention score matrix and the value matrix, followed by residual concatenation, including: Perform the transpose dot product operation on the i-th query submatrix and the i-th key submatrix, divide by the square root of the feature dimension, and process by the normalized exponential function to obtain the original attention numerator matrix of the i-th group. Iterate through M groups and perform the above operation to obtain the attention score matrix. The difference between 1 and the i-th coefficient in the first-level gating coefficient is calculated as the pass rate scalar of the i-th group. The i-th group pass rate scalar is used to perform element-wise multiplication on the molecular matrix of the original attention of the i-th group. The above operation is performed on M groups respectively to obtain the gating attention score matrix. The i-th group of gated attention is obtained by performing matrix multiplication with the i-th group of value submatrix to obtain the i-th group of cross attention output. The M groups of cross attention outputs are concatenated on the feature dimension and projected through linear output to obtain the cross attention fusion output. The cross-attention fusion output is added element-wise to the observed hidden state to complete the residual connection, and the revised fusion feature is obtained after processing by the normalization layer.

[0010] Optionally, step S3 involves outputting a rolling temporal action block sequence from the revised fusion features via an action prediction head, including: The revised fusion features are input into the subsequent self-attention coding layer to continue self-attention calculation and feedforward network processing to obtain the deep fusion hidden state; Take the hidden state vector corresponding to the last time step from the deep fusion hidden state as the decision anchor point feature; The decision anchor features are regressed and mapped through an action prediction head composed of two layers of linear transformation and a nonlinear activation function to obtain a flat output vector with dimension K multiplied by the dimension of the end incremental action. The flat output vector is reshaped in K steps to obtain a rolling temporal motion block sequence containing K-step end-increment actions, where each end-increment action contains a 3D position increment and a 3D attitude increment.

[0011] Optionally, step S4 includes: Take the first step end increment action from the rolling time domain action block sequence, and then pass it through the absolute limiting layer to trim each component of the position increment to a preset displacement threshold range and each component of the attitude increment to a preset angle threshold range to obtain the limited action. The target end pose is obtained by summing the current end pose with the limited action. The target end pose is compared with the preset three-dimensional safety envelope boundary coordinate by coordinate. Components that exceed the three-dimensional safety envelope boundary are set to zero. The target end pose is then verified by solving the target joint angle through inverse kinematics and comparing it with the joint limit and the Jacobian matrix condition number threshold. Incremental commands that fail the verification are set to zero as a whole to obtain the safe incremental action. The safety incremental action and the incremental command actually executed in the previous control cycle are filtered by a first-order exponential moving average with a preset filtering coefficient to obtain a smoothed incremental command, which is then sent to the robot controller for execution. The remaining action sequence consisting of steps 2 to K in the rolling time-domain action block sequence is written into the draft cache, overwriting the old draft of the previous control cycle, so that the planning draft token sequence can be generated from the draft cache in step S1 of the next control cycle.

[0012] Secondly, this application provides a robot learning system based on action segmentation and scrolling time-domain control, the robot learning system based on action segmentation and scrolling time-domain control includes: The encoding module is used to encode multiple consecutive frames of images and their corresponding robot states into an observation token sequence, and to project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence. The mapping module is used to obtain the observation difference vector by subtracting the visual features of the current frame from the visual features of the previous frame, and to map the observation difference vector into several head-level gating coefficients by linear projection and activation function; The prediction module is used to encode the observed token sequence through self-attention to obtain the observed hidden state and use it as a query matrix. The planning draft token sequence is mapped to a key matrix and a value matrix respectively. An attention score matrix is ​​calculated from the query matrix and the key matrix. The attention score matrix is ​​scaled head-by-head with the head-level gating coefficient to obtain a gated attention score matrix. The gated attention score matrix and the value matrix are weighted and summed and connected through residuals to obtain a revised fusion feature. The revised fusion feature is output as a rolling temporal action block sequence through the action prediction head. The generation module is used to extract the first action from the rolling time-domain action block sequence in each rolling time-domain control cycle, issue it for execution after security arbitration, and write the remaining steps in the rolling time-domain action block sequence into the draft cache for the generation of the planning draft token sequence in the next control cycle.

[0013] The technical solution provided in this application constructs a dual-stream input structure of observation token sequence and planning draft token sequence at the input end of the policy network. This allows the reasoning process of each control cycle to no longer rely solely on the observation information at the current moment to independently generate action blocks from zero. Instead, it simultaneously receives the planning intent information encoded by the tail of the action predicted but not executed in the previous control cycle. Before the policy network begins attention calculation, it provides a priori reference on how the model in the previous cycle believes the robot should move next. This dual-stream input structure breaks the limitation of the complete isolation of prediction information between adjacent control cycles in the existing action block method from the information source, enabling the policy network to have a physical path to reference the planning intent of the previous cycle within the network. Based on this, this application maps the observation difference vector obtained by subtracting the visual features of the current frame from the visual features of the previous frame to several head-level gating coefficients through linear projection and activation function. This design injects the judgment information of whether the environment has changed into the subsequent attention calculation process in the form of continuous values, rather than relying on manually set fixed thresholds for binary judgment. The generation of gating coefficients depends only on the visual feature difference between the current frame and the previous frame and does not depend on the prediction results of the current cycle that have not yet been completed. In terms of causality, it strictly precedes the policy reasoning process, avoiding the logical circular dependency that requires the generation of new predictions before it can determine whether to reuse old predictions. At the same time, all parameters of the gating projection module are jointly trained end-to-end with the policy network as a whole. The transition boundary of the gating behavior is automatically learned by training data rather than manually debugged by engineers for each specific task.

[0014] This application embeds a cross-attention computation layer modulated by head-level gating coefficients between the self-attention encoding layers within the policy network. The observed hidden state is used as the query matrix, and the key matrix and value matrix of the planning draft token sequence mapping are used as the search objects. Through the cross-attention mechanism, the current observation information actively retrieves the time steps and feature dimensions related to the current motion stage from the old planning draft. The head-level gating coefficients are applied to the normalized exponential function output of the attention score matrix in a head-by-head scalar scaling manner. When the visual scene is stable, the pass rate scalar of each head approaches a high value, allowing the cross-attention to fully utilize the continuous information in the old plan, thereby suppressing random fluctuations between adjacent period predictions. When the visual scene changes abruptly, the pass rate scalar of each head approaches zero, causing the cross-attention output to approach an all-zero matrix. After residual connection, it automatically degenerates into the standard single-stream policy behavior that only relies on the current observation, thus avoiding being misled by outdated old plans. This mechanism solves two seemingly contradictory needs within the same network structure: revising the old plan to maintain smoothness when the scene is stable and abandoning the old plan to maintain responsiveness when the scene changes abruptly, without having to make a hard switch between the two modes. The rolling temporal action block sequence output by the action prediction head extracts only the first step of the action in each control cycle, which is then executed after safety arbitration. The remaining steps are written into the draft cache as the source for generating the planning draft token sequence in the next control cycle. This forms a closed information loop of prediction, caching, back-injection, gating revision, and re-prediction between adjacent control cycles. This ensures that the action output of each cycle in the rolling temporal control process inherits the temporal continuity of the previous cycle's planning while retaining the ability to respond instantly to real-time changes in the environment. In industrial multi-station robot control scenarios, this eliminates the technical contradiction of inter-block trajectory jitter and delayed response to sudden environmental changes from the network structure level rather than the post-processing level. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of an embodiment of the robot learning method based on action segmentation and rolling temporal control in this application. Figure 2 This is a schematic diagram of the thermal distribution of the first-stage gating coefficient as a function of the control cycle in an embodiment of this application. Figure 3 This is a schematic diagram of the loss value convergence curve during the training process of the policy network in this embodiment of the application. Detailed Implementation

[0017] This application provides a robot learning method and system based on action segmentation and rolling time-domain control. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the robot learning method based on action segmentation and rolling temporal control in this application includes: Step S1: Encode multiple consecutive frames of images and their corresponding robot states into observation token sequences, and project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence. Specifically, the observation token sequence is a two-dimensional floating-point tensor with shape H rows and 384 columns, where H represents the number of historical frames (set to 8 in this scheme), and 384 represents the feature dimension of the token, equal to the dimension of the global pooling output of the DINOv2-small visual encoder. The reason for choosing H equal to 8 rather than a larger value is that when an industrial robotic arm performs a single-station task at a control frequency of 10 to 15 Hz, 8 historical observation frames cover a time window of approximately 0.5 to 0.8 seconds. This window is sufficient to capture the motion trend of the end effector within the current task phase. Too many frames would introduce early frame information unrelated to the current motion direction and increase the computational load of inference. The planning draft token sequence is a two-dimensional floating-point tensor with K-1 rows and 384 columns, where K is the action block length and is set to 12. K-1 equals 11, representing the tokens mapped from the 11 unexecuted actions (steps 2 to 12) predicted in the previous cycle after being mapped by the draft projection layer. The draft projection layer is a two-layer perceptron structure with a hidden layer dimension of 192 using the GELU activation function. The output dimension of 384 is consistent with the feature dimension of the observed tokens. This dimensional alignment is a prerequisite for cross-attention computation. When there are no historical predictions available for reading in the first cycle after the control system starts, all elements of the 11-row, 384-column planning draft token sequence are filled with zero values. At this time, the key matrix and value matrix of the cross-attention layer are both zero, and the cross-attention output is also zero. The behavior of the entire network is equivalent to a standard single-stream policy network without the cross-attention layer. The control system only truly enters the revision inference mode from the second cycle onwards.

[0019] Step S2: Calculate the difference between the visual features of the current frame and the visual features of the previous frame to obtain the observation difference vector, and map the observation difference vector to a number of head-level gating coefficients through linear projection and activation function; Specifically, the observation difference vector is a 384-dimensional vector obtained by subtracting the visual feature vector of the previous frame from the current frame element-wise. Each element of this vector reflects the magnitude of change in the corresponding dimension in the DINOv2 feature space between adjacent frames. The head-level gating coefficient is a set of M scalar values, where M equals the number of attention heads in the cross-attention mechanism and is set to 8 in this scheme. Each coefficient ranges from 0 to 1 in an open interval. The bias parameter of the second linear layer of the gating projection module is initialized to a specific value of -1.0. Its function is that when the observation difference vector is zero (i.e., the visual scene remains completely unchanged between adjacent frames), the output of the second linear layer is equal to the bias value of -1.0. After mapping with the Sigmoid function, the gating coefficient is approximately 0.27, and the corresponding pass rate scalar is 1 minus 0.27, which equals 0.73. This means that under the default state where the scene remains unchanged, approximately 73% of the old planning draft information enters the current inference process through the cross-attention layer. The reason for choosing -1.0 instead of other negative values ​​is that the output 0.27 of the Sigmoid function when the input is -1.0 is in the middle of the biased trust region, which neither over-trusts nor over-suppresses the old draft. If the bias is set to -3.0, the default pass rate is as high as 95%, which makes the model react slowly when the environment changes. If the bias is set to 0, the default pass rate is only 50%, which makes the revision effect insufficient. -1.0 is the value determined after balancing the smoothness of revision and the sensitivity of environmental response in multiple rounds of experiments.

[0020] Step S3: The observation token sequence is self-attention encoded to obtain the observation hidden state and used as the query matrix. The planning draft token sequence is mapped to the key matrix and the value matrix respectively. The attention score matrix is ​​calculated by the query matrix and the key matrix. The attention score matrix is ​​scaled head-by-head with the head-level gating coefficient to obtain the gated attention score matrix. The gated attention score matrix and the value matrix are weighted and summed and connected by residuals to obtain the revised fusion feature. The revised fusion feature is output as a rolling temporal action block sequence through the action prediction head. Specifically, the key difference between the generation process of the gated attention score matrix and the standard multi-head cross-attention is that the standard cross-attention directly uses the attention score matrix to perform a weighted summation of the value matrix after Softmax normalization, while this scheme inserts a head-by-head scalar multiplication operation after Softmax normalization and before the weighted summation. The revised fusion feature is not the direct output of the cross-attention layer, but rather the result of the cross-attention fusion output and the observed hidden state being concatenated via residuals and processed by a normalization layer. The specific operation of the residual concatenation is to directly add each element of the cross-attention fusion output to the corresponding element of the observed hidden state. This residual structure ensures that even if the cross-attention fusion output approaches zero due to the gating coefficients approaching 1, the revised fusion feature still retains complete observed hidden state information, and the policy network will not lose its normal action prediction ability simply because the old draft is blocked. The rolling temporal motion block sequence is a two-dimensional floating-point tensor with a shape of 12 rows and 6 columns. The 12 rows correspond to the future motion prediction for K equal to 12 time steps, and the 6 columns correspond to the 6 components of the end-effector incremental motion for each step. The first 3 components are the position increment in the end-effector coordinate system in millimeters, and the last 3 components are the attitude increment in the end-effector coordinate system in radians.

[0021] Step S4: In each rolling time-domain control cycle, the first action is extracted from the rolling time-domain action block sequence, executed after security arbitration, and the remaining steps in the rolling time-domain action block sequence are written into the draft cache for the next control cycle to generate the planning draft token sequence.

[0022] Specifically, the safety arbitration is a four-layer serial processing pipeline. Each layer independently verifies and corrects the output of the layer above, and only after all layers pass the verification is the incremental command sent to the robot controller. The absolute limiting layer trims each of the six components. The trimming range for each component of the position increment is from -1.0 mm to +1.0 mm, and the trimming range for each component of the attitude increment is from -0.0175 radians to +0.0175 radians (±1 degree). These two thresholds are set based on the maximum allowable motion of an industrial six-axis robot arm in a single cycle at a control frequency of 10 to 15 Hz. Incremental commands exceeding these ranges indicate abnormal jumps in the model output. The workspace blocking layer adds the current end-effector pose to the limited increment to obtain the expected target pose. Each coordinate of this target pose is compared with the six boundary values ​​of the pre-calibrated three-dimensional cuboid safety envelope. If any coordinate exceeds the boundary, the corresponding incremental component is forcibly set to zero. The singularity interception layer converts the target end-effector pose into six joint angle values ​​through inverse kinematics. It checks each joint angle to ensure the distance from its mechanical limit is less than 5 degrees and calculates the condition number of the Jacobian matrix under the current configuration. If any condition is triggered, all six incremental components for the current cycle are set to zero, and the robotic arm remains stationary in its current pose. The smoothing filter layer uses a first-order exponential moving average with a filter coefficient of 0.7. This means the final incremental command issued in the current cycle equals 0.7 multiplied by the incremental command validated by the first three layers, plus 0.3 multiplied by the actual incremental command issued in the previous cycle. This filter eliminates residual high-frequency jitter components in the policy network output. The draft cache is a fixed-size memory area. At the end of each control cycle, it is overwritten by the 11 action vectors predicted from steps 2 to 12 of the current cycle. During the observation token sequence generation phase of the next control cycle, it is read and encoded by the draft projection layer into a new planning draft token sequence. This write-read cycle constitutes the physical path for the transmission of prediction information between adjacent control cycles.

[0023] In one specific embodiment, step S1 includes: H consecutive frames of images are input frame by frame into a pre-trained visual encoder for global pooling feature extraction, resulting in a visual feature sequence composed of H visual feature vectors. The joint angle, end pose, joint angular velocity and gripper state corresponding to each frame are concatenated into a robot state vector. The robot state vector is then input into a two-layer state encoder for nonlinear mapping to obtain a state feature sequence aligned with the frame number of the visual feature sequence. The visual feature sequence and the state feature sequence are concatenated along the feature dimension and then transformed by a linear projection layer to obtain the observation token sequence. The unexecuted action tail, consisting of steps 2 to 3 from the action block output in the previous control cycle, is read from the draft cache. The unexecuted action tail is then input into the draft projection layer for dimension mapping to obtain the planning draft token sequence. In the first control cycle, when the draft cache is empty, each element of the planning draft token sequence is set to a zero vector.

[0024] Specifically, the process of acquiring the visual feature sequence is as follows: An industrial camera continuously acquires RGB images at a frequency of 30 frames per second. Each frame has an original resolution of 1280 x 1024 pixels, and is immediately cropped and scaled to a fixed size of 224 x 224 pixels after acquisition. The scaled images are then fed frame by frame into the DINOv2-small model, a self-supervised pre-trained visual Transformer released by Meta in 2023. This approach uses its small-scale variant, with global pooling output as a 384-dimensional floating-point vector. During training and inference, all parameters of DINOv2-small are frozen and do not participate in gradient updates; they are only used as a fixed feature extraction tool. This freezing strategy is based on the fact that the amount of teaching data in industrial scenarios is usually only 300 to 800 episodes, which is far from sufficient for stable fine-tuning of a pre-trained visual Transformer. Unfreezing the parameters would destroy the general visual features learned on large-scale data. Eight consecutive frames of images were processed by DINOv2-small to obtain eight 384-dimensional vectors, which were arranged in chronological order to form a visual feature sequence with a shape of 8 rows and 384 columns.

[0025] Simultaneously, at the same moment each frame of image is acquired, corresponding state data is retrieved from the robot controller, including 6 joint angles (in radians), 6 joint angular velocities (in radians per second), the 3D position (in millimeters) and 3D pose (represented by Euler angles, in radians) of the end-effector center point, and the 1D gripper opening degree, totaling 19 scalars, which are then stitched together into a robot state vector. Since the robot controller's state reporting frequency is typically 100 Hz while the image frame rate is 30 Hz, the state data needs to be downsampled and aligned to nearest neighbors according to the image timestamp before stitching. The robot state vectors of 8 frames are input frame by frame into a state encoder composed of a two-layer sensing mechanism. The first layer of this encoder maps the 19-dimensional input to a 128-dimensional hidden layer and activates it using ReLU. The second layer maps the 128-dimensional hidden layer to a 128-dimensional output. After processing 8 frames, a state feature sequence with a shape of 8 rows and 128 columns is obtained. Each row (384 dimensions) of the visual feature sequence is concatenated with the corresponding row (128 dimensions) of the state feature sequence along the feature dimension to obtain a concatenated feature with a shape of 8 rows and 512 columns. Then, a linear projection layer with 512-dimensional input and 384-dimensional output is used for dimensional transformation to produce an observation token sequence with a shape of 8 rows and 384 columns. The 384 dimension is strictly consistent with the feature dimension of the subsequent cross-attention layer and the planning draft token sequence.

[0026] The generation of the planning draft token sequence depends on the data stored in the draft buffer. The draft buffer is a fixed-size memory space that holds 11 6-dimensional floating-point vectors, corresponding to the 11 unexecuted end-point incremental actions from step 2 to step 12 of the 12-step action block output by the policy network in the previous control cycle. Each 6-dimensional vector contains a 3-dimensional position increment (in millimeters) and a 3-dimensional pose increment (in radians), with an overall shape of 11 rows and 6 columns.

[0027] After reading these 11 vectors, they are input into the draft projection layer for dimension mapping. The draft projection layer is a two-layer perceptron independent of the state encoder. The first layer maps the 6-dimensional input to a 192-dimensional hidden layer and processes it through the GELU activation function. The second layer maps the 192-dimensional input to a 384-dimensional output. GELU was chosen instead of ReLU because GELU has smooth nonlinear characteristics near zero, and can retain more delicate gradient information for small incremental movements close to zero (typical output of the low-speed precision movement phase of a robotic arm). After processing by the draft projection layer, a planning draft token sequence with a shape of 11 rows and 384 columns is obtained, where the 384-dimensional feature dimension is consistent with the feature dimension of the observed token sequence. When the control system executes the first control cycle after power-on, the draft buffer has not yet been written with any prediction results. At this time, the 11-row, 384-column planning draft token sequence is filled with zero values. The cross-attention layer cannot extract any valid old planning information from the all-zero key and value matrices. The policy network behaves exactly the same as the standard Transformer policy without the cross-attention layer in the first cycle. Starting from the second control cycle, the draft buffer contains the real prediction data of the previous cycle, and the revised inference officially takes effect.

[0028] In one specific embodiment, step S2 includes: The visual feature vectors of the current frame and the previous frame are extracted from the visual feature sequence, and the visual feature vectors of the current frame and the previous frame are subtracted element by element to obtain the observation difference vector. The observed difference vector is input into the first linear layer of the gated projection module for dimensionality reduction mapping. The intermediate features after dimensionality reduction mapping are then processed by a normalization layer and a nonlinear activation function to obtain the gated intermediate features. The gated intermediate features are input into the second linear layer of the gated projection module for projection, resulting in a gated original score vector with a dimension equal to the number of cross attention heads M. The Sigmoid activation function is applied element by element to the original gating score vector to obtain M first-level gating coefficients with values ​​ranging from zero to one. The bias parameter of the second linear layer is initialized to a negative value so that each first-level gating coefficient is less than 0.5 when the observation difference vector is zero.

[0029] Specifically, the observation difference vector is calculated on the two frames with the latest time index in the visual feature sequence, namely the H-th frame (current frame) and the H-minus-1 frame (previous frame), both of which are 384-dimensional floating-point vectors. The result of element-wise subtraction is still a 384-dimensional vector. When the workpiece is not displaced and the lighting conditions are stable in an industrial scene, the features extracted by DINOv2-small from adjacent frames are highly similar, and the absolute values ​​of each element of the difference vector are extremely small. Conversely, when the workpiece is pushed by an external force or an obstruction enters the field of view, the difference vector will show significant non-zero components in the feature dimensions related to this visual change. The gated projection module consists of two linear layers and normalization and activation processing between them. The first linear layer maps the 384-dimensional input to a 64-dimensional intermediate space. This dimensionality reduction ratio is approximately 6 times. The 64-dimensional dimension is chosen because the gated signal itself only needs to reflect the coarse-grained level of scene changes without retaining all the details of the visual features. The 64-dimensional intermediate features after dimensionality reduction are normalized by LayerNorm to eliminate the numerical scale differences between different dimensions, and then nonlinearity is introduced by the ReLU activation function.

[0030] After normalization and activation processing, the 64-dimensional gated intermediate features are input to the second linear layer. This layer maps the 64-dimensional input to an M-dimensional output, where M equals 8, representing the number of heads in the cross-attention mechanism, resulting in an 8-dimensional gated raw score vector. A sigmoid activation function is applied element-wise to this vector, compressing any real value into an open interval between 0 and 1, yielding 8 head-level gating coefficients. The bias parameter of the second linear layer is set to -1.0 during network initialization instead of PyTorch's default random initialization. The direct effect of this setting is that when the observation difference vector is exactly all zero (i.e., the scene is completely still), the weight matrices of the first and second linear layers multiplied by the zero vector result in zero. The output of the second linear layer only has a bias term of -1.0, and the sigmoid (-1.0) is approximately equal to 0.27, corresponding to a pass rate scalar of 0.73. Setting the bias to 0 results in a Sigmoid output of 0.5, meaning that even in static scenes, half of the old planning information is suppressed, significantly reducing the revision effect. Setting it to -3.0 results in a Sigmoid output of approximately 0.05, with a pass rate as high as 0.95, indicating that the model's gating response is too sluggish when the environment actually changes. A bias of -1.0 strikes a balance between these two extremes, allowing the default state to favor the old draft while retaining sufficient adjustment margin. The eight head-level gating coefficients act independently on the eight attention heads involved in subsequent cross-attention. Different heads focus on different aspects of information in the old planning draft (such as position-related dimensions and pose-related dimensions). The gating coefficients of each head are updated independently during training through backpropagation, allowing the degree of trust in the old plan by different attention heads to vary rather than being forced to unify under the same scene changes.

[0031] Figure 2This is a schematic diagram of the thermal distribution of the head-level gating coefficient as a function of the control cycle in an embodiment of this application. The vertical axis represents the index number of the eight cross-attention heads, and the horizontal axis represents the time progression direction of 30 consecutive control cycles. The grayscale of each color block represents the gating coefficient value of the corresponding attention head in that control cycle. The darker the color, the closer the gating coefficient is to 1, i.e., the stronger the suppression of old planning draft information; the lighter the color, the closer the gating coefficient is to 0, i.e., the higher the degree of passage of old planning draft information. During control cycles 1 to 11 and 21 to 30, the visual scene remained stable with no workpiece displacement. The gating coefficients of all eight attention heads remained in the low range of 0.15 to 0.35. At this time, the cross-attention layer fully utilized the planning draft from the previous cycle for revised inference. During control cycles 12 to 20, a disturbance event was simulated in which the workpiece was pushed by an external force. The gating coefficients of all attention heads rapidly climbed to the high range of 0.7 to 0.95. The cross-attention layer automatically suppressed outdated planning information, and the policy network reverted to an independent prediction mode that relied solely on the current observation. After the disturbance ended, the gating coefficients recovered to the low range within 2 to 3 control cycles.

[0032] In one specific embodiment, step S3 involves encoding the observation token sequence using self-attention to obtain the observed hidden state, which is then used as the query matrix. The planning draft token sequence is then mapped to a key matrix and a value matrix, respectively, including: The observation token sequence is input into a multilayer self-attention encoder, and the observation hidden state is obtained after layer-by-layer self-attention calculation and feedforward network processing. Based on the M query projection matrices, the observed hidden state is linearly transformed to obtain M sets of query sub-matrices; Based on the M key projection matrices, the planning draft token sequence is linearly transformed to obtain M sets of key sub-matrices; Based on the M value projection matrices, the planning draft token sequence is linearly transformed to obtain M sets of value sub-matrices.

[0033] Specifically, the observation token sequence has a shape of 8 rows and 384 columns. The input self-attention encoder contains 4 layers, with a cross-attention layer inserted between the 2nd and 3rd layers. Therefore, in this step, the observation token sequence only undergoes the first two layers of self-attention encoding. The computation process within each layer is as follows: the input tensor first passes through an 8-head self-attention sub-layer (the query, key, and value all originate from the input tensor itself). The self-attention output and input are added by residuals and then normalized by LayerNorm. Next, it passes through a feedforward network sub-layer (the feedforward network consists of two linear transformations sandwiching a GELU activation function, with an internal expansion dimension of 384 multiplied by 4, equaling 1536). The feedforward output is also normalized by adding residuals and LayerNorm. After two consecutive layers of this processing, the shape of the output tensor remains unchanged at 8 rows and 384 columns; this output is the observation hidden state. The 384-dimensional feature space has fused the interdependencies of visual semantic information from 8 frames of images and robot body state information in the temporal dimension. Each row corresponds to a comprehensive hidden layer representation of a time step.

[0034] Before entering the cross-attention computation, the observed hidden state needs to be split into a multi-head structure. The splitting method is as follows: the 384-dimensional feature space is evenly divided into M equal to 8 attention heads, each head receiving a subspace of 384 divided by 8 equal to 48 dimensions. Each head has an independent query projection matrix with a shape of 384 rows and 48 columns. Multiplying the observed hidden state (8 rows and 384 columns) on the right by this matrix yields a set of query submatrices with a shape of 8 rows and 48 columns. This operation is performed once by each of the 8 heads, generating a total of 8 sets of query submatrices. The planning draft token sequence has a shape of 11 rows and 384 columns and also needs to be split into a multi-head structure through key projection and value projection. The key projection matrix of each head has a shape of 384 rows and 48 columns. Multiplying the planning draft token sequence on the right by this matrix yields a set of key submatrices with a shape of 11 rows and 48 columns; the value projection matrix also has a shape of 384 rows and 48 columns, and multiplying it on the right yields a set of value submatrices with a shape of 11 rows and 48 columns. The query submatrix, key submatrix, and value submatrix all have 48 columns in each head. This dimensional consistency is a mathematical prerequisite for calculating the dot product attention score. It's important to note that the query matrix originates from the observed hidden state, while the key and value matrices originate from the planning draft token sequence. This structure, where the query and key-value pairs have different origins, is the essential characteristic that distinguishes cross-attention from self-attention. In self-attention, the query, key, and value all originate from the same input, while the design intent of cross-attention here is to allow the current observation information to actively retrieve content relevant to the current time step from the old planning draft.

[0035] In one specific embodiment, step S3 involves performing head-by-head scaling on the attention score matrix using the head-level gating coefficients to obtain a gated attention score matrix. The revised fusion feature is obtained by weighted summation of the gated attention score matrix and the value matrix, followed by residual concatenation. This includes: Perform the transpose dot product operation on the i-th query submatrix and the i-th key submatrix, divide by the square root of the feature dimension, and process by the normalized exponential function to obtain the original attention numerator matrix of the i-th group. Iterate through M groups and perform the above operation to obtain the attention score matrix. The difference between 1 and the i-th coefficient in the first-level gating coefficient is calculated as the pass rate scalar of the i-th group. The i-th group pass rate scalar is used to perform element-wise multiplication on the molecular matrix of the original attention of the i-th group. The above operation is performed on M groups respectively to obtain the gating attention score matrix. The i-th group of gated attention is obtained by performing matrix multiplication with the i-th group of value submatrix to obtain the i-th group of cross attention output. The M groups of cross attention outputs are concatenated on the feature dimension and projected through linear output to obtain the cross attention fusion output. The cross-attention fusion output is added element-wise to the observed hidden state to complete the residual connection, and the revised fusion feature is obtained after processing by the normalization layer.

[0036] Specifically, taking the i-th group as an example, the specific calculation process of the attention score is explained. The query submatrix of the i-th group has a shape of 8 rows and 48 columns, and the key submatrix of the i-th group has a shape of 11 rows and 48 columns. The key submatrix is ​​transposed to obtain a matrix of 48 rows and 11 columns, and then matrix multiplication is performed with the query submatrix, that is, 8 rows and 48 columns multiplied by 48 rows and 11 columns, to obtain an original dot product matrix of 8 rows and 11 columns. The element in the r-th row and c-th column of this matrix represents the correlation between the hidden state at the r-th time step in the current observation and the c-th unexecuted action in the old planning draft. The numerical scale of this original dot product matrix is ​​proportional to the feature dimension of 48. Direct use will cause the Softmax function to enter the saturation region of minimal gradient, so it needs to be scaled by dividing by the square root of 48, which is about 6.93. The scaled 8-row and 11-column matrix is ​​then subjected to the Softmax normalization exponential function along each row (i.e., along the 11-column direction), so that the sum of the 11 elements in each row is equal to 1, forming the original attention numerator matrix in the form of a probability distribution. Perform the above operation on all 8 heads to obtain 8 sets of original attention molecular matrices, each with a shape of 8 rows and 11 columns.

[0037] The gating scaling operation occurs after Softmax normalization, and this order has a clear technical consideration. The i-th head-level gating coefficient g_i obtained from the previous step is a scalar between 0 and 1, and the pass rate scalar is equal to 1 minus g_i. This pass rate scalar is multiplied one by one by each element of the numerator matrix (8 rows and 11 columns) of the original attention set to obtain the numerator matrix of the gated attention set to the i-th set. The shape is still 8 rows and 11 columns, but the value of each element is reduced proportionally. When g_i equals 0, the pass rate scalar equals 1, the numerator matrix of the gated attention set to the same as the original matrix, and the information of the old planning draft is not suppressed in any way; when g_i equals 1, the pass rate scalar equals 0, the entire numerator matrix of the gated attention set to zero, and the information of the old planning draft is completely blocked. The reason gating scaling must be performed after Softmax, rather than before, is that if scaling is placed before Softmax, it's equivalent to applying a uniform additive offset or multiplicative scaling to the Softmax input. After Softmax normalization, the relative proportions of each element remain unchanged, only the absolute values ​​change, and the total output of Softmax remains 1, failing to compress the attention weights to near zero. Performing multiplication after Softmax directly changes the absolute sum of the weights, shrinking it from 1 to the value of the pass rate scalar, thus truly achieving continuous adjustment of the throughput of old planning information. Each of the eight gating attention numerator matrices undergoes matrix multiplication with its corresponding value submatrix (11 rows, 48 ​​columns), i.e., 8 rows, 11 columns multiplied by 11 rows, 48 ​​columns, resulting in eight sets of 8 rows, 48 ​​columns of cross-attention outputs. These eight outputs are concatenated end-to-end along the feature dimension to form an 8 rows, 384 columns tensor, which is then transformed by a 384 rows, 384 columns linear output projection matrix to obtain an 8 rows, 384 columns of cross-attention fusion output. The fused output is element-wise added to the observed hidden state (both in 8 rows and 384 columns) to complete the residual connection. The significance of the residual connection is that when all 8 head-level gating coefficients simultaneously approach 1, the cross-attention fusion output approaches an all-zero matrix. The result of the addition is equal to the observed hidden state itself. The inference path of the policy network automatically degenerates into a standard single-stream structure without any old planning information, and will not lose the features of the current observation due to the old planning being masked. The result of the residual addition is normalized by LayerNorm to eliminate numerical scale drift, producing a revised fused feature with a shape of 8 rows and 384 columns.

[0038] In one specific embodiment, step S3, which outputs a rolling temporal action block sequence via the action prediction head using the revised fusion features, includes: The revised fusion features are input into the subsequent self-attention coding layer to continue self-attention calculation and feedforward network processing to obtain the deep fusion hidden state; Take the hidden state vector corresponding to the last time step from the deep fusion hidden state as the decision anchor point feature; The decision anchor features are regressed and mapped through an action prediction head composed of two layers of linear transformation and a nonlinear activation function to obtain a flat output vector with dimension K multiplied by the dimension of the end incremental action. The flat output vector is reshaped in K steps to obtain a rolling temporal motion block sequence containing K-step end-increment actions, where each end-increment action contains a 3D position increment and a 3D attitude increment.

[0039] Specifically, the revised fusion feature has a shape of 8 rows and 384 columns, carrying the observation information after cross-attention gating fusion and the old planning revision information. This tensor is input into the 3rd and 4th layers of the self-attention encoder for further processing. The internal structure of these two layers is exactly the same as the first two layers. Each layer consists of an 8-head self-attention sublayer and a feedforward network sublayer connected in series. Each sublayer is followed by residual connections and LayerNorm normalization. The internal expansion dimension of the feedforward network is also 1536. After processing by the 3rd layer, the representations of each time step in the revised fusion feature are further cross-fused. After processing by the 4th layer, the output shape is still a deep fusion hidden state with 8 rows and 384 columns. At this time, each of the 8 rows contains the observation information from all 8 historical time steps and the old planning revision information after gating, but the information emphasis of each row is different due to the different distribution of self-attention weights. The decision anchor feature is taken from the last row (row 8) of the deep fusion hidden state, corresponding to the current frame with the latest time index. This row is the only one that can simultaneously consider all seven preceding frames (gathering information from historical frames through causal relationships) and superimpose the hidden state vector of the latest observation in the current frame during self-attention calculation. The last row is selected instead of performing average pooling or weighted aggregation on all eight rows because the core logic of the rolling temporal control is to predict the action sequence in the future based on the latest state at the current moment. The hidden state of the current frame has already fully absorbed the context of historical frames through the self-attention mechanism, and no additional pooling operation is needed to introduce redundancy.

[0040] The decision anchor feature is a 384-dimensional vector, which is input to the action prediction head for regression mapping from the latent space to the action space. The action prediction head consists of two linear transformation layers. The first layer maps the 384-dimensional input to a 384-dimensional latent layer and processes it with the ReLU activation function. The second layer maps the 384-dimensional latent layer to a 72-dimensional output. The value 72 is equal to K multiplied by the dimension of the terminal incremental action, i.e., 12 multiplied by 6. The first layer maintains the same 384 dimensions as the input instead of expanding or compressing it because the dimensionality reduction from the 384-dimensional latent space to the 72-dimensional action space is already significant. Adding another nonlinear transformation of the same dimension in the middle gives the network the opportunity to reorganize the features before dimensionality reduction. The 72-dimensional flattened output vector of the second layer is segmented and reshaped into groups of 6 elements, forming a 12-row, 6-column two-dimensional tensor, which contains a rolling temporal action block sequence of 12 terminal incremental actions. The first row of the 12 rows corresponds to the immediate action to be executed in the current control cycle, while rows 2 through 12 correspond to the predicted actions for the next 11 control cycles. Each row contains 6 elements: the first 3 are the position increments in the end-effector coordinate system, corresponding to translations in the X, Y, and Z axes, respectively, in millimeters; the last 3 are the attitude increments, corresponding to rotations around the X, Y, and Z axes, expressed in Euler angles in radians. End-effector increments, rather than absolute joint angles, are chosen as the action representation because the increment form naturally has a smaller numerical range. The output fluctuations of the policy network are constrained to the millimeter and milliradian level. Combined with the limiting threshold of the subsequent safety arbitration layer, this effectively prevents abnormal jumps in single-step output from directly driving the robotic arm into dangerous postures.

[0041] Figure 3 This diagram illustrates the convergence curve of the loss value during the training process of the policy network in this embodiment. The horizontal axis represents the training epoch, and the vertical axis represents the loss value. The solid line represents the total loss, the dashed line represents the action prediction loss, and the dotted line represents the revision constraint loss. The vertical dotted line marks the starting position of the gradual increase phase of the revision constraint loss weight. The action prediction loss participates in optimization from the beginning of training and shows a smooth decreasing trend, decreasing most significantly in the first 30 epochs and stabilizing at around 0.2 after the 50th epoch. The revision constraint loss linearly increases its weight coefficient from 0 to 0.3 in the first 4 epochs (the gradual weight increase phase). After the 4th epoch, it officially participates in gradient updates, and its loss value gradually decreases from approximately 0.38, stabilizing below 0.05 after the 60th epoch. This indicates that the gating projection module has learned to guide predictions closer to the draft rather than deviating significantly in stable scenarios. The total loss is the sum of the two losses, and the overall convergence trend is stable, without any training oscillations or divergences caused by the intervention of the revision constraint loss.

[0042] In one specific embodiment, step S4 includes: Take the first step end increment action from the rolling time domain action block sequence, and then pass it through the absolute limiting layer to trim each component of the position increment to a preset displacement threshold range and each component of the attitude increment to a preset angle threshold range to obtain the limited action. The target end pose is obtained by summing the current end pose with the limited action. The target end pose is compared with the preset three-dimensional safety envelope boundary coordinate by coordinate. Components that exceed the three-dimensional safety envelope boundary are set to zero. The target end pose is then verified by solving the target joint angle through inverse kinematics and comparing it with the joint limit and the Jacobian matrix condition number threshold. Incremental commands that fail the verification are set to zero as a whole to obtain the safe incremental action. The safety incremental action and the incremental command actually executed in the previous control cycle are filtered by a first-order exponential moving average with a preset filtering coefficient to obtain a smoothed incremental command, which is then sent to the robot controller for execution. The remaining action sequence consisting of steps 2 to K in the rolling time-domain action block sequence is written into the draft cache, overwriting the old draft of the previous control cycle, so that the planning draft token sequence can be generated from the draft cache in step S1 of the next control cycle.

[0043] Specifically, the rolling temporal action block sequence is a 12-row, 6-column tensor. The first row represents the end-point incremental action to be executed in the current control cycle, containing six components: 3D position increment and 3D attitude increment. The absolute limiting layer independently prunes these six components. The X, Y, and Z components of the position increment are each pruned to a range of -1.0 mm to +1.0 mm. The three components of the attitude increment around the X-axis, Y-axis, and Z-axis are each pruned to a range of -0.0175 radians to +0.0175 radians, i.e., ±1 degree. The pruning operation works as follows: if the value of a component exceeds the upper bound, it is forcibly set to the upper bound value; if it is below the lower bound, it is forcibly set to the lower bound value. Components within the range retain their original values. The thresholds of 1.0 mm and 1 degree are set based on the upper limit of the physical motion capability of an industrial six-axis robotic arm in a single control cycle at a control frequency of 10 to 15 Hz. Taking 10 Hz as an example, each cycle lasts about 100 milliseconds. The maximum translational speed of the end effector is usually no more than 10 mm per second, corresponding to 1.0 mm per cycle, and the maximum rotational speed of the end effector is usually no more than 10 degrees per second, corresponding to 1 degree per cycle. Increments exceeding these ranges indicate that the policy network has generated abnormal jump outputs rather than normal motion planning. The six components after clipping constitute the limited motion.

[0044] The current end-effector pose is a 6-dimensional vector read in real-time from the robot controller. It includes the X, Y, and Z coordinates (in millimeters) of the end-effector's center point in the base coordinate system and its Euler angles (in radians) around the three axes. This 6-dimensional vector is element-wise added to the six components of the limited motion to obtain the target end-effector pose. The 3D safety envelope is a pre-calibrated axis-aligned cuboid region defined by six boundary values: Xmin, Xmax, Ymin, Ymax, Zmin, and Zmax. This region covers the entire spatial range that the end-effector is allowed to reach when the robot performs the target task. The coordinate-wise comparison logic is as follows: the X coordinate of the target end-effector pose is compared with Xmin and Xmax. If it exceeds these values, the X-direction position increment in the limited motion is forcibly set to zero instead of setting all six components to zero. The Y and Z directions are independently evaluated using the same criteria. The attitude components do not participate in the envelope determination because the safety constraints of the attitude are handled by the next layer of singularity interception. After the incremental command is processed by workspace blocking, it enters the singularity interception and verification. The target end-effector pose is converted into six target joint angle values ​​by the inverse kinematics solver of the robotic arm. Each target joint angle is checked to see if the absolute value of the difference between it and the mechanical limit angle of that joint is less than 5 degrees. The 5-degree margin is set because the robotic arm has inertial overshoot during high-speed movement. If the joint angle is close to the limit, even if the increment itself is not large, it may collide with the mechanical limit due to subsequent inertia. At the same time, the condition number of the Jacobian matrix under the current configuration is calculated. The Jacobian matrix is ​​a 6-row, 6-column matrix that maps the end-effector Cartesian velocity to the joint velocity. The condition number is equal to the ratio of its maximum singular value to its minimum singular value. The larger the value, the closer the robotic arm is to a kinematically singular configuration, that is, a small end-effector movement in some directions requires a large change in the joint angle. The condition number threshold is set to 1000. When the condition number exceeds 1000, it is considered to be close to a singular pose. All six incremental components of the current cycle are set to zero to keep the robotic arm stationary, rather than just shielding some components. This is because any movement in any direction near a singular pose carries the risk of unpredictable joint velocity amplification. Incremental commands that pass joint limit checks and condition number checks are considered safe incremental actions.

[0045] Before being sent to the robot controller, the safety incremental action undergoes a first-order exponential moving average filter to eliminate residual high-frequency jitter. The filter is calculated as follows: the final smoothed incremental command issued in the current cycle equals 0.7 multiplied by the current safety incremental action plus 0.3 multiplied by the smoothed incremental command actually issued in the previous control cycle, where 0.7 is a preset filter coefficient. This coefficient physically means that the new command in the current cycle accounts for 70% of the final output weight, while historical commands account for 30%. The filter coefficient is set to 0.7 instead of a smaller value such as 0.5 or a larger value such as 0.9 because 0.5 would result in an overly strong smoothing effect, causing the robot arm to significantly lag in its response to the strategy network's output, leading to sluggish performance in scenarios requiring rapid workpiece displacement tracking; 0.9 would have almost no filtering effect, and the high-frequency noise components in the strategy network output would be directly transmitted to the robot arm, causing end effector chatter. 0.7 has been verified in multi-station tests as a reasonable value that balances smoothness and response speed. The smooth incremental commands are sent to the robot controller via the TCP communication interface in MoveL incremental motion mode. The controller's built-in servo interpolation module is responsible for converting the incremental commands into current drive signals for each joint. The execution of the current control cycle ends here. Subsequently, the 11 end incremental actions from lines 2 to 12 of the rolling time-domain action block sequence are written to the draft buffer to overwrite the old data of the previous cycle. The draft buffer always retains only the prediction tail of the most recent control cycle. When the next control cycle starts, it is read and encoded into a new planning draft token sequence through the draft projection layer. This establishes a link for the transmission of prediction information between adjacent control cycles, enabling the policy network of the current cycle to refer to the planning intent of the previous cycle for revised reasoning in the cross-attention layer instead of generating entirely new action blocks from scratch.

[0046] The robot learning method based on action segmentation and scrolling time-domain control in the embodiments of this application has been described above. The robot learning system based on action segmentation and scrolling time-domain control in the embodiments of this application is described below. One embodiment of the robot learning system based on action segmentation and scrolling time-domain control in the embodiments of this application includes: The encoding module is used to encode multiple consecutive frames of images and their corresponding robot states into an observation token sequence, and to project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence. The mapping module is used to obtain the observation difference vector by subtracting the visual features of the current frame from the visual features of the previous frame, and to map the observation difference vector into several head-level gating coefficients by linear projection and activation function; The prediction module is used to encode the observed token sequence through self-attention to obtain the observed hidden state and use it as a query matrix. The planning draft token sequence is mapped to a key matrix and a value matrix respectively. An attention score matrix is ​​calculated from the query matrix and the key matrix. The attention score matrix is ​​scaled head-by-head with the head-level gating coefficient to obtain a gated attention score matrix. The gated attention score matrix and the value matrix are weighted and summed and connected through residuals to obtain a revised fusion feature. The revised fusion feature is output as a rolling temporal action block sequence through the action prediction head. The generation module is used to extract the first action from the rolling time-domain action block sequence in each rolling time-domain control cycle, issue it for execution after security arbitration, and write the remaining steps in the rolling time-domain action block sequence into the draft cache for the generation of the planning draft token sequence in the next control cycle.

[0047] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot learning method based on action segmentation and rolling temporal control, characterized in that, The method includes: Step S1: Encode multiple consecutive frames of images and their corresponding robot states into observation token sequences, and project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence. Step S2: Calculate the difference between the visual features of the current frame and the visual features of the previous frame to obtain the observation difference vector, and map the observation difference vector to a number of head-level gating coefficients through linear projection and activation function; Step S3: The observation token sequence is self-attention encoded to obtain the observation hidden state and used as the query matrix. The planning draft token sequence is mapped to the key matrix and the value matrix respectively. The attention score matrix is ​​calculated by the query matrix and the key matrix. The attention score matrix is ​​scaled head-by-head with the head-level gating coefficient to obtain the gated attention score matrix. The gated attention score matrix and the value matrix are weighted and summed and connected by residuals to obtain the revised fusion feature. The revised fusion feature is output as a rolling temporal action block sequence through the action prediction head. Step S4: In each rolling time-domain control cycle, the first action is extracted from the rolling time-domain action block sequence, executed after security arbitration, and the remaining steps in the rolling time-domain action block sequence are written into the draft cache for the next control cycle to generate the planning draft token sequence.

2. The robot learning method based on action segmentation and rolling temporal control according to claim 1, characterized in that, Step S1 includes: H consecutive frames of images are input frame by frame into a pre-trained visual encoder for global pooling feature extraction, resulting in a visual feature sequence composed of H visual feature vectors. The joint angle, end pose, joint angular velocity and gripper state corresponding to each frame are concatenated into a robot state vector. The robot state vector is then input into a two-layer state encoder for nonlinear mapping to obtain a state feature sequence aligned with the frame number of the visual feature sequence. The visual feature sequence and the state feature sequence are concatenated along the feature dimension and then transformed by a linear projection layer to obtain the observation token sequence. The unexecuted action tail, consisting of steps 2 to 3 from the action block output in the previous control cycle, is read from the draft cache. The unexecuted action tail is then input into the draft projection layer for dimension mapping to obtain the planning draft token sequence. In the first control cycle, when the draft cache is empty, each element of the planning draft token sequence is set to a zero vector.

3. The robot learning method based on action segmentation and rolling time-domain control according to claim 2, characterized in that, Step S2 includes: The visual feature vectors of the current frame and the previous frame are extracted from the visual feature sequence, and the visual feature vectors of the current frame and the previous frame are subtracted element by element to obtain the observation difference vector. The observed difference vector is input into the first linear layer of the gated projection module for dimensionality reduction mapping. The intermediate features after dimensionality reduction mapping are then processed by a normalization layer and a nonlinear activation function to obtain the gated intermediate features. The gated intermediate features are input into the second linear layer of the gated projection module for projection, resulting in a gated original score vector with a dimension equal to the number of cross attention heads M. The Sigmoid activation function is applied element by element to the original gating score vector to obtain M first-level gating coefficients with values ​​ranging from zero to one. The bias parameter of the second linear layer is initialized to a negative value so that each first-level gating coefficient is less than 0.5 when the observation difference vector is zero.

4. The robot learning method based on action segmentation and rolling temporal control according to claim 1, characterized in that, In step S3, the observed token sequence is self-attention encoded to obtain the observed hidden state, which is then used as the query matrix. The planning draft token sequence is mapped into a key matrix and a value matrix, respectively, including: The observation token sequence is input into a multilayer self-attention encoder, and the observation hidden state is obtained after layer-by-layer self-attention calculation and feedforward network processing. Based on the M query projection matrices, the observed hidden state is linearly transformed to obtain M sets of query sub-matrices; Based on the M key projection matrices, the planning draft token sequence is linearly transformed to obtain M sets of key sub-matrices; Based on the M value projection matrices, the planning draft token sequence is linearly transformed to obtain M sets of value sub-matrices.

5. The robot learning method based on action segmentation and rolling temporal control according to claim 4, characterized in that, In step S3, the attention score matrix is ​​scaled head-to-head using the head-level gating coefficients to obtain a gated attention score matrix. The revised fusion feature is obtained by weighted summation of the gated attention score matrix and the value matrix, followed by residual concatenation. This includes: Perform the transpose dot product operation on the i-th query submatrix and the i-th key submatrix, divide by the square root of the feature dimension, and process by the normalized exponential function to obtain the original attention numerator matrix of the i-th group. Iterate through M groups and perform the above operation to obtain the attention score matrix. The difference between 1 and the i-th coefficient in the first-level gating coefficient is calculated as the pass rate scalar of the i-th group. The i-th group pass rate scalar is used to perform element-wise multiplication on the molecular matrix of the original attention of the i-th group. The above operation is performed on M groups respectively to obtain the gating attention score matrix. The i-th group of gated attention is obtained by performing matrix multiplication with the i-th group of value submatrix to obtain the i-th group of cross attention output. The M groups of cross attention outputs are concatenated on the feature dimension and projected through linear output to obtain the cross attention fusion output. The cross-attention fusion output is added element-wise to the observed hidden state to complete the residual connection, and the revised fusion feature is obtained after processing by the normalization layer.

6. The robot learning method based on action segmentation and rolling time-domain control according to claim 5, characterized in that, In step S3, the revised fusion features are processed by the action prediction head to output a rolling temporal action block sequence, including: The revised fusion features are input into the subsequent self-attention coding layer to continue self-attention calculation and feedforward network processing to obtain the deep fusion hidden state; Take the hidden state vector corresponding to the last time step from the deep fusion hidden state as the decision anchor point feature; The decision anchor features are regressed and mapped through an action prediction head composed of two layers of linear transformation and a nonlinear activation function to obtain a flat output vector with dimension K multiplied by the dimension of the end incremental action. The flat output vector is reshaped in K steps to obtain a rolling temporal motion block sequence containing K-step end-increment actions, where each end-increment action contains a 3D position increment and a 3D attitude increment.

7. The robot learning method based on action segmentation and rolling temporal control according to claim 1, characterized in that, Step S4 includes: Take the first step end increment action from the rolling time domain action block sequence, and then pass it through the absolute limiting layer to trim each component of the position increment to a preset displacement threshold range and each component of the attitude increment to a preset angle threshold range to obtain the limited action. The target end pose is obtained by summing the current end pose with the limited action. The target end pose is compared with the preset three-dimensional safety envelope boundary coordinate by coordinate. Components that exceed the three-dimensional safety envelope boundary are set to zero. The target end pose is then verified by solving the target joint angle through inverse kinematics and comparing it with the joint limit and the Jacobian matrix condition number threshold. Incremental commands that fail the verification are set to zero as a whole to obtain the safe incremental action. The safety incremental action and the incremental command actually executed in the previous control cycle are filtered by a first-order exponential moving average with a preset filtering coefficient to obtain a smoothed incremental command, which is then sent to the robot controller for execution. The remaining action sequence consisting of steps 2 to K in the rolling time-domain action block sequence is written into the draft cache, overwriting the old draft of the previous control cycle, so that the planning draft token sequence can be generated from the draft cache in step S1 of the next control cycle.

8. A robot learning system based on action segmentation and rolling time-domain control, characterized in that, For implementing the robot learning method based on action segmentation and rolling time-domain control as described in any one of claims 1-7, the robot learning system based on action segmentation and rolling time-domain control comprises: The encoding module is used to encode multiple consecutive frames of images and their corresponding robot states into an observation token sequence, and to project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence. The mapping module is used to obtain the observation difference vector by subtracting the visual features of the current frame from the visual features of the previous frame, and to map the observation difference vector into several head-level gating coefficients by linear projection and activation function; The prediction module is used to encode the observed token sequence through self-attention to obtain the observed hidden state and use it as a query matrix. The planning draft token sequence is mapped to a key matrix and a value matrix respectively. An attention score matrix is ​​calculated from the query matrix and the key matrix. The attention score matrix is ​​scaled head-by-head with the head-level gating coefficient to obtain a gated attention score matrix. The gated attention score matrix and the value matrix are weighted and summed and connected through residuals to obtain a revised fusion feature. The revised fusion feature is output as a rolling temporal action block sequence through the action prediction head. The generation module is used to extract the first action from the rolling time-domain action block sequence in each rolling time-domain control cycle, issue it for execution after security arbitration, and write the remaining steps in the rolling time-domain action block sequence into the draft cache for the generation of the planning draft token sequence in the next control cycle.

9. The robot learning system based on action segmentation and rolling time-domain control according to claim 8, characterized in that, Encode consecutive frames of images and their corresponding robot states into an observation token sequence, and project and encode the tail of the unexecuted action block predicted in the previous control cycle into a planning draft token sequence, including: H consecutive frames of images are input frame by frame into a pre-trained visual encoder for global pooling feature extraction, resulting in a visual feature sequence composed of H visual feature vectors. The joint angle, end pose, joint angular velocity and gripper state corresponding to each frame are concatenated into a robot state vector. The robot state vector is then input into a two-layer state encoder for nonlinear mapping to obtain a state feature sequence aligned with the frame number of the visual feature sequence. The visual feature sequence and the state feature sequence are concatenated along the feature dimension and then transformed by a linear projection layer to obtain the observation token sequence. The unexecuted action tail, consisting of steps 2 to 3 from the action block output in the previous control cycle, is read from the draft cache. The unexecuted action tail is then input into the draft projection layer for dimension mapping to obtain the planning draft token sequence. In the first control cycle, when the draft cache is empty, each element of the planning draft token sequence is set to a zero vector.

10. The robot learning system based on action segmentation and rolling time-domain control according to claim 8, characterized in that, The observation difference vector is obtained by subtracting the visual features of the current frame from the visual features of the previous frame. This observation difference vector is then mapped to several head-level gating coefficients via linear projection and an activation function, including: The visual feature vectors of the current frame and the previous frame are extracted from the visual feature sequence, and the visual feature vectors of the current frame and the previous frame are subtracted element by element to obtain the observation difference vector. The observed difference vector is input into the first linear layer of the gated projection module for dimensionality reduction mapping. The intermediate features after dimensionality reduction mapping are then processed by a normalization layer and a nonlinear activation function to obtain the gated intermediate features. The gated intermediate features are input into the second linear layer of the gated projection module for projection, resulting in a gated original score vector with a dimension equal to the number of cross attention heads M. The Sigmoid activation function is applied element by element to the original gating score vector to obtain M first-level gating coefficients with values ​​ranging from zero to one. The bias parameter of the second linear layer is initialized to a negative value so that each first-level gating coefficient is less than 0.5 when the observation difference vector is zero.

Images