Robot continuous action generation method based on local anchor sampling and frequency domain optimization

Abstract

The application discloses a robot coherent action generation method based on local anchor sampling and frequency domain optimization, and relates to the field of robot control strategy. The method comprises the following steps: splicing continuous action blocks in expert action trajectories into composite action blocks; providing sample data for robot operation strategy training by randomly sampling environment observation information and matched composite action blocks of expert demonstration data; deploying the trained robot operation strategy to the control platform of a corresponding entity robot, taking real-time collected environment observation information as the input of the trained operation strategy, and taking the first action block in the predicted composite action block output by the trained operation strategy as a specific execution control instruction, and then receiving new environment observation information in real time and iterating the cycle, so that coherent autonomous action control of the robot is finally realized. The application can generate long-sequence actions with foresight and coherence while maintaining the accuracy of proximal operation.

Description

Technical Field

[0001] This invention relates to the field of robot control strategy technology, specifically to a method for generating coherent robot movements based on local anchoring sampling and frequency domain optimization. Background Technology

[0002] In the field of autonomous robot operation, vision-driven strategies based on imitation learning are the core technology for achieving automated control of complex robot tasks. Currently, conventional imitation learning techniques typically segment the motion trajectories demonstrated by experts, dividing the imitated expert motion trajectories into multiple fixed-length action blocks. Each action block contains execution actions across multiple consecutive time steps. These conventional methods use environmental observation information at any single time step as input, and the action blocks matching the expert motion trajectories at the corresponding time step as the learning target distribution. The robot's operation strategy is trained based on the sample data of the segmented action blocks, enabling the trained strategy to predict and output a continuous sequence of future actions in a single step based on real-time environmental observation information.

[0003] The aforementioned imitation learning scheme based on action block decomposition can effectively reduce the compound error between the robot's predicted action trajectory and the learned expert demonstration action trajectory (i.e., the cumulative error between the corresponding action blocks in the robot's predicted action block and the expert action trajectory under the same environmental observation information). However, in long-sequence continuous operation task scenarios, this technical solution has many inherent deep-seated defects, making it difficult to meet the needs of practical applications. The specific defects are as follows:

[0004] First, there is a problem of discontinuity in the execution of robot actions. Existing robot operation strategy learning methods only focus on optimizing and fitting the probability distribution of actions within a single action block, without constraining and optimizing the smooth transition characteristics during the execution of adjacent action blocks. When robots perform long-term continuous tasks, the transition points between adjacent action blocks are prone to fluctuations in motion trajectories and discontinuous action execution, resulting in obvious motion stuttering. This not only directly reduces the precision of the robot's operation but also significantly weakens the overall success rate of the robot's task execution.

[0005] Secondly, there is a lack of forward-looking prediction capabilities for long-term actions. For long-term expert action trajectory learning and training scenarios, conventional solutions typically increase the length of individual action blocks to reduce the composite error of policy prediction. However, increasing the length of action blocks directly leads to a rapid increase in the probability complexity of the action distribution of the target action block being learned. Furthermore, the loss function used in existing policy training primarily performs time-by-time action alignment in the temporal domain (referred to as temporal loss). Configuring only a single temporal loss makes it difficult to effectively capture the macroscopic action change trends and long-term spatiotemporal correlations of long-cycle tasks. When attempting to forcibly fit the extended, large-scale action blocks, the policy model is prone to action prediction averaging effects, resulting in blurred and distorted action predictions. This fails to simultaneously meet the dual requirements of precise near-end action execution accuracy and accurate prediction of long-term action trends.

[0006] Third, the consistent flow matching model suffers from performance bottlenecks such as low training efficiency and poor convergence stability. Current core foundational models for robot manipulation strategies include two types of generative models: diffusion models and consistent flow matching models. Among them, the consistent flow matching model leverages the technological advantage of rapid action generation through single-step sampling, resulting in higher action generation efficiency and better adaptability for engineering applications. The training objective of the consistent flow matching model is to fit a velocity vector field that can map from random noise (typically Gaussian noise) to a target distribution (i.e., expert action blocks). This model typically models the evolution from random noise to the target distribution as a continuous temporal evolution process from the starting time 0 to the ending time 1. During training, conventional methods randomly sample two times between 0 and 1. The consistent flow matching model predicts the corresponding velocity field based on these two sampled times. Then, it calculates the temporal loss between the two velocity fields mapped to the action blocks at the ending time, guiding the consistent flow matching model to predict the velocity field that can be mapped to the target distribution at each time by minimizing this temporal loss. Specifically, for the two random sampled times, conventional methods perform uniform random sampling between time 0 and time 1. However, this uniform random sampling scheme can easily reduce the gradient propagation efficiency of the target distribution during the training of the consistency flow matching model. The strength of the target distribution's participation in gradient guidance during consistency flow matching strategy training is significantly related to the random sampling time during training: when the random sampling time is biased towards time 0, the consistency flow matching model is more likely to learn noise features, and the gradient guidance involved by the target distribution is weaker; conversely, when the sampling time is biased towards time 1, the gradient guidance involved by the target distribution is stronger. The propagation efficiency of the target distribution during training directly affects the training performance of the consistency flow matching model. For complex expert action trajectories, uniform sampling cannot provide sufficient gradient propagation efficiency for the target distribution, resulting in slow learning efficiency, poor convergence stability, and insufficient robustness of the robot's operation strategy for long sequence trajectories, making it difficult to adapt to the application requirements of various complex real-world robot operation scenarios. Summary of the Invention

[0007] In view of at least one of the above-mentioned technical problems, this application provides a method for generating continuous robot actions based on local anchor sampling and frequency domain optimization. This method aims to solve the technical problems of unsmooth transitions between action blocks, difficulty in capturing long-range action trends, and low target distribution transfer efficiency during the training of the core generative model within the strategy in existing vision-driven robot operation strategies during the generation of long sequence actions. This invention, by introducing a frequency domain optimization mechanism and employing local anchor sampling technology in the training of the flow matching strategy, strives to achieve long sequence action generation with foresight and coherence while maintaining proximal operation accuracy.

[0008] The technical solution of this invention is:

[0009] A method for generating coherent robot motions based on local anchoring sampling and frequency domain optimization, comprising the following steps:

[0010] By aggregating consecutive action blocks in the expert's action trajectory, a composite action block is formed;

[0011] Randomly sample environmental observation information from expert demonstration data, and obtain composite action blocks that match the environmental observation information, namely expert composite action blocks;

[0012] The environmental observation information is paired with the corresponding expert compound action blocks, and all paired samples constitute the training set.

[0013] Randomly sampled paired samples from the training set are used to train a robot operation strategy based on a consistent flow matching model. The sampled environmental observation information serves as the input to the operation strategy, and the output of the operation strategy is a predicted composite action block. The corresponding expert composite action blocks in the paired samples serve as the training supervision target for the output of the strategy.

[0014] The trained robot operation strategy is deployed to the control platform of the corresponding physical robot. Real-time environmental observation information is used as the input of the trained operation strategy, and the first action block in the predicted composite action block output by the trained operation strategy is used as the specific execution control command. After execution, new environmental observation information is received in real time and iterative loop is performed to finally achieve coherent autonomous motion control of the robot.

[0015] Furthermore, according to the robot coherent motion generation method, the step of randomly sampling paired samples from the training set to train a robot operation strategy based on a consistency flow matching model includes:

[0016] The two times between sampling start time 0 and terminal time 1 and ;

[0017] Calculate sampling time and Timing interpolation of corresponding composite action blocks and ;

[0018] Predicting sampling time using a consistent flow matching model and Corresponding velocity field and ,in Represents the current time step Corresponding environmental observation information ;

[0019] Based on respectively and Generate the corresponding predicted compound action block;

[0020] Design a formula for calculating total loss and complete the training of robot operation strategies based on a consistent flow matching model.

[0021] Furthermore, according to the aforementioned method for generating coherent robot movements, the time... , representing the noisy moment, obtained through random sampling.

[0022] Furthermore, according to the aforementioned method for generating coherent robot movements, the time... The specific formula for determining the time based on the probability distribution is as follows:

[0023] (1)

[0024] in, The bias sampling function is determined by adjusting the distribution parameters. and , making The sampling points fall near the endpoint of the flow matching velocity field during the training process, that is, near the terminal time 1.

[0025] Furthermore, according to the aforementioned method for generating coherent robot movements, the calculation of sampling time... and Timing interpolation of corresponding composite action blocks and This includes: according to the optimal transmission law, the sampling time... and The interpolation intermediate quantity is defined as follows:

[0026] (2)

[0027] in, , representing Gaussian noise; This represents expert compound action blocks from paired samples sampled from the training set.

[0028] Furthermore, according to the aforementioned method for generating coherent robot movements, the steps respectively based on and The corresponding predicted compound action block is generated, and the specific implementation formula is as follows:

[0029] (3)

[0030] in, This represents the prediction process of composite action blocks based on the predicted velocity field.

[0031] Furthermore, according to the robot continuous motion generation method described above, the total loss calculation formula is composed of a weighted fusion of time-domain loss and frequency-domain loss, and the specific calculation formula is as follows:

[0032] (4)

[0033] in, Indicates the total loss; Indicates time-domain loss; Indicates frequency domain loss; These are the weighting coefficients for the frequency domain loss.

[0034] Furthermore, according to the aforementioned method for generating coherent robot motions, the method includes: calculating the temporal loss using mean square error based on the first motion block in the predicted composite motion block. The specific calculation formula is as follows:

[0035] (5)

[0036] in This represents the completed consistency flow matching model.

[0037] Furthermore, according to the robot coherent motion generation method, the method includes: mapping expert composite action blocks and corresponding predicted composite action blocks sampled from the training set to the frequency domain; and using the coefficients of the predicted composite action blocks and expert composite action blocks across the entire frequency band. Distance calculated frequency domain loss The specific calculation formula is as follows:

[0038] (7)

[0039] in ; Indicates the length of the compound action block; To predict the frequency coefficients corresponding to each action dimension of a composite action block; These are the frequency coefficients corresponding to each action dimension of the expert compound action block.

[0040] The beneficial effects of adopting the above technical solution are as follows:

[0041] (1) This application introduces the long-range motion trend in robot operation as a regularization term by calculating the frequency domain loss between the action block predicted by the operation strategy and the action block of the learned expert action trajectory, thereby fundamentally solving the "boundary break" problem inherent in the learning scheme based on action blocks; experimental verification shows that the robot operation strategy trained by the method of this invention generates operation trajectories that are smoother in three-dimensional space, which can significantly improve the action coherence and operation stability of the robot in complex long sequence operation tasks.

[0042] (2) This application combines the frequency domain loss of multiple consecutive action blocks with the time domain loss of a single action block into a Foresight Composite Objective (FCO), which enables the robot to fully consider the temporal evolution trend of multiple action blocks in the future, and then predict the optimal action sequence that is adapted to the smooth connection of subsequent action blocks, effectively alleviating the composite error in the long sequence of continuous operation of the robot.

[0043] (3) Locally Anchored Sampling (LAS) technique changes the sampling form at random moments in the training of the consistent flow matching model. It changes the conventional uniform random sampling to local anchored sampling, which can enhance the strength of the learned target distribution in gradient guidance in the training of the consistent flow matching strategy, obtain higher target distribution transmission efficiency, and thus enable the consistent flow matching model to have a faster convergence speed when learning complex action blocks. Under the same number of training iterations, compared with the standard flow matching method, the operation strategy trained by this invention can generate action distributions with higher fidelity and more accurate alignment with expert demonstration trajectories.

[0044] (4) Because it effectively reduces the compound error of long-term continuous operation, ensures the continuity and smoothness of robot action generation and execution, the robot control instructions output by the method of this invention can reduce the operating losses caused by frequent start-stop of robot joint motors and fluctuations in operating conditions, extend the service life of robot hardware, and better meet the application needs of actual industrial and service scenarios for robot deployment. Attached Figure Description

[0045] Figure 1 This embodiment presents a flowchart of a robot continuous motion generation method based on local anchoring sampling and frequency domain optimization.

[0046] Figure 2This is a schematic diagram of the process for training the robot operation strategy based on the consistency flow matching model in this embodiment.

[0047] Figure 3 Figure 1 shows a comparison between local anchored sampling and traditional uniform random sampling in this embodiment. Figure 2(a) is a schematic diagram of the conventional uniform random sampling time, and Figure 3(b) is a schematic diagram of the local anchored sampling time.

[0048] Figure 4 Figure (a) is a schematic diagram comparing the conventional robot operation strategy with the robot operation strategy guided by the method of the present invention, and Figure (b) is a schematic diagram of the conventional robot operation strategy.

[0049] Figure 5 The figures show a comparison of the operation trajectories and composite errors generated by the method of this invention and the comparative method FlowPolicy under eight multi-step operation tasks on the simulation platform MetaWorld. Figure (a) corresponds to the Bin Picking task, Figure (b) corresponds to the Push Wall task, Figure (c) corresponds to the Pick Place task, Figure (d) corresponds to the Pick Out of Hole task, Figure (e) corresponds to the Sweep Into task, Figure (f) corresponds to the Pick Place Wall task, Figure (g) corresponds to the Disassemble task, and Figure (h) corresponds to the Shelf Place task. Detailed Implementation

[0050] To facilitate understanding of this application, a more comprehensive description of this application will be provided below with reference to the accompanying drawings.

[0051] Figure 1 This is a flowchart illustrating the training process for the robot coherent motion generation strategy based on local anchoring time sampling and predictive composite target optimization. (Example) Figure 1 As shown, the robot continuous motion generation method based on local anchoring sampling and frequency domain optimization includes the following steps:

[0052] Step 1: Aggregate the continuous action blocks in the expert action trajectory to form a composite action block;

[0053] This implementation extracts the current time step from expert operation demonstration data. Corresponding environmental observation information Based on the training requirements of the robot's operation strategy, the expert motion trajectories are segmented into fixed-length action blocks, along with the corresponding expert motion trajectories. .Right now ,in Indicates based on environmental observation information The single-step action taken by the expert strategy; The length of the action block.

[0054] To endow the robot's operation strategy with the ability to predict remote actions, in a preferred embodiment, several consecutive action blocks are aggregated and defined as a composite action block that exceeds the length of a single action block. .in, This indicates the length of the compound action block, that is, the length of the compound action block. It is composed of a series of action blocks; This indicates the dimension of the step action.

[0055] Step 2: Randomly sample environmental observation information from the expert demonstration data, obtain the composite action block that matches the environmental observation information, i.e., the expert composite action block, pair the sampled environmental observation information with the corresponding expert composite action block, and all paired samples constitute the training set.

[0056] Step 3: Randomly sample paired samples from the training set constructed in Step 2 to train the existing robot operation strategy based on the consistency flow matching model.

[0057] Figure 2 This is a flowchart illustrating the training process for the robot operation strategy based on the consistent flow matching model in this embodiment. In a preferred embodiment, the environmental observation information obtained from sampling the training set is used as the input to the robot operation strategy, with reference to... Figure 2 The output of the robot's operation strategy is a predicted compound action block; the corresponding expert compound action block in the paired sample is used as the training supervision target for the predicted compound action block output by the robot's operation strategy.

[0058] Based on the environmental observation information input from the training samples, the operation strategy with the consistency flow matching model as the core will generate the predicted compound action block through the following three steps.

[0059] Step 3.1: Sample the two time points between the starting time 0 and the ending time 1, including the noise time. and anchoring time ;

[0060] Figure 3 This diagram compares the local anchored sampling and the traditional uniform random sampling scheme in this embodiment. Figure (a) shows the timing of the conventional uniform random sampling, and Figure (b) shows the timing of the local anchored sampling. Step 3.1 specifically includes the following steps:

[0061] Step 3.1.1, for noise time Perform random sampling, where .

[0062] Step 3.1.2: Sample the intermediate moment of the velocity flow anchored by the probability distribution. ;

[0063] In the framework of consistent flow matching, in order to accelerate the learning from Gaussian noise to the action distribution of expert demonstration, this invention designs a local anchoring time sampling mechanism, specifically sampling a velocity flow intermediate time anchored by a probability distribution according to equation (1). :

[0064] (1)

[0065] in, This implementation introduces a bias sampling function, which adjusts the distribution parameters. and , making The sampling points fall more frequently near the endpoint of the flow matching velocity field during the training process, i.e., near the terminal time 1.

[0066] Step 3.2, calculate the sampling time. and Timing interpolation of corresponding composite action blocks and .

[0067] In a preferred embodiment, according to the Optimal Transport (OT) theorem, the sampling time is... and The interpolation intermediate quantity is defined as follows:

[0068] (2)

[0069] in, Indicates Gaussian noise. ; This represents the expert compound action block sampled for training the robot's operation strategy.

[0070] Step 3.3: Predict the sampling time using the consistent flow matching model. and Corresponding velocity field and ;in This represents the parameters of the consistent flow matching model.

[0071] Step 3.4, based on the velocity field respectively and Generate the corresponding predicted compound action block:

[0072] (3)

[0073] in This represents the prediction process of composite action blocks based on the predicted velocity field.

[0074] Step 3.5: Design the total loss function and complete the training of the robot operation strategy based on the consistency flow matching model;

[0075] In this implementation method, when training a robot operation strategy based on a consistent flow matching model, the total training loss is composed of a weighted fusion of time-domain loss and frequency-domain loss.

[0076] (4)

[0077] This implementation method calculates the temporal loss based on the first action block in each predicted composite action block, using the mean square error. The specific formula is as follows:

[0078] (5)

[0079] in This represents the completed consistency flow matching model.

[0080] To fundamentally eliminate the action abruptness problem at the transitions between action blocks, this implementation maps expert composite action blocks and their corresponding predicted composite action blocks from paired samples sampled from the training set to the frequency domain. For each composite action block... Each action dimension Frequency coefficients are extracted using Discrete Cosine Transform (DCT). :

[0081] (6)

[0082] in This implementation uses predictive composite action blocks and expert composite action blocks to evaluate the coefficients across the entire frequency band. Distance-defined frequency domain loss for:

[0083] (7)

[0084] in To predict the frequency coefficient corresponding to each action dimension of the composite action block; For each action dimension of the expert compound action block; Figure 4This diagram illustrates a comparison between conventional robot operation strategies and robot operation strategies guided by the method of this invention. Figure (a) shows a schematic diagram of the conventional robot operation strategy, and Figure (b) shows a schematic diagram of the robot operation strategy guided by the method of this invention. (Reference) Figure 4 Since the discontinuous jumps in the motion trajectory manifest as a surge in high-frequency energy in the frequency domain, by minimizing this frequency domain loss, it is possible to constrain and guide the robot operation strategy model to generate a highly smooth and coherent motion sequence in the time domain.

[0085] Based on the above total loss function, the iterative training and parameter update of the robot operation strategy of this invention are completed through gradient backpropagation.

[0086] Step 4: Deploy the trained robot operation strategy to the control platform of the corresponding physical robot. Use the real-time collected environmental observation information as the input of the strategy, and use the first action block in the predicted composite action block output by the strategy as the specific execution control command. After execution, receive new environmental observation information in real time and iterate in a loop to finally realize the continuous and coherent autonomous action control of the robot.

[0087] In this embodiment, the trained robot operation strategy model is deployed to the control platform of the physical robot. The environmental observation information collected by the sensor is used as the input of the trained robot operation strategy model. The first action block in the predicted composite action block output by the trained robot operation strategy model is used as the specific execution control command. After execution, new environmental observation information is received in real time and iteratively looped to finally realize the continuous and coherent autonomous action control of the robot.

[0088] Figure 5This figure shows a comparison of the operational trajectories and composite errors generated by the method of this invention and the comparative method FlowPolicy under eight multi-step operation tasks on the simulation platform MetaWorld. Figure (a) corresponds to the Bin Picking task, Figure (b) to the Push Wall task, Figure (c) to the Pick Place task, Figure (d) to the Pick Out of Hole task, Figure (e) to the Sweep Into task, Figure (f) to the Pick Place Wall task, Figure (g) to the Disassemble task, and Figure (h) to the Shelf Place task. The comparison of eight typical predicted trajectories on the commonly used robot operation platform MetaWorld demonstrates that the method of this invention has lower trajectory error in imitating trajectories. The left side of the figure shows the robot's motion space coordinate trajectory in three-dimensional space, including the learned expert motion trajectory (black), the trajectory predicted by the method of this invention (red), and the trajectory predicted by the existing method (blue). The right side of the figure shows a comparison of the cumulative error of the trajectory space coordinates (the error calculation formula is: ,in (in three-dimensional space coordinates).

[0089] Furthermore, the method of this invention was compared in 59 operation tasks under two simulation environments and on a real robotic arm. Table 1 shows the average success rate of the method of this invention in the above simulation experiments, where the method achieved an optimal result of 83.6%. Table 2 shows the average success rate of the method of this invention in multi-stage operation tasks on a real machine. The overall average success rate of the method of this invention is higher than that of existing methods, and this performance is more pronounced in multi-stage tasks.

[0090] Table 1. Comparison of average success rates (%) of 53 tasks in the simulation environment

[0091] Table 2. Comparison of average success rates (%) for 6 multi-stage operation tasks on real devices

[0092] It should be understood that, inspired by the technical concept of this invention, those skilled in the art can make various improvements or modifications based on the above content without departing from the scope of this invention, and these modifications still fall within the protection scope of this invention.

Claims

1. A method for generating coherent robot movements based on local anchoring sampling and frequency domain optimization, characterized in that, The method includes the following steps: By aggregating consecutive action blocks in the expert's action trajectory, a composite action block is formed; Randomly sample environmental observation information from expert demonstration data, and obtain composite action blocks that match the environmental observation information, namely expert composite action blocks; The environmental observation information is paired with the corresponding expert compound action blocks, and all paired samples constitute the training set. Randomly sampled paired samples from the training set are used to train a robot operation strategy based on a consistent flow matching model. The sampled environmental observation information serves as the input to the operation strategy, and the output of the operation strategy is a predicted composite action block. The corresponding expert composite action blocks in the paired samples serve as the training supervision target for the output of the strategy. The trained robot operation strategy is deployed to the control platform of the corresponding physical robot. Real-time environmental observation information is used as the input of the trained operation strategy, and the first action block in the predicted composite action block output by the trained operation strategy is used as the specific execution control command. After execution, new environmental observation information is received in real time and iterative loop is performed to finally achieve coherent autonomous motion control of the robot.

2. The method for generating continuous robot motions according to claim 1, characterized in that, The step of randomly sampling paired samples from the training set to train a robot operation strategy based on a consistency flow matching model includes: The two times between sampling start time 0 and terminal time 1 and ; Calculate sampling time and Timing interpolation of corresponding composite action blocks and ; Predicting sampling time using a consistent flow matching model and Corresponding velocity field and ,in Represents the current time step Corresponding environmental observation information ; Based on respectively and Generate the corresponding predicted compound action block; Design a formula for calculating total loss and complete the training of robot operation strategies based on a consistent flow matching model.

3. The method for generating continuous robot motions according to claim 2, characterized in that, The time , representing the noisy moment, obtained through random sampling.

4. The method for generating continuous robot motions according to claim 2, characterized in that, The time The specific formula for determining the time based on the probability distribution is as follows: （1） in, The bias sampling function is determined by adjusting the distribution parameters. and , making The sampling points fall near the endpoint of the flow matching velocity field during the training process, that is, near the terminal time 1.

5. The method for generating continuous robot motions according to claim 2, characterized in that, The calculation sampling time and Timing interpolation of corresponding composite action blocks and This includes: according to the optimal transmission law, the sampling time... and The interpolation intermediate quantity is defined as follows: （2） in, , representing Gaussian noise; This represents expert compound action blocks from paired samples sampled from the training set.

6. The method for generating continuous robot motions according to claim 5, characterized in that, The respectively based on and The corresponding predicted compound action block is generated, and the specific implementation formula is as follows: （3） in, This represents the prediction process of composite action blocks based on the predicted velocity field.

7. The method for generating continuous robot motions according to claim 6, characterized in that, The total loss calculation formula is composed of a weighted fusion of time-domain loss and frequency-domain loss, and the specific calculation formula is as follows: （4） in, Indicates the total loss; Indicates time-domain loss; Indicates frequency domain loss; These are the weighting coefficients for the frequency domain loss.

8. The method for generating continuous robot motions according to claim 7, characterized in that, The method includes: calculating the temporal loss using mean square error based on the first action block in the predicted composite action block. The specific calculation formula is as follows: （5） in This represents the completed consistency flow matching model.

9. The method for generating continuous robot motions according to claim 7, characterized in that, The method includes: mapping expert composite action blocks and corresponding predicted composite action blocks sampled from the training set to the frequency domain; and using the coefficients of the predicted composite action blocks and expert composite action blocks across the entire frequency band. Distance calculated frequency domain loss The specific calculation formula is as follows: （7） in ; Indicates the length of the compound action block; To predict the frequency coefficients corresponding to each action dimension of a composite action block; These are the frequency coefficients corresponding to each action dimension of the expert compound action block.

Images