Method and apparatus for robot motion sequence optimization for chemical laboratory operation tasks

Abstract

The application relates to a robot action sequence optimization method and device for chemical experiment operation tasks, which comprises the following steps: collecting a corresponding expert data set for each chemical experiment operation task, quantifying the corresponding precision requirement, smoothness feature and task duration according to the expert data set; determining the action block size of adaptive length of each chemical experiment operation task according to the quantified task features; replacing the fixed block size setting in the ACT model with the action block size of adaptive length, training the ACS- ACT model by using the corresponding expert data set; and performing inference by using the corresponding trained ACS- ACT model according to the observation information to obtain the predicted future action. After the block size and the output action are optimized, the execution effect of the task is obviously improved, especially for the task with relatively high precision and smoothness requirements, the task duration, success rate and action smoothness are all better than those of the original ACT model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to a robot action sequence optimization method and device for a chemical experiment operation task. BACKGROUND

[0002] In the field of robots, imitation learning (IL) as an efficient autonomous learning method has become one of the key technologies to help robots complete various complex tasks like humans. The core idea of imitation learning is to let the robot learn the strategy of task execution by imitating the behavior of human experts, which not only avoids the tedious work in traditional programming methods, but also accelerates the expansion of application scenarios of the robot system. Especially in the field of chemical experiments, robots can independently perform experimental operations, not only improving the efficiency of experiments, but also effectively reducing human errors and safety risks in the experimental process, and have great application potential.

[0003] The research of imitation learning mainly focuses on training robots through demonstration data, such as robot action data collected by human remote operation of robots or supervised training through expert video demonstration data. By utilizing expert demonstration data, imitation learning enables robots to autonomously learn complex control strategies, thereby reducing dependence on time-consuming and labor-intensive manual programming. The behavior cloning (BC) algorithm published in 1988 is one of the most basic methods of imitation learning. BC trains the model by directly imitating the action sequence of experts, which is simple and effective, but prone to error accumulation and poor generalization when facing complex tasks. With the development of deep learning technology, neural network-based imitation learning methods have gradually replaced traditional BC methods, enabling robots to learn more accurate task execution strategies in more complex and dynamic environments.

[0004] In recent years, many researchers have proposed advanced algorithm-based imitation learning methods to improve the accuracy and continuity of task execution. For example, ACT (Action Chunking with Transformer) is an early advanced algorithm-based imitation learning algorithm, and later pi0, DP (Diffusion Policy), DP3 (3D Diffusion Policy), and pi0.5 are also released. However, whether it is DP, pi0, or pi0.5, it has a high requirement for the number of data sets, and the deployment cost is relatively high. The ACT algorithm improves the accuracy and continuity of robot task execution by decomposing the task into multiple action chunks and learning based on the Transformer architecture. Compared with some platforms that use expensive high-precision sensors for state estimation, the advantage of the ACT algorithm is that it can be trained and executed on a lower-cost platform, and through continuous self-optimization, it can reduce the accumulation of errors, thereby improving the operation accuracy and task success rate of the robot.

[0005] However, although the ACT algorithm has made significant progress in many fields, it still has certain limitations, especially when dealing with tasks with different complexity, precision requirements, and control speed. The ACT algorithm uses a fixed block size to process all tasks, which may cause performance bottlenecks in some fine operation tasks. For example, in chemical experiments, some tasks may require high precision, such as precise reagent grabbing and transfer, while other tasks may have higher tolerance for roughness of actions, such as rough robot movement, which can have relatively large action blocks to achieve better trajectory planning. The mismatch between the fixed block size and the task may cause several problems, such as: when the block size is too large relative to the high-precision task, the model is difficult to capture the necessary fine-grained control signals, resulting in task failure. For rough movement tasks, if the action block size is too small, it cannot effectively eliminate error accumulation. SUMMARY

[0006] To solve the problems in the above-mentioned traditional method, the present application provides a robot action sequence optimization method and device for chemical experiment operation tasks.

[0007] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0008] On the one hand, a robot action sequence optimization method for chemical experiment operation tasks is provided, comprising the steps of:

[0009] Step 1: Collect the corresponding expert data set for each chemical experiment operation task, and quantize the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set to obtain the quantized task feature.

[0010] Step 2: According to the quantized task feature, determine the action block size of adaptive length of each chemical experiment operation task.

[0011] Step 3: Replace the fixed block size setting in the ACT model with the action block size of adaptive length of each chemical experiment operation task, train the ACS-ACT model using the corresponding expert data set, and obtain the trained ACS-ACT model corresponding to the chemical experiment operation task.

[0012] Step 4: According to the observation information, use the corresponding trained ACS-ACT model to infer to obtain the predicted future action.

[0013] Step 5: Smooth the predicted future action to generate a smooth action execution instruction.

[0014] Step 6: The smooth action execution instruction is issued to the execution end of the robot to control the execution end of the robot to complete the chemical experiment operation task, and the task execution state data is fed back.

[0015] On the other hand, an ACS-ACT optimization device for chemical experiment operation tasks is also provided, comprising:

[0016] The task feature quantization module is configured to collect the corresponding expert data set for each chemical experiment operation task, and quantize the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set to obtain the quantized task feature.

[0017] The adaptive action block size adjustment module is configured to determine the action block size of adaptive length of each chemical experiment operation task according to the quantized task feature.

[0018] The ACT model training module is configured to replace the fixed block size setting in the ACT model with the action block size of adaptive length of each chemical experiment operation task, train the ACS-ACT model using the corresponding expert data set, and obtain the trained ACS-ACT model corresponding to the chemical experiment operation task.

[0019] The inference module is configured to infer using the corresponding trained ACS-ACT model according to the observation information to obtain the predicted future action.

[0020] The prediction action smoothing and execution module is configured to smooth the predicted future action to generate a smoothed action execution instruction, and send the smoothed action execution instruction to an execution end of the robot to control the execution end of the robot to complete the chemical experiment operation task and feed back task execution state data.

[0021] One of the above technical solutions has the following advantages and beneficial effects:

[0022] The above robot action sequence optimization method and device for a chemical experiment operation task, the method collects a corresponding expert data set for each chemical experiment operation task, quantizes the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set, and obtains quantized task features; according to the quantized task features, the size of the action block of the adaptive length of each chemical experiment operation task is determined; the fixed block size in the ACT model is replaced by the size of the action block of the adaptive length of each chemical experiment operation task, the corresponding expert data set is used to train the ACS-ACT model, and a trained ACS-ACT model corresponding to the chemical experiment operation task is obtained; the corresponding trained ACS-ACT model is used to infer according to the observation information, and a predicted future action is obtained. After optimizing the block size and the output action, the execution effect of the task is obviously improved, especially for tasks with relatively high precision and smoothness requirements, the task time consumption, success rate and action smoothness are better than those of the original ACT model. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 A flowchart of a robot action sequence optimization method for a chemical experiment operation task in an embodiment;

[0025] Figure 2 An ACT framework structure diagram in an embodiment;

[0026] Figure 3 An ACS-ACT structure diagram in an embodiment;

[0027] Figure 4 A schematic diagram of the process of grabbing an object and placing it in an embodiment;

[0028] Figure 5 A schematic diagram of the process of pipetting in an embodiment;

[0029] Figure 6 Schematic diagram of the process of opening the volumetric bottle in an embodiment;

[0030] Figure 7 Schematic diagram of the process of transferring the reagent bottle in an embodiment;

[0031] Figure 8 Schematic diagram of the process of transferring the reagent bottle in an embodiment; Figure 8 In (a), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACT optimization, Figure 8 In (b), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACS-ACT optimization;

[0032] Figure 9 Schematic diagram of the process of transferring the reagent bottle in an embodiment; Figure 9 In (a), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACT optimization, Figure 9 In (b), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACS-ACT optimization;

[0033] Figure 10 Schematic diagram of the process of transferring the reagent bottle in an embodiment; Figure 10 In (a), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACT optimization, Figure 10 In (b), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACS-ACT optimization;

[0034] Figure 11 Schematic diagram of the process of transferring the reagent bottle in an embodiment; Figure 11 In (a), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACT optimization, Figure 11 In (b), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACS-ACT optimization;

[0035] Figure 12 Schematic diagram of the process of transferring the reagent bottle in an embodiment; Figure 12 In (a), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACT optimization, Figure 12 In (b), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACS-ACT optimization, Figure 12 In (c), a schematic diagram of the smoothness of the reasoning effect and the number of task frames of the process of picking and placing objects after ACS-ACT optimization,Figure 12 Figure 9 is a comparative bar chart of the transfer reagent bottle experiment. DETAILED DESCRIPTION

[0036] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and are not intended to limit the present application.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing specific embodiments only and is not intended to be limiting of the present application.

[0038] It should be noted that the reference herein to "embodiments" means that the particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phraseology "in one embodiment" is not used to refer to the same embodiment in every instance in which the phrase is used. Rather, the phrase is used to describe a particular embodiment in one instance and a different embodiment in another instance. One of skill in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, as well as all possible combinations of the items, and includes these combinations.

[0039] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0040] In one embodiment, as shown in FIG. 1, a robot action sequence optimization method for chemical experiment operation tasks can include the following processing steps 1 to 6: Figure 1

[0041] Step 1: Collect a corresponding expert data set for each chemical experiment operation task, and quantify the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set to obtain quantified task features.

[0042] ​Specifically, in chemical experiment operation, it usually involves the scene of interaction with reagents, instruments and equipment, and many instruments are more precise, so chemical experiment is different from general mechanical operation or daily task, it may have higher risk, which requires precision and stability when performing chemical experiment, otherwise it may cause instrument damage and reagent spillage, in addition, for the ACT algorithm, the longer the duration of the entire task flow, such as heating and waiting task, the higher the timing correlation, and whether the algorithm needs to calculate the influence of historical action with higher weight needs to be adjusted according to the timing correlation. Therefore, the precision requirement, smoothness feature and task length of the task are taken as input parameters to adjust the algorithm model.

[0043] Two types of hardware environments are used, including UR G1-29edu combined with inspire RH56DFXW dexterous hand and UR6 degree of freedom industrial robot combined with flexible gripper, remote operation is used to collect chemical operation task training data set, the collected data includes robot joint state data, multi-source vision image data (including experimental equipment, reagent, sample vision information) and corresponding task execution action sequence; the training data set includes multiple data samples, each data sample contains experimental observation data s_t at time t and corresponding real action sequence a_t.

[0044] Step 2: according to the quantified task characteristics, determine the action block size of the adaptive length of each chemical experiment operation task.

[0045] Specifically, in the chemical laboratory, if multiple robots are deployed for operation and are equipped with high-performance servers for model inference, it is a very large cost for the laboratory, so it is necessary to be able to deploy the imitation learning algorithm in ordinary home computers or even small computers attached to the robot, in the ACT algorithm, the characteristics of action block and timing integration can make the model output more accurate and smooth action, but the consumption of hardware computing resources is very large, such as when the action block size is 200, before each execution action, the model needs to output the future 200 steps of action, then each step of the 200 steps of action is weighted and averaged with the predicted action in the previous 200 steps, which is too large for a home computer. In fact, in the actual machine test, if the action block size is 200, the frequency of the model output action after timing integration to the robot is only about 12 frames, which is difficult to complete for some tasks that require fast response.

[0046] Furthermore, while the ACT algorithm is a general imitation learning algorithm with applicability to various tasks, its application in chemical experiments revealed shortcomings in certain scenarios. For example, when using a pipette, the lack of a fixing device between the dexterous hand and the pipette led to task failure when the action block length was short. This was due to occasional large changes in the output action, causing pipette displacement. Conversely, if the action block was too long, the model's output frame rate decreased, and the current action was affected by the preceding actions. The impact of footwork reduces its sensitivity to changes in the current environment. For example, if external interference disrupts the target's position during mission execution, the target's sensitivity will decrease. The actions predicted by the sub-model are based on the state before the disturbance. The robot will still execute according to the previous prediction, which will cause the task to take longer or even fail.

[0047] The ACT algorithm uses fixed-size action blocks, which is not very effective for some long-running tasks or tasks requiring smoothness, compared to the original ACT model (e.g., ...). Figure 2 As shown), this application incorporates a module for adaptive action block size adjustment (such as...). Figure 3 As shown, before training the ACT model, an expert dataset is collected for each task. Then, based on these datasets, the accuracy requirements, smoothness features, and task duration of each task are quantified and input into the algorithm to train the model. During model inference, action blocks with adaptive lengths based on the task are obtained, and the size of the blocks is determined by the attributes of the task. Figure 2 The image, joint angles, and style variable Z represent observational information. n For the first n The action block is predicted in the next step; where the style variable Z is a parameter that controls the diversity of behavior. For example, a more conservative Z may generate similar things multiple times, while a more aggressive Z will be biased towards a variety of different results.

[0048] Step 3: Replace the fixed block size setting in the ACT model with the action block size of adaptive length for each chemical experiment operation task, and train the ACS-ACT model using the corresponding expert dataset to obtain the trained ACS-ACT model for each task.

[0049] Specifically, the trained ACS-ACT model is abbreviated as ACS-ACT model.

[0050] The ACT model samples the same parameters on the output action block for any operation task, which makes the algorithm have a certain universality, but for some tasks, it may not achieve the best implementation effect, so based on the accuracy, smoothness requirements and task length of each task, the optimal action block size is calculated to make the model play the maximum performance according to the properties of the task.

[0051] (1) ACT framework

[0052] The goal of imitation learning is to learn a policy that maps observations to actions, so that the learned policy can replicate the behavior and actions of an expert when executing the program.

[0053] ACT model as a kind of imitation learning, using the Transformer encoder to synthesize images from multiple perspectives, joint angle data and action style variables Z, and then using the Transformer decoder to predict the action sequence of the future step, and the output action sequence of this time is integrated with the action sequence of the previous k-1 times, and the actual output to the robot action is calculated.

[0054] ACT uses the Transformer architecture to model the temporal dependence of the action sequence, and its core component is the self-attention mechanism, which calculates the attention weight of all positions in the input sequence:

[0055] ;

[0056] where , , is the query vector derived from the input embedding, the key vector and the value vector, is and the vector dimension.

[0057] For action blocks, the input of the Transformer includes the encoded observation information and the previous predicted action, and the model uses a causal mask to ensure that the prediction of the th position can only rely on the information of the 1st to the th position, so as to maintain the property of autoregression.

[0058] The ACT model architecture solves the problem of error accumulation in single-step prediction by predicting the action of a sequence in the future, and by using the observation value at time step , ACT predicts the action of the future time steps:

[0059] ;

[0060] where denotes the learnable hyperparameters of the model, is the number of time steps in a block. During the execution of actions, the policy only uses the first action of the predicted action block and weights it with the predicted corresponding action in the previous time steps:

[0061] ;

[0062] where, is the output action of this time. After the output of this action, move to the next time step, start a new prediction of the action block from .

[0063] This block mechanism has several advantages: (1) it promotes temporal consistency by predicting a coherent sequence of actions; (2) it provides an open-loop control form, which can effectively reduce the influence of observation noise.

[0064] In the imitation learning of laboratory automation, a set of expert demonstration data set is obtained by teleoperation of unitree_g1 robot , where each data includes observation sequence , and corresponding action sequence . The observation includes image data from the binocular camera of the robot head and the total of 4 cameras of the double wrist, and the perception information from the robot body such as joint angle, speed and force sensor feedback data.

[0065] The ACS-ACT model is trained using the expert data set corresponding to each chemical experiment operation task, and the trained ACS-ACT model corresponding to each chemical experiment operation is obtained.

[0066] Step 4: According to the observation information, the corresponding trained ACS-ACT model is used for reasoning to obtain the predicted future action.

[0067] Step 5: Smooth the predicted future action to generate a smooth action execution instruction.

[0068] Step 6: The smooth action execution instruction is issued to the execution end of the robot to control the robot execution end to complete the chemical experiment operation task, and the task execution state data is fed back.

[0069] The robot action sequence optimization method for the chemical experiment operation task described above, the method collects a corresponding expert data set for each chemical experiment operation task respectively, quantizes the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set, and obtains the quantized task feature; according to the quantized task feature, the adaptive length of each chemical experiment operation task is determined. The size of the action block; replace the fixed block size setting in the ACT model with the adaptive length of each chemical experiment operation task, train the ACS-ACT model using the corresponding expert data set, and obtain the trained ACS-ACT model corresponding to the chemical experiment operation task; according to the observation information, the corresponding trained ACS-ACT model is used for reasoning, and the predicted future action is obtained. After optimizing the block size and output action in this method, the execution effect of the task is obviously improved, especially for tasks with relatively high precision and smoothness requirements, the task time consumption, success rate and action smoothness are better than the original ACT model.

[0070] In one embodiment, step 1 includes: collecting a corresponding expert data set for each chemical experiment operation task respectively; according to the expert data set corresponding to each chemical experiment operation task, determining the acceptable error tolerance, the variance of the action acceleration, the total number of data frames during the task, and the frame rate during data acquisition and model inference of each chemical experiment operation task; the reciprocal of the acceptable error tolerance of each chemical experiment operation task is taken as the precision requirement of the corresponding chemical experiment operation task; the reciprocal of the variance of the action acceleration of each chemical experiment operation task is taken as the smoothness feature of the corresponding chemical experiment operation task; the quotient of the total number of data frames during the task of each chemical experiment operation task and the frame rate during data acquisition and model inference is taken as the duration of the corresponding chemical experiment operation task.

[0071] Specifically, in chemical experiment operation, it usually involves scenarios of interacting with reagents, instruments and equipment, and many instruments are relatively precise. Therefore, chemical experiments are different from general mechanical operations or daily tasks, and may have higher risks, which requires precision and stability when performing chemical experiments, otherwise there is a probability of causing instrument damage and reagent spillage. In addition, for the ACT algorithm, the longer the duration of the entire task flow, such as heating and waiting tasks, the higher the timing correlation, and whether the algorithm needs to calculate the influence of historical actions with higher weight needs to be adjusted according to the timing correlation. Therefore, the precision requirement, smoothness feature and task duration of the task are taken as input parameters to adjust the algorithm model in this embodiment.

[0072] (1) Precision requirement

[0073] For different chemical operation tasks in the chemical laboratory, the requirements for action accuracy are also different. Higher accuracy requires smaller action blocks to maintain fine-grained control, and smaller action blocks are also needed to ensure that the robot responds more sensitively, reduce the computational cost of a single action sequence, and output action sequences faster. This inverse relationship ensures that as the accuracy requirement increases, the block size decreases.

[0074] In general understanding, accuracy requirements are related to errors, and the greater the acceptable error, the lower the accuracy requirement. Therefore, the accuracy requirement is defined as the inverse of the acceptable error tolerance of the operation task, and the accuracy requirement calculation formula is:

[0075] ;

[0076] wherein, is the accuracy requirement, is the maximum acceptable error of the task.

[0077] (2) Smoothness feature

[0078] In chemical experiment operation, some tasks involve the interaction between the robot and the chemical reagent. In order to avoid spilling of the reagent, the higher the smoothness requirement, the higher the smoothness of the robot action sequence, which means that the action sequence has a higher correlation in time, and is more suitable for large action blocks. For actions with low smoothness, if the action block is too large, it may result in a large error in the predicted action sequence.

[0079] For the action of the robot, the acceleration reflects the speed of the action change, and the variance can effectively capture whether there is a sudden and discontinuous change in the action. Therefore, the variance of the action acceleration is used to quantify the smoothness of the task. The smoothness feature calculation formula is:

[0080] ;

[0081] wherein, is the smoothness feature, represents the action acceleration, is the average value of the action acceleration.

[0082] (3) Task duration

[0083] In long-time tasks, larger action blocks can make the action sequences before and after the task more correlated. The longer the action, the greater the error accumulation when predicting. Since the current step action output by the ACT model is a weighted average of the previous prediction values, the larger the action block means that if there is an error in this prediction, the impact of this error will be smaller.

[0084] ​The duration of the whole task is from the beginning of the task to the end of the task, since the embodiment uses teleoperation to collect action data, the task duration is defined as the duration from the beginning of teleoperation to the end of teleoperation; the task duration calculation formula is:

[0085] ;

[0086] wherein, is the task duration, is the total number of data frames during the task, is the frame rate during data collection and model inference, which is set to 30 on a humanoid robot and 20 on a single-arm industrial robot.

[0087] For small home PCs or robot on-board PCs, their performance is often not very powerful, so the calculation speed of the inference model is not fast, therefore, the embodiment sets the upper limit of the size of the model output action block to 200, and when the limit is exceeded, the model calculation speed is too slow, so that the frequency of the output action cannot reach the control frequency of the robot, at this time, since the frequency of the action and other experimental results are inconsistent, the calculated action smoothness and other parameters do not have reference value.

[0088] In one embodiment, step 2 comprises: determining the task-aware adaptive length action block size according to the quantified task characteristics:

[0089] ;

[0090] wherein, is the task-aware adaptive length action block size, is the smoothness feature of the task, is the accuracy requirement of the task, is the task duration, is the preset upper limit value of the size of the model output action block, is the ceiling function, is the minimum value function.

[0091] Specifically, for a specific expert dataset of a chemical experiment procedure, the goal of the embodiment is to determine the optimal block size based on the inherent task characteristics to improve the applicability of the algorithm in a specific scenario. The present application proposes a task-aware adaptive block size optimization framework (as shown in Figure 3 ), which calculates the block size according to three basic task attributes:

[0092] ;

[0093] wherein, is the accuracy requirement of the task, For the smoothness characteristics of the task, The duration of the task.

[0094] The size of the action block is directly proportional to the smoothness and duration of the task, and inversely proportional to the accuracy requirement. It must be an integer; therefore, the formula for calculating the block size is defined as follows:

[0095] ;

[0096] Based on the task category and actual operational conditions, the accuracy requirements for each task were calculated. Smoothness features and task duration This includes three chemical operations, covering most of the accuracy requirements, smoothness characteristics, and task duration in chemical operations. In addition, to verify the effectiveness of the algorithm improvement in general fields, this embodiment adds an experimental task of grasping and releasing objects.

[0097] In one embodiment, step 3 includes: replacing the fixed block size setting in the ACT model with an action block size of adaptive length for each chemical experiment task; training the ACS-ACT model using supervised learning with an expert dataset corresponding to the chemical experiment task; the training process employs the standard gradient descent optimization method; iteratively updating network parameters through the backpropagation algorithm; determining the loss function of the ACT model based on the predicted future actions and the actions in the expert dataset; and stopping training when the loss function reaches a preset value, thereby obtaining the trained ACS-ACT model corresponding to the chemical experiment task.

[0098] In one embodiment, the loss function expression for the ACT model is:

[0099] ;

[0100] in, The loss function for the ACT model is... Represents the learnable hyperparameters of the model. For task duration, Indicates from time The initial real action block, For expert datasets, These represent the trajectories and action sequences in the dataset, respectively. Indicates the model in observation Predicting the future The action at each time step.

[0101] Specifically, the ACT model is trained using supervised learning, and the goal of the algorithm is to minimize the difference between the model's predicted actions and the expert dataset's action demonstrations in the real world. By constructing a prediction model based on the Transformer architecture, the algorithm learns the mapping from the observation state to the action sequence. The training process uses the standard gradient descent optimization method, iteratively updates the network parameters through the backpropagation algorithm, and then calculates the mean square error between the model's predicted actions and the expert dataset's actions to determine the difference between the two. The loss function is shown in the loss function expression of the ACT model above.

[0102] In an ideal case, when the training is fully conducted, the loss function tends to be minimized, at which point the model's predicted action distribution approximates the true distribution of expert actions. This convergence ensures that the ACT model can effectively learn a robust control strategy from limited expert demonstrations, achieving reliable mapping from observations to actions.

[0103] In a verification embodiment, in order to enable operation in a chemical experiment scenario, two different hardware devices were tested to test the optimization effect of the algorithm on robots with different degrees of freedom performing tasks. These include a 29-degree-of-freedom humanoid robot equipped with two 6-degree-of-freedom dexterous hands, and a 7-degree-of-freedom industrial robot (as shown in Table 1).

[0104] Table 1 Hardware parameters

[0105]

[0106] The experimental hardware platform of the Yushu G1 robot uses Yushu G1-29edu with inspire RH56DFXW dexterous hands. Among them, the 14 degrees of freedom of the Yushu G1 robot and the 12 degrees of freedom of the dexterous hand are used, a total of 26 degrees of freedom of the dual-arm platform to ensure that the operation can be realized without being limited by the range of hand joints. In terms of vision, it includes a head binocular camera and two wrist cameras, both with a resolution of 480 640. When the camera resolution is too high, due to limited computing resources, this results in a significant decrease in the inference speed of the model. At the current resolution, if the block size is set to 200 and the timing integration is turned on, the model inference frequency is less than 15 frames, so the algorithm is not suitable for too large a camera resolution.

[0107] In the industrial robot platform, a 6-degree-of-freedom robot arm of UR robot is used. In chemical experiments, most instruments such as beakers are made of glass or plastic, and it is difficult to control the force of rigid grippers, which can easily damage the instruments. Therefore, a flexible gripper is used as the end execution device of the robot. In terms of vision, this embodiment uses two fixed-position cameras and one camera that moves with the robot, all with a resolution of 480 640 resolution.

[0108] (1) Task design

[0109] To verify the effect of the algorithm improvement, the embodiment tests the operations for the chemical laboratory: reagent transfer, pipetting and opening the volumetric flask, which covers different requirements for accuracy, task duration and smoothness in chemical operations, such as the operation of pipetting, which has higher requirements for the accuracy of robot action and longer task duration, while the accuracy requirement for opening the volumetric flask and transferring the reagent bottle is relatively low, and the task duration is shorter. At the same time, these operations also cover robots with different degrees of freedom to test their adaptability to the degrees of freedom of the robot. In the operation of transferring the reagent bottle, a 7-degree-of-freedom single-arm robot is used, while in the operations of pipetting and opening the volumetric flask, a 26-degree-of-freedom dual-arm humanoid robot is used. In addition, a group of more general operations: picking up and placing objects are added to verify the effect of the algorithm improvement in the general field.

[0110] In the comparative experiment of the same task, in order to control the variables, the parameters of the model training remain the same except the block size, and in addition, the loss of the model is controlled at about 0.05.

[0111] Figure 4 The process of picking up and placing objects is shown: (1) use the right hand to pick up the red and white box from the right lower corner of the table (the box is placed in the black and white boundary line at the right lower corner of the table); (2) hand it over to the left hand; (3) put it into the designated position marked on the table (i.e. the lower left part of the cross symbol on the white paper pasted on the table).

[0112] Figure 5 The pipetting process is shown: (1) first take the pipette off the shelf where the pipette is placed (use the index finger to hook the pipette); (2) press the button of the pipette; (3) put it into the solution in the beaker; (4) release the button of the pipette to suck the solution; (5) move the pipette to another beaker; (6) press the button of the pipette to squeeze out the solution, completing a whole pipetting operation.

[0113] Figure 6 The process of randomly placing the volumetric flask on the left black area of the table is shown: (1) the robot first grabs the neck of the volumetric flask with the left hand; (2) pinches the stopper of the volumetric flask with the fingers of the right hand; (3) takes off the stopper; (4) places it on the table at the center of the cross mark.

[0114] Figure 7The reagent bottles are randomly placed within a circular range of 5 cm radius on the shelf, and the placement process is as follows: (1) the mechanical arm first clamps the first reagent bottle on the shelf; (2) places it on another shelf (the placement position is random); (3) removes the bottle cap after placing it; (4) places it to the right of the reagent bottle.

[0115] (3) Data collection

[0116] In the expert operation data collection stage, for the humanoid robot, the embodiment uses a VR glasses to capture the position and posture information of the human hand, and then calculates the position information into the joint angles of the arm and fingers through repositioning, and maps the joint angles to the dexterous hand. For each operation task, 100-200 trajectories are collected, and for some test results of tasks that are not ideal, the data set is increased to more than 200. The data set includes image data of 4 RGB cameras and 26-dimensional joint angle data. The master-slave arm form is used on the industrial robot to perform remote operation, and a small mechanical arm composed of servos is controlled by manual dragging to map its joint angle data to the slave arm, thereby achieving control of the motion of the industrial robot.

[0117] For the parameters required by the task accuracy, the embodiment tests the error acceptance range of each task, and if it exceeds this range, the robot will not be able to grasp the object. The smoothness feature of the task is obtained by calculating the joint angle data in the remote operation data set, such as 900 frames of joint angle data. The task duration is calculated by the average frame number in the data set according to the task duration formula, and the feature quantization of each task is shown in Table 2.

[0118] Table 2 Task feature quantization

[0119]

[0120] (4) Experimental results

[0121] The embodiment simultaneously trains different models for each task using ACT and ACS-ACT algorithms based on the same data set, compares and tests the success rate, motion smoothness and time efficiency of the tasks, and records the success rate, motion smoothness and time efficiency of the tasks to compare the effects of the two algorithms in the chemical experiment scene.

[0122] 1) Task success rate

[0123] Task success rate is one of the most basic indicators to evaluate a robot control system, especially in practical operation, the success or failure of the task directly determines the practicability and reliability of the system. In chemical experiments, the tasks of the robot not only include the completion of a single action, but also involve a series of complex and precise operation steps, for example, in the tasks of "pipetting" and "opening the volumetric flask", the robot is required to accurately complete multiple actions, and there is a strict time and space dependency between each action. The success rate is compared in Table 3.

[0124] For each task and algorithm, the model is trained for inference, and the success rate of the output action in executing the task is tested. The evaluation standard for task success is defined as being able to complete the entire task flow and achieve the task goal. The success criteria for each task are as follows:

[0125] Object grasping and placing: the box needs to be grasped with the right hand first and then handed over to the left hand, and then placed on the specified position of the table by the left hand.

[0126] Pipetting: the robot takes the pipette from the shelf and successfully uses the pipette to pipette from beaker A to beaker B (the reagent volume has been set in the pipette in advance).

[0127] Opening the volumetric flask: the robot successfully removes the cap of the volumetric flask and places the cap and the volumetric flask stably on the specified position of the table.

[0128] Transferring reagent bottles: the robot clamps the reagent bottle from the shelf and places it at the specified position, and successfully removes the cap and places it next to the reagent bottle.

[0129] Table 3 success rate comparison

[0130]

[0131] As in the pipetting operation, the main reasons for task failure are mainly concentrated on the pipette falling off the shelf to the fingertips, the robot cannot correctly press the pipette button when sucking and expelling liquid, and sometimes it will even fall from the hand. After the algorithm is improved, the smoothness of the model output action is improved, the hand tremor of the robot is significantly improved, so that the task can be completed with a higher success rate. Although the hand tremor of the robot is not so sensitive in other tasks, it can also benefit from it, thereby improving the success rate by more than 10%.

[0132] (2) Action smoothness and time efficiency

[0133] In the chemical experiment task, the smoothness of the robot's action is crucial to ensure the success of the task. Especially in the operation involving fragile instruments and reagent transfer, if the robot's action is too violent or not smooth, it may cause problems such as reagent overflow, container breakage, etc. In the action execution stage of the ACS-ACT model, the output action is smoothed to prevent hardware damage caused by excessive changes in the robot's action.

[0134] In the four tasks, the action smoothness of the ACS-ACT model is improved by 12% to 25% compared to the ACT model, and the probability of robot arm action twitching is greatly reduced. In order to more intuitively represent the change of the robot's action, the recorded state data of the robot's joint angle is calculated by second-order difference to obtain its joint angle acceleration, and it is plotted as a line graph. It can be seen in Figures 8 to 11 compared to the action of ACT, the action acceleration of ACS-ACT model has much less peak, and the amplitude of action change is relatively more gentle, which is very important for the stable operation in chemical experiments. In the actual operation of the robot, when using the ACT algorithm to perform the pipetting operation, the pipette often slips from the hand due to finger joint twitching, resulting in task failure. For the tasks of object grasping and placing, opening the volumetric bottle and transferring the reagent bottle, since the action smoothness is not very sensitive, sometimes the placement position is slightly deviated due to the problem of twitching, but the overall impact is not big. Especially in the task of transferring the reagent bottle, since the robot has only 7 degrees of freedom, the inference effect of the model is much better. Whether it is the algorithm before or after improvement, the acceleration fluctuation of the output action sequence has almost no peak. And in terms of overall smoothness, ACS-ACT has improved by 12% to 25% compared to the original algorithm. Figure 8 Figure (a) in the Figure 8 Figure (b) in the Figure 9 Figure (a) in the Figure 9 Figure (b) in the Figure 10 Figure (a) in the Figure 10 Figure (b) in the Figure 11Fig. 6 is a diagram showing the comparison between the smoothness of the action and the number of task frames of the transfer reagent bottle process reasoning effect optimized by ACT, Figure 11 Fig. 7 is a diagram showing the comparison between the smoothness of the action and the number of task frames of the transfer reagent bottle process reasoning effect optimized by ACS-ACT.

[0135] In the process of the model outputting actions to perform tasks, such as grabbing an object, the model output action fails to successfully grab, and then the subsequent output action needs to be adjusted to re-grab the object. In the actual task execution process, such errors are obviously less the better, because it means that if it can be successful at one time, it can greatly reduce the time required to perform the task. Through the evaluation of the time efficiency index, the balanced performance of the algorithm in processing different tasks can be quantified, and whether it can optimize the total time of task execution without affecting the accuracy is verified.

[0136] In each task execution process, the embodiment records the frame number from the start of the model output action to the end of each successful task completion, and converts it into the time required for the entire task to judge the time efficiency of different algorithms in executing tasks. The comparison of the four groups of experiments is shown in Fig. 8, wherein, Figure 12 Figure 12 Fig. 8(a) is a comparison bar chart of the object grabbing and placing experiment, Figure 12 Fig. 8(b) is a comparison bar chart of the pipetting experiment, Figure 12 Fig. 8(c) is a comparison bar chart of the opening of the volumetric flask experiment, Figure 12 Fig. 8(d) is a comparison bar chart of the transfer reagent bottle experiment. Figure 12 The blue column in Fig. 8 represents the smoothness of the action, and the orange column represents the frame number of the action output by the model during the execution of the entire task.

[0137] Compared with the original ACT algorithm, the ACS-ACT model controlled robot action has higher smoothness, and the time required to complete the task is relatively shorter, especially in some tasks that are sensitive to jitter, such as in the operation of pipetting, the jitter of the robot arm may cause the pipette to fall or not to hold the pipette correctly, which makes the model need to keep trying, so as to greatly increase the time required for the task, and in some cases it may take more than 3000 frames to complete the task (generally about 1500 frames to complete the task under normal execution conditions), the average time required for the improved algorithm to execute the task is only 66.75% of the original algorithm. In other tasks that do not require high smoothness, the average time required for the task is generally less than 85% of the original algorithm.

[0138] ​From the overall experimental results, the ACS-ACT (Adaptive Chunk Size ACT) algorithm proposed in the present application performs better than the original ACT algorithm in some chemical experiment operation tasks, which is reflected in the improvement of success rate, smoother motion and certain shortening of time required to complete the same task. The original ACT algorithm performs poorly in some scenarios, and due to the more jitter of output motion, it will frequently make mistakes in some tasks with high precision requirements. After optimization of the block size and output motion, the task execution effect is obviously improved, especially for tasks with relatively high precision and smoothness requirements, the task time consumption, success rate and motion smoothness are better than ACT, such as the time consumption is 66.75% of the original algorithm, and the success rate is improved by 26%. For these chemical experiment tasks with low parameter requirements, the task time consumption is also lower than 85% of the original algorithm, the motion smoothness is improved by 12% to 25%, and the task success rate is improved by 10% to 20%.

[0139] It should be understood that although the above Figure 1 steps are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the above Figure 1 steps can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately executed with other steps or at least part of the sub-steps or stages of other steps.

[0140] In one embodiment, an ACS-ACT optimization device for chemical experiment operation tasks is also provided, comprising:

[0141] A task feature quantization module is configured to collect a corresponding expert data set for each chemical experiment operation task, and quantize the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set to obtain quantized task features.

[0142] An adaptive action block size adjustment module is configured to determine the adaptive length of the action block size of each chemical experiment operation task according to the quantized task features.

[0143] The ACT model training module is configured to replace a fixed block size setting in the ACT model with an action block size of an adaptive length for each chemical experiment operation task, train the ACS-ACT model using a corresponding expert data set, and obtain a trained ACS-ACT model corresponding to the chemical experiment operation task.

[0144] The inference module is configured to perform inference using the corresponding trained ACS-ACT model according to observation information, and obtain a predicted future action.

[0145] The predicted action smoothing and execution module is configured to perform smoothing processing on the predicted future action, generate a smoothed action execution instruction, and send the smoothed action execution instruction to an execution end of the robot to control the execution end of the robot to complete the chemical experiment operation task and feed back task execution state data.

[0146] In an embodiment, the task feature quantification module is further configured to collect a corresponding expert data set for each chemical experiment operation task, determine an acceptable error tolerance, an action acceleration variance, a total number of data frames during the task, and a frame rate during data collection and model inference for each chemical experiment operation task according to the corresponding expert data set, take the reciprocal of the acceptable error tolerance of each chemical experiment operation task as a precision requirement of the corresponding chemical experiment operation task, take the reciprocal of the action acceleration variance of each chemical experiment operation task as a smoothness feature of the corresponding chemical experiment operation task, and take the quotient of the total number of data frames during the task and the frame rate during data collection and model inference for each chemical experiment operation task as a task duration of the corresponding chemical experiment operation task.

[0147] In an embodiment, the adaptive action block size adjustment module is further configured to determine the action block size of the adaptive length based on task perception according to the quantified task features.

[0148] ;

[0149] wherein, the action block size of the adaptive length based on task perception, the smoothness feature of the task, the precision requirement of the task, the task duration, the preset upper limit value of the size of the model output action block, the ceiling function, the minimum function.

[0150] In one embodiment, the ACT model training module is further configured to replace the fixed block size setting in the ACT model with an action block size of adaptive length for each chemical experiment task, and to train the ACS-ACT model using supervised learning with an expert dataset corresponding to the chemical experiment task. The training process employs the standard gradient descent optimization method, iteratively updates the network parameters through the backpropagation algorithm, and determines the loss function of the ACT model based on the predicted future actions and the actions in the expert dataset. When the loss function reaches a preset value, training is stopped, and the trained ACS-ACT model corresponding to the chemical experiment task is obtained.

[0151] In one embodiment, the loss function of the ACT model in the ACT model training module is:

[0152] ;

[0153] in, The loss function for the ACT model is... Represents the learnable hyperparameters of the model. For task duration, Indicates from time The initial real action block, For expert datasets, These represent the trajectories and action sequences in the dataset, respectively. Indicates the model in observation Predicting the future The actions at a given time. It is understood that for a detailed explanation of the ACS-ACT optimization device for chemical experiment tasks, please refer to the corresponding explanations of the various embodiments of the robot action sequence optimization method for chemical experiment tasks described above, which will not be repeated here. Each module in the aforementioned ACS-ACT optimization device for chemical experiment tasks can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in hardware or independently of a device with data processing capabilities, or stored in software in the memory of the aforementioned device, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of data processing computer devices already existing in the art.

[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the protection scope of the present application. It should be pointed out that, for ordinary skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and all belong to the protection scope of the present application.

Claims

1. A method for robot motion sequence optimization for chemical experiment operation tasks, characterized in that, The method comprises the steps of: Step 1: Collecting a corresponding expert data set for each chemical experiment operation task, and quantifying the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set to obtain quantified task features; Step 2: According to the quantified task features, determine the adaptive length of the action block size of each chemical experiment operation task; Step 3: Replace the fixed block size setting in the ACT model with the adaptive length of the action block size of each chemical experiment operation task, train the ACS-ACT model using the corresponding expert data set to obtain the trained ACS-ACT model corresponding to each chemical experiment operation task; Step 4: According to the observation information, use the corresponding trained ACS-ACT model to infer to obtain the predicted future action; Step 5: Smooth the predicted future action to generate a smooth action execution instruction; Step 6: The smooth action execution instruction is sent to the execution end of the robot to control the robot execution end to complete the chemical experiment operation task, and the task execution state data is fed back.

2. The method for robot motion sequence optimization for chemical experiment operation tasks according to claim 1, characterized in that, Step 1 comprises: Collecting a corresponding expert data set for each chemical experiment operation task; According to the expert data set corresponding to each chemical experiment operation task, determine the acceptable error tolerance, action acceleration variance, total data frame number during the task, and frame rate during data acquisition and model inference of each chemical experiment operation task; Take the reciprocal of the acceptable error tolerance of each chemical experiment operation task as the precision requirement of the corresponding chemical experiment operation task; Take the reciprocal of the action acceleration variance of each chemical experiment operation task as the smoothness feature of the corresponding chemical experiment operation task; Take the quotient of the total data frame number during the task and the frame rate during data acquisition and model inference of each chemical experiment operation task as the duration of the corresponding chemical experiment operation task. 3.The robot motion sequence optimization method for a chemical experiment operation task according to claim 1, wherein, Step 2 comprises: according to the quantified task features, determine the adaptive length of the action block size based on the task perception as: ； wherein, is a task-aware adaptive length of action block size, is a smoothness feature of the task, is a precision requirement of the task, is a task duration, is a pre-set upper limit value of the size of the model output action block, is a ceiling function, is a minimum function.

4. The method for robot action sequence optimization for chemical laboratory operation tasks according to claim 1, wherein, Step 3 comprises: Replace the fixed block size setting in the ACT model with the adaptive length of the action block size of each chemical experiment operation task, and train the ACS-ACT model using the expert data set of the corresponding chemical experiment operation task using supervised learning. The training process uses a standard gradient descent optimization method, iteratively updates the network parameters through a backpropagation algorithm, and determines the loss function of the ACS-ACT model according to the predicted future action and the expert data set action. When the loss function reaches a preset value, stop training to obtain the trained ACS-ACT model corresponding to the chemical experiment operation task.

5. The method for robot action sequence optimization for chemical laboratory operation tasks according to claim 4, characterized in that, The loss function of the ACT model is: ； in, The loss function for the ACT model is... Represents the learnable hyperparameters of the model. For task duration, Indicates from time The initial real action block, For expert datasets, These represent the trajectories and action sequences in the dataset, respectively. Indicates the model in observation Predicting the future The action at each time step.

6. An ACS-ACT optimization apparatus for chemical laboratory operation tasks, characterized by, It comprises: A task feature quantification module for collecting a corresponding expert data set for each chemical experiment operation task, and quantifying the precision requirement, smoothness feature and task duration of the corresponding chemical experiment operation task according to the expert data set to obtain quantified task features; An adaptive action block size adjustment module for determining the adaptive length of the action block size of each chemical experiment operation task according to the quantified task features; An adaptive action block size adjustment module for determining the adaptive length of the action block size of each chemical experiment operation task according to the quantified task features; The ACT model training module is configured to replace a fixed block size setting in the ACT model with an adaptive length action block size for each chemical experiment operation task, train the ACS-ACT model using a corresponding expert data set, and obtain a trained ACS-ACT model corresponding to the chemical experiment operation task. The inference module is configured to perform inference using the corresponding trained ACS-ACT model according to observation information, and obtain predicted future actions. The predicted action smoothing and execution module is configured to perform smoothing processing on the predicted future actions, generate a smoothed action execution instruction, and send the smoothed action execution instruction to a robot execution end to control the robot execution end to complete a chemical experiment operation task and feed back task execution state data.

7. ACS-ACT optimization apparatus for chemical laboratory operation tasks according to claim 6, characterized in that The task feature quantization module is further configured to collect a corresponding expert data set for each chemical experiment operation task, determine an acceptable error tolerance, an action acceleration variance, a total number of data frames during the task, and a frame rate during data collection and model inference for each chemical experiment operation task, use the reciprocal of the acceptable error tolerance for each chemical experiment operation task as a precision requirement for the corresponding chemical experiment operation task, use the reciprocal of the action acceleration variance for each chemical experiment operation task as a smoothness feature for the corresponding chemical experiment operation task, and use the quotient of the total number of data frames during the task and the frame rate during data collection and model inference for each chemical experiment operation task as a duration of the corresponding chemical experiment operation task.

8. The ACS-ACT optimization apparatus for chemical laboratory operation tasks according to claim 6, characterized in that, The adaptive action block size adjustment module is further configured to determine a task-aware adaptive length action block size based on the quantified task features. ； wherein, is a task-aware adaptive length of action block size, is a smoothness feature of the task, is a precision requirement of the task, is a task duration, is a pre-set upper limit value of the size of the model output action block, is a ceiling function, is a minimum function.

9. The ACS-ACT optimization apparatus for chemical laboratory operation tasks according to claim 6, characterized in that, The ACT model training module is further configured to replace a fixed block size setting in the ACT model with an adaptive length action block size for each chemical experiment operation task, train the ACS-ACT model using an expert data set corresponding to the chemical experiment operation task using supervised learning, use a standard gradient descent optimization method in the training process, iteratively update network parameters through a back propagation algorithm, determine a loss function of the ACT model according to the predicted future actions and the expert data set actions, and stop training when the loss function reaches a preset value to obtain a trained ACS-ACT model corresponding to the chemical experiment operation task.

10. The ACS-ACT optimization apparatus for chemical laboratory operation tasks according to claim 9, characterized in that, The loss function of the ACT model in the ACT model training module is: ； in, The loss function for the ACT model is... Represents the learnable hyperparameters of the model. For task duration, Indicates from time The initial real action block, For expert datasets, These represent the trajectories and action sequences in the dataset, respectively. Indicates the model in observation Predicting the future The action at each time step.

Images