Robot control strategy migration method, system and device, medium and program product

Abstract

The invention provides a robot control strategy migration method and device, a medium and a program product, and the method comprises the steps: collecting a state-action sequence, generated by a pre-training strategy model, of a robot in a real operation environment, and carrying out the preprocessing, and obtaining structured training data; judging whether a preset updating condition is met or not based on the running state, and when the preset updating condition is met, performing fine adjustment on the strategy model by utilizing the training data to generate an updated strategy model; thermally deploying the updated strategy model to a real operation environment of the robot, so that the robot operates based on the updated strategy model and generates new operation data under the condition of not interrupting task execution; the three stages of data collection, preprocessing, strategy fine adjustment and model deployment are executed circularly, data interaction and flow decoupling between the stages are achieved through a message queue, and continuous optimization based on real operation feedback is achieved. According to the method, the stability and generalization ability of strategy migration are improved through condition-triggered cyclic fine tuning and a hot deployment mechanism of uninterrupted operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a robot control strategy migration method, device, medium and program product. BACKGROUND

[0002] In the field of robot control, a strategy determines the actions performed by a robot in different operating states, such as speed adjustment, joint angle change, trajectory tracking, etc. A strategy model is usually optimized and trained in a static simulation environment, and after training is completed, it is directly migrated to a real environment to perform a task. However, there are complex factors in a real environment, such as external disturbances, diversified terrain, changing contact conditions, and differences in friction coefficients, which are often difficult to accurately model in a simulation environment. Due to the lack of adaptive optimization mechanisms for real physical properties, existing migrated strategies often have decreased control accuracy, increased motion trajectory deviation, or even failure when deployed in reality.

[0003] In the prior art, strategy updating usually adopts an offline batch processing method, that is, after simulation training is completed, a fixed strategy model is deployed to a real environment, and after running for a period of time, running data is collected manually, and then offline retraining and complete replacement are performed. This method has the following disadvantages: first, it lacks real-time adaptability, and the strategy cannot be adjusted in real time in response to immediate physical feedback during operation, resulting in a poor response to dynamic disturbances and non-ideal factors, and poor running stability; second, it lacks a closed-loop optimization mechanism, and the strategy issuing process is a one-way flow, and cannot realize automatic closed-loop linkage of data collection, model updating and strategy deployment based on real machine running data.

[0004] Therefore, in a simulation-to-reality (Sim2Real) migration task with significant differences in physical properties, there is an urgent need for a technical solution that can perform online closed-loop strategy optimization in combination with real running data. SUMMARY

[0005] In view of the deficiencies in the prior art, the present application provides a robot control strategy migration method, device, medium and program product, at least to realize online closed-loop optimization based on real running data to improve the adaptability and stability of the strategy in a real environment.

[0006] To achieve the above-mentioned purposes and other advantages, some embodiments of the present application provide the following aspects: In a first aspect, some embodiments of the present application provide a robot control strategy migration method, comprising: Collecting running data of a robot in a real running environment, the running data being composed of state-action sequences generated by a pre-trained strategy model when running in a real physical environment; Preprocessing the operation data to obtain structured training data including the state-action sequence; Determining whether a preset update condition is satisfied based on the running state of the structured training data, and when the preset update condition is satisfied, fine-tuning the strategy model using the structured training data as an expert strategy to generate an updated strategy model; Hot deploying the updated strategy model to the real operating environment of the robot, so that the robot can run based on the updated strategy model without interrupting task execution and generate new operating data; The new operating data is collected again, and the steps of collecting operating data, preprocessing operating data, fine-tuning the training strategy model, and deploying and updating the strategy model are executed cyclically to drive continuous optimization of the strategy model using real operating feedback. During the cyclic execution, the three stages of data collection, strategy training, and model hot deployment are executed in parallel, and data interaction and process decoupling between the stages are achieved through message queues.

[0007] In a second aspect, some embodiments of the present application further provide an electronic device, comprising: One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the robot control strategy migration method as described in any one of the above.

[0008] On the third aspect, some embodiments of the present application also provide a computer-readable storage medium on which a computer program and / or instructions are stored, and when the computer program and / or instructions are executed by a processor, a robot control strategy migration method as described in any one of the above is implemented.

[0009] In a fourth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program and / or instructions, which, when executed by a processor, implements a robot control strategy migration method as described in any one of the above.

[0010] Compared with related technologies, the solution provided in the embodiments of the present application uses the state-action sequence generated by the pre-trained policy model in the real operating environment as structured training data, performs condition-triggered cyclic fine-tuning on the policy model, and combines it with hot deployment of the model, so that the policy model can be updated without interrupting task execution. In the process of cyclic execution of data collection, data preprocessing, policy fine-tuning and model deployment, the three stages are parallelized and data interaction and process decoupling are carried out through message queues, thereby utilizing real operation feedback to achieve continuous optimization of the policy model, improve the stability and generalization ability of policy migration, and reduce the risk of performance degradation of traditional migration methods in actual deployment. Experimental results show that this method can reduce the overall migration error of traditional migration strategies in real deployment from approximately 10% to 20% to less than 5%, achieving more stable control performance and higher task execution reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 This is one of the flow charts of a robot control strategy migration method provided in an embodiment of the present application; Figure 2 This is the second flow chart of a robot control strategy migration method provided in an embodiment of the present application; Figure 3 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] The embodiment of the application relates to a robot control strategy migration method, which runs in a distributed computing and communication architecture, wherein an edge computing node is deployed at a robot end for executing data preprocessing, running data caching and inference of a lightweight strategy model; a high-performance GPU server is deployed at a cloud end for executing training and large-scale fine-tuning of the strategy model. The robot end and the cloud end server communicate with each other through a wireless network or a wired network to support real-time uploading of running data and downloading of updated strategy models.

[0015] In an embodiment, an edge computing architecture can be used to complete the collection, preprocessing and caching of running data at the robot end, and then batch upload the running data to the cloud end server for training, thereby reducing communication delay and reducing network transmission load.

[0016] In another embodiment, a hybrid cloud architecture can be used to deploy a fine-tuning training task of a lightweight strategy model at the robot end for execution, and deploy a training or complex fine-tuning process of a full-amount strategy model at the cloud end GPU server for execution, thereby improving the utilization of computing resources while ensuring the efficiency of model updating. Referring to FIG. 1, the method can include the following steps: Figure 1 、 Figure 2 Step S1: Collecting running data of a robot in a real running environment, the running data being a state-action sequence generated by a pre-trained strategy model running in a real physical environment.

[0017] For step S1, specifically, a pre-trained strategy model obtained by simulation training is deployed in a real running environment of a robot, and running data thereof is continuously collected during task execution. The running data includes a control command generated by the strategy model at a current state, and action state data collected by a robot sensor, which constitute a state-action sequence. The action state data can be obtained by an inertial measurement unit (IMU), a magnetic encoder and the like, and covers dynamic parameters such as position, speed and acceleration, for reflecting motion characteristics of the robot in the actual physical environment; the control command can include joint driving instructions, end effector instructions and the like, for instructing the robot to perform corresponding actions. Through this step, the input and output corresponding relationship of the strategy model can be accurately recorded in the real physical environment, thereby providing a reliable data basis for subsequent strategy optimization based on real data.

[0018] Step S2: Preprocessing the running data to obtain structured training data containing state-action sequences.

[0019] In this embodiment, step S2 specifically includes: Step S201: Batch caching the running data based on a preset sampling period.​

[0020] In this step, the robot continuously collects motion state data (position, velocity, acceleration) and corresponding control commands from its sensors (such as the inertial measurement unit (IMU) and magnetic encoder) during operation, using a preset sampling period (e.g., 5 minutes) as a time window. The caching process uses a time window aggregation approach, allocating a separate window buffer within the acquisition thread for each time window. Raw sampled data from different sensor channels is written in chronological order, indexed by timestamps. When the window deadline or the upper limit of the sampling number is reached, the window is closed. Late data within the allowed time drift range is merged into the current window; otherwise, data is rolled into the next window and marked.

[0021] Due to the high sampling frequency of sensors (e.g., 100Hz to 1kHz), directly writing or transmitting data to disk can introduce file I / O latency or network congestion, leading to acquisition thread stalls or even data loss. To improve acquisition and processing efficiency, sensor data is first written to a circular buffer structure in memory, where it is then merged and aligned by timestamp by the window manager to form window data blocks. The circular buffer structure employs a double-buffering strategy: while window A is being acquired, the closed window B is processed by a preprocessing thread and written to local persistent storage to avoid blocking the acquisition thread. In a hybrid cloud architecture, the local message queue is used as a cold cache, retaining only the most recent K windows locally. The remaining data is compressed, encrypted, and resumably transmitted to the cloud-based training service during network downtime. This hierarchical cache storage mechanism enables efficient batch caching and secure transmission of operational data while ensuring real-time performance and reliability.

[0022] Step S202: performing data validity screening and noise suppression processing on the cached operating data to remove abnormal sampling values ​​that do not conform to the expected distribution.

[0023] In this step, the validity of the motion state data (including position, velocity, acceleration, etc.) collected by each sensor channel is checked. Specifically, a reasonable operating range for each parameter is pre-set based on the robot's task conditions and sensor specifications. For example, joint angles are between −180° and 180°, velocity is between 0 and 2.5 m / s, and acceleration is between −10 and 10 m / s². If a sample value is detected outside this range, or if the change between adjacent sample points exceeds a preset mutation threshold (e.g., an acceleration change exceeding 5 m / s²), the sample point is marked as invalid and removed. Gaps in the time series caused by removal can be filled using linear interpolation or previous value filling to maintain time series continuity. Subsequently, based on the data that has eliminated invalid values, noise suppression processing is performed, including using a sliding average filter to calculate the mean within a fixed-length window (such as 5-10 sampling points) to smooth instantaneous fluctuations, and applying a Kalman filter to sensor data with strong dynamics (such as IMU angular velocity and linear acceleration) to further suppress high-frequency noise through weighted fusion of predicted values ​​and measured values.

[0024] During different mission phases, filter parameters are adaptively adjusted based on the noise variance calculated in real time, enhancing smoothing during stable action phases and maintaining responsiveness during rapid action phases. This process effectively removes physically unreasonable values ​​and reduces sensor noise, ensuring that the input data used for policy model training is stable, realistic, and reproducible, thereby reducing training bias caused by outliers or high noise.

[0025] Step S203: performing alignment and synchronization processing on the operation data after removing abnormal sampling values ​​according to a unified time reference.

[0026] In this step, the high-precision system clock or synchronized Network Time Protocol (NTP) time provided by the robot's main controller can be used to timestamp the sensor data. For sensor data with low sampling frequency or irregular sampling periods (such as magnetic encoder position feedback), linear interpolation or spline interpolation can be used to extrapolate the values ​​of intermediate sampling points on the target time scale. For sensor data with higher sampling frequencies (such as acceleration and angular velocity from an IMU), a resampling strategy is used to extract sampling points at a unified target sampling rate (e.g., 100 Hz). Low-pass filtering is used when necessary to eliminate aliasing effects introduced by resampling.

[0027] To eliminate the time delay between different sensors, the delay deviation between each data stream and the global time reference can be calculated before synchronization (obtained by comparing the synchronization pulse signal or timestamp), and the timestamp is compensated to align it under the unified time reference. For data segments with short data loss or delay, a previous value retention or prediction completion strategy based on the last few samples can be used to ensure that the time series after synchronization is continuous and available on the entire time axis.

[0028] Through the above interpolation, resampling and delay compensation mechanism under the unified time reference, the data of different sensor channels can be strictly aligned on the same time scale, thereby ensuring the consistency and comparability of each input feature in the time dimension during subsequent feature extraction, state-action sequence construction and policy model fine-tuning process, avoiding training errors and performance degradation caused by time misalignment.

[0029] Step S204: Extracting feature parameters from the synchronized running data through mean statistical operation to form a feature representation for model training.

[0030] In this step, the core variables that can directly reflect the motion and control state of the robot are selected from the running data, including but not limited to position, velocity, acceleration, joint angle, driving torque, and control input, etc. Based on the unified time series within the time window, sliding window operation is performed on each key running variable, and the window length can be set according to the task dynamics and sensor noise level (such as 0.5-2 seconds). For each window, the arithmetic mean of the variable (i.e. the sum of all sample values in the window divided by the number of sample points) is calculated to depict the steady state level of the time period; at the same time, the standard deviation or variance is calculated to reflect the fluctuation amplitude.

[0031] In scenarios with significant high-frequency fluctuations, moving average filtering or exponential weighted moving average can be used to smooth the variable sequence to obtain trend features. For high-frequency data such as inertial measurement units, peak-to-valley values, vibration frequencies, and other dynamic features can also be extracted within the window. If necessary, derived indicators can be generated in combination with statistical features, such as calculating instantaneous power from speed and acceleration, calculating trajectory curvature from position changes, or calculating joint power consumption from torque and angular velocity. For multi-dimensional sensor data such as three-axis acceleration and three-axis angular velocity, the mean and fluctuation features of each dimension can be calculated simultaneously, and the vector length and direction change rate can be extracted to form a comprehensive kinematic feature description.

[0032] Through the above processing, the system suppresses instantaneous noise while retaining key information reflecting the true running trend of the robot, thereby providing stable and generalizable training samples for policy model updates and reducing the risk of policy performance degradation caused by transient anomalies.

[0033] Step S205: The feature representation is mapped into structured training data that conforms to the training input format to serve as the expert strategy for fine-tuning the strategy model.

[0034] In this step, the encoding method for the state vector and action vector is determined based on the input and output interface definitions of the policy model. The state vector is composed of the key operational features extracted in step S204, combined in a fixed dimensional order. For example, it includes the robot's joint angles, joint angular velocities, end effector position and posture, linear velocity and angular velocity, and external environment perception information (such as distance sensor range values ​​and visual feature encoding vectors). The action vector is derived from the control commands actually executed by the robot at the corresponding timestamp, such as joint target positions, joint driving torques, and end-effector trajectory control parameters.

[0035] To ensure temporal consistency between state and action, state features extracted within the same time window are paired with the corresponding control instructions at that time point, forming {state vector, action vector} sample pairs. These pairs are then arranged in chronological order to construct structured training data. For multi-dimensional sensor inputs (such as vision and touch), feature-encoded multimodal information can be appended to the state vector, enabling the model to simultaneously utilize multi-source sensory features for decision optimization.

[0036] In specific implementations, structured training data is organized and stored as tensors or matrices and batched into standard input formats compatible with deep learning frameworks such as PyTorch and TensorFlow. Where necessary, normalization, dimensionality compression, or masking mechanisms can be introduced to mitigate the impact of numerical span variations on model training and improve training efficiency. The resulting structured training data can be directly input into the model fine-tuning module as expert policy, enabling the policy model to selectively correct for deviations and improve performance in real-world environments while retaining its original decision-making capabilities.

[0037] Therefore, the expert strategy is composed of sample pairs consisting of state vectors and action vectors generated by the pre-trained strategy running in the real environment. No manual labeling is required, which significantly reduces the cost of data preparation, avoids the subjective bias introduced by manual labeling, and can continuously obtain highly consistent and highly reliable training data during long-term operation.

[0038] Step S3: Based on the running status of the structured training data, determine whether the preset update conditions are met, and when the preset update conditions are met, use the structured training data as the expert strategy to fine-tune the strategy model to generate an updated strategy model.

[0039] In this embodiment, the preset update conditions may include but are not limited to the following trigger mechanisms: Data volume threshold: When the current accumulated number of structured training data samples Dk When the number of samples is greater than or equal to a preset value M (for example, M = 2048 samples), fine-tuning training is triggered once; Performance threshold: when the average reward value of the last 10 running cycles decreases by more than a preset proportion δ (for example, δ = 3%) compared with the historical baseline, fine-tuning training is triggered once; Time threshold: when the time interval from the last model update exceeds a preset maximum duration T max (for example, T max = 30 minutes), fine-tuning training is triggered once.

[0040] When any of the above trigger conditions is met, the fine-tuning training process of the policy model is started. In step S3, the structured training data is used to fine-tune the expert policy to generate an updated policy model, which specifically includes: Step S301: Map the structured training data to ordered pairs of state vectors and action vectors, and train a function approximation model of the expert policy based on the ordered pairs, which is used to generate a reference action vector in the fine-tuning process as a target output to constrain the update direction of the policy model.

[0041] In this step, the structured training data is mapped to ordered pairs of state vectors and action vectors in time sequence , where the state vector contains key running variables of the robot at that time, and the action vector represents the corresponding control instructions. Then, a function approximation model (for example, based on a multi-layer perception MLP or a lightweight Transformer model structure) of the expert policy is trained based on these ordered pairs, which can generate a reference action vector in the policy fine-tuning stage as a target output to constrain the update direction of the policy model.

[0042] Step S302: When performing parameter update on the policy model, the weights of the feature extraction layer, the normalization layer, and the non-terminal fully connected layer of the environment perception are fixed based on a hierarchical parameter freezing mechanism, and only the trainable parameters of the terminal output layer are updated by gradient backpropagation using an incremental update mechanism.

[0043] In this step, the functional layers of the policy model are identified based on the preset network layer configuration table, including: a convolutional feature extraction layer for environmental perception, a normalization layer for feature distribution standardization, several non-terminal fully connected layers, and a terminal decision output layer. During the update process, the system locks the weights of the feature extraction layer, normalization layer, and non-terminal fully connected layer into a read-only state through the parameter freeze mark, prohibiting them from participating in the back-propagation calculation, and only allowing the weights and bias parameters of the terminal output layer to remain trainable. To this end, the training scheduling module of the policy model will skip the weight update operation of the frozen layer in the gradient calculation graph during the back-propagation phase, retaining only the gradient calculation and weight update process of the terminal output layer; at the same time, the parameter index that needs to be updated is registered in the optimizer to ensure that the frozen layer maintains its original weight value unchanged during multiple rounds of iterations.

[0044] Because the front-end feature extraction layer of the policy model typically learns universal features of the robot's perceived environment (such as the geometric structure and dynamic patterns of sensor input), these features are highly transferable and stable across different tasks or environments. When the underlying perception components remain unchanged, the updated model maintains consistency with the perception feature space of the previous version. This facilitates smooth policy switching during hot deployment and reduces sudden changes in action or control instability caused by changes in feature distribution. Therefore, this hierarchical parameter freezing mechanism prevents the destruction of the underlying universal perception features during fine-tuning, reduces the size of training parameters to accelerate convergence, and reduces the risk of overfitting when real data is limited, thereby ensuring the stability and generalization of policy transfer.

[0045] Step S303: When optimizing the loss function of the policy model, an imitation learning constraint based on the reference action vector is introduced to minimize the weighted difference measure between the policy model's predicted action vector and the reference action vector, thereby causing the policy model to converge to a distribution consistent with the expert policy in the output space.

[0046] In this step, an imitation learning constraint based on a reference action vector is introduced when optimizing the policy model's loss function. Specifically, during the policy model parameter optimization process, the reference action vector generated by the expert policy under the corresponding operating state of the training sample is used as a comparison benchmark for the target output. The difference between the action vector predicted by the policy model under the same operating state and the reference action vector is measured.

[0047] The difference metric can be a combination of weighted mean squared error (WME) and cosine similarity. The WME measures the deviation in the numerical magnitude of the predicted action. Weights are assigned to each action dimension based on its importance to the task, for example, dimensions related to the end-effector position are weighted higher than redundant joint dimensions. Cosine similarity measures the directional consistency between the predicted and reference actions, ensuring that the policy update is not only numerically close but also maintains the reasonableness of the trajectory direction. After normalization, these difference metrics are added to the total loss function as imitation learning constraints and weighted summed with the environment-task loss (e.g., the negative value of the reward function). The training scheduling module dynamically adjusts the weights of the two losses during batch iterations to strike a balance between convergence speed and imitation accuracy. This mechanism enables the policy model to gradually converge to a distribution consistent with the expert policy in the output space during the update process, effectively reducing the risk of sudden action changes during policy switching and improving stability and execution security during deployment.

[0048] Step S4: hot deploy the updated strategy model to the real operating environment of the robot, so that the robot can run based on the updated strategy model without interrupting task execution and generate new operating data.

[0049] In this embodiment, step S4 specifically includes: Step S401: Determine whether to adopt a full update method or a differential update method based on the parameter difference measurement value between the updated strategy model and the currently running strategy model; Step S402: When the parameter difference metric exceeds a preset percentage threshold of the full model parameter scale, a full model package of the updated strategy model is generated; Step S403: When the parameter difference metric value does not exceed the percentage threshold, a binary difference package of the updated policy model relative to the current running policy model is generated based on a binary difference generation algorithm; Step S404: compress the binary differential package or the full model package to form a compressed package and store it in the cloud server, and generate a globally unique version identifier for each version of the compressed package based on the timestamp and random number; Step S405: Obtain the target version identifier and download the corresponding compressed package through the robot's polling request interface or a network interface based on support for cloud server push; Step S406: The robot receives the compressed package and performs an integrity check. If the check passes, the updated strategy model is loaded into the backup control channel, and the current operating status parameters of the robot are migrated to the updated strategy model, so as to replace the current operating strategy model with the updated strategy model without interrupting task execution.

[0050] Specifically, the parameter difference between the updated policy model and the current running policy model is measured to obtain a parameter difference value Δθ, and the proportion of Δθ in the total parameter size of the current running policy model is calculated. When the proportion of Δθ is greater than a preset percentage threshold (such as 90% of the total model parameter size), the full update mode is selected, and the updated policy model is packaged as a full model package; when the proportion of Δθ does not exceed the percentage threshold, the binary differential update mode is used, and the bsdiff4 difference generation algorithm is called to generate a binary difference package of the updated policy model relative to the current running policy model, and the zstd-3 compression algorithm is used to compress the difference package, and the compression rate can reach 92% on average, so as to reduce the transmission bandwidth occupation.

[0051] The cloud server generates a globally unique 64-bit version identifier (model_id) based on the combination of the generation timestamp and the random number for each generated full model package or difference compressed package, and stores the compressed model package in the cloud model distribution service. The cloud server provides model version acquisition and download services through a RESTful interface. The interface supports HTTP / 2 protocol and Server Push mechanism, and can actively push version information to the robot side when a new version is detected. The robot side can poll the target version model_id by calling the GetVersion() interface. After confirming that the new version is available, the robot side initiates a download request to the RESTful interface, and receives an HTTP 204 No-Content status code as the end of transmission after the model data transmission is completed, and then enters the model integrity verification stage.

[0052] After receiving the compressed package from the cloud server, the robot side first decompresses the compressed package, and performs integrity and source credibility verification based on a preset verification algorithm (such as SHA-256 hash value comparison or digital signature verification) to ensure that the model file has not been tampered with and the source is legal during transmission. If the verification fails, the compressed package is discarded and an alarm information is recorded; if the verification passes, the updated policy model is loaded into the standby control channel. During the loading process, the system continues to retain the current running policy model in the main control channel to ensure the continuity of task execution. Then, the system migrates the current running state parameters (including sensor input cache, control output cache, task execution progress flag, etc.) to the running context of the updated policy model, and performs a smooth switching operation. By loading the updated policy in the standby control channel and maintaining the dual-buffer mechanism of the current running policy, it is ensured that even if an exception occurs during policy switching, the original policy can be quickly rolled back, avoiding task interruption and ensuring the continuous and stable operation of the robot.

[0053] Optional implementations also support manually triggered deployment processes, including one-click rollback and forced pull modes. The one-click rollback mode quickly restores the policy model to a previous version if a performance regression risk is detected during a policy update. The forced pull mode bypasses version consistency checks and directly deploys a specified version of the policy model in emergency situations. This mechanism enables safe, smooth, and controllable hot updates of policy models without interrupting task execution.

[0054] Furthermore, in step S406, a smooth transition mechanism is adopted to replace the current running policy model with the updated policy model, which specifically includes: Step S4061: After the updated policy model is loaded, a double-buffered policy pool is used to retain the current running policy model and the updated policy model respectively; Step S4062: performing a weighted combination of the output action vectors of the current running strategy model and the updated strategy model based on a preset interpolation function to obtain a strategy output result of the transition phase; Step S4063: During the process of gradually increasing the weight coefficient of the interpolation function from the initial value to the target value, monitoring the difference between the output of the updated strategy model and the output of the current running strategy model; Step S4064: When the difference does not exceed the preset action mutation threshold, the weight coefficient is continuously increased according to the weight coefficient increasing rule until the model switching is completed; Step S4065: When the difference exceeds the preset action mutation threshold, the weight coefficient is rolled back to the previous safe value, and the weight coefficient increment process is restarted after a preset delay until a smooth switch to the updated strategy model is completed.

[0055] Specifically, after the updated strategy model is loaded into the standby channel and the running state migration is completed, a double-buffer strategy pool is constructed to store the current running strategy model and update strategy model The preset interpolation strategy function is called to perform a weighted combination of the output action vectors of the two models to obtain the strategy output of the transition phase: Among them, the weight The initial value is 0 and increases linearly to 1 within the transition time window (e.g., 2 seconds).

[0056] exist During the incremental process, the update strategy output is calculated in real time Output with current strategy The difference and compare it with the preset action mutation threshold ε (such as 0.2 rad); when the difference does not exceed the threshold ε, continue to increase according to the increasing rule , until =1 to complete the model switching; when the difference exceeds the threshold ε, Roll back to the previous safety value (e.g., 50%), and restart after a preset safety waiting time (e.g., 500 ms). The process continues incrementally until the safety conditions are met and the switch is completed.

[0057] Utilizing an interpolation strategy function with incremental weighting over time, the updated strategy output is gradually introduced, effectively preventing sudden changes in control output at the moment of switching, thereby reducing execution risks such as joint impact and trajectory deviation. Combined with real-time monitoring of motion mutation thresholds, weights are immediately rolled back and retries are delayed when the output difference exceeds a safe limit, ensuring that the strategy switching process remains within safe control and improving the smoothness and safety of the switching process. Therefore, the use of a weight rollback and delayed restart mechanism allows for rapid rollback to a safe state and self-recovery in the event of instability during the strategy switching process, enhancing the system's fault tolerance and adaptability.

[0058] Furthermore, in step S4, the step of hot deploying the updated strategy model to the real operating environment of the robot also includes a continuous operation guarantee mechanism for the real-time control task of the robot, specifically including: A dual-process architecture is used to isolate the main control process used to perform real-time robot control tasks from the policy loading process used to load and update the policy model. The main control process runs continuously at a fixed frequency and is not interrupted by the policy loading process. When an exception occurs in the policy loading process during model input and output processing, the watchdog timer restarts the policy loading process within the preset time limit and rolls back to the previous version of the policy model to ensure the continuous operation of the main control process.

[0059] Specifically, a dual-process architecture is employed, isolating the main control process (which performs the robot's core real-time control tasks) from the policy loading process (which is responsible for loading and updating the policy model). The main control process runs as a real-time thread with a configurable 1 kHz cycle frequency. It performs core tasks such as joint actuation, motion planning, sensor sampling, and control command issuance, and maintains continuous operation throughout the system's operating cycle, uninterrupted by policy updates or model loading. The policy loading process, on the other hand, runs as a non-real-time thread, dedicated to handling input and output operations such as file reading, parameter parsing, differential merging, and model initialization for updating the policy model.

[0060] If the policy loading process exits abnormally during model loading due to file corruption, memory overflow, or communication interruption, a built-in watchdog timer periodically monitors its status. If the policy loading process stops responding or crashes, the system automatically triggers a restart within a preset safety timeout (e.g., within 200ms) and rolls back the policy model to the last stable version before the update, ensuring the robot continues to operate based on the verified policy.

[0061] Through the above design, the main control process can maintain the continuity and stability of the control loop during the entire policy hot deployment process. Abnormalities in the policy loading process will not affect the execution of real-time control, thereby achieving the decoupling of policy updates and task execution, significantly improving the system's security, fault tolerance and task continuity.

[0062] Furthermore, after the step of replacing the currently running policy model with the updated policy model, a performance verification mechanism after policy deployment is also included, specifically including: After the deployment of the updated strategy model is completed, the lightweight simulator is called to load the updated strategy model and run the preset number of steps. The performance index is calculated based on the running results. If the performance index is lower than the preset ratio threshold of the historical performance index, the update strategy model online process is aborted. When the performance indicators meet the preset ratio threshold, part of the real-time task traffic will be switched to the updated strategy model for online verification. Multiple operating indicators including average reward value, trajectory tracking error and motion smoothness will be collected within the preset test duration. When the performance degradation of any operating indicator exceeds the preset tolerance threshold, it will trigger an automatic rollback to the previous version of the strategy model.

[0063] Specifically, the policy model is first verified offline using a built-in lightweight simulator. For example, the updated policy model is loaded in headless mode using the MuJoCo physics simulation engine. The model is then run in the simulation environment for a preset number of steps (e.g., 1000 steps, approximately 10 seconds). The average reward value is calculated based on the results and compared with the average reward value of the historical stable version. If the average reward value is less than 0.95 times the historical stable version, the updated policy model is deemed to not meet deployment requirements, and the rollout process is aborted.

[0064] After the updated policy model is deployed to the robot's operating environment, an online A / B testing phase is conducted (using the currently running old policy model as the baseline group and the newly deployed updated policy model as the test group). This phase verifies the actual performance of the updated policy model without disrupting the normal operation of most real-time control tasks. A preset ratio (e.g., 10%) of the real-time control task input streams are allocated to the updated policy model for processing, while the remaining input streams are processed by the currently running baseline policy model, forming a test group and a baseline group running in parallel. During the testing phase, multiple performance metrics are collected and analyzed for both the test and baseline groups over a preset test duration (e.g., 2 minutes), including but not limited to average reward, root mean square error (RMSE), and motion smoothness (calculated based on acceleration variance). If the updated policy model degrades beyond a preset tolerance threshold (e.g., 3%) in any performance metric, an automatic rollback mechanism is triggered, switching all real-time task traffic back to the baseline policy model to prevent performance degradation from adversely affecting robot task execution. This online traffic diversion and comparative testing verifies the feasibility of the new policy in a real-world environment. Strict performance monitoring and rollback mechanisms are used to ensure that even if there are problems with the new strategy, it will not affect the security and stability of the overall task.

[0065] Step S5: New operating data is collected again, and the steps of collecting operating data, preprocessing operating data, fine-tuning the training strategy model, and deploying and updating the strategy model are executed cyclically to utilize real operating feedback to drive continuous optimization of the strategy model. During the cyclic execution process, the three stages of data collection, strategy training, and model hot deployment are executed in parallel, and data interaction and process decoupling between the stages are achieved through message queues.

[0066] Specifically, after a policy model update and deployment, new operational data is generated, and the next round of operational data collection and optimization begins. By cyclically executing the steps of operational data collection, operational data preprocessing, fine-tuning and training the policy model, and deploying and updating the policy model, a closed-loop policy model optimization process driven by real-world operational feedback is formed. During this cyclical execution, the system employs a closed-loop operational scheduling mechanism, including a time-based scheduling strategy and a parallel pipeline execution architecture.

[0067] In the time scheduling strategy, the data collection cycle Tc is set to 5 minutes. This means that a complete operational dataset, including key operational variables, task execution logs, and raw sensor data, is collected from the robot's real-time operating environment every 5 minutes. The policy training cycle T𝑡 is set to 3–10 minutes and dynamically adjusted based on the current amount of collected data and task complexity. When the data volume is large, a shorter cycle is used to quickly respond to performance degradation; when the data volume is small, the cycle is extended to ensure sufficient training samples. The model deployment cycle Td is set to 30 seconds, which includes a smooth transition process for replacing the current operating policy model with the updated policy model, keeping changes in control output within a safe range. This cycle configuration achieves an orderly temporal connection and overall rhythm control between the three phases of collection, training, and deployment, ensuring timely response to changes in the operating environment while avoiding instability caused by frequent switching.

[0068] The parallel pipeline execution architecture designates the data collection phase, policy training phase, and model hot deployment phase as three independent execution pipelines, each running in a different compute thread or process. Message queues (such as ZeroMQ) are used to transmit data and trigger events between these phases. Collected data is sent to the training pipeline via the message queue, and the trained model package is passed to the deployment pipeline via the message queue. This decouples the stages and prevents delays in a single phase from blocking the overall process. The message queue transmits the model package with a version number, timestamp, and validation information, ensuring that each stage processes a consistent and valid version of the input, preventing data mismatches or model rollback errors.

[0069] In this parallel mode, while the training pipeline is fine-tuning the model based on the previous batch of run data, the acquisition pipeline can continue to acquire new run data without having to wait for training to complete. Upon receiving the updated policy model, the deployment pipeline can immediately perform hot deployment and policy switching without affecting ongoing data collection and training tasks. Through parallel pipelines, the system can simultaneously process multiple rounds of run data and model update requests, significantly improving the throughput and real-time performance of policy optimization.

[0070] This closed-loop operation scheduling mechanism enables continuous learning and autonomous correction of the policy model based on real-world operational feedback without interrupting the robot's real-time control tasks. This not only ensures the stability of the policy update process and the continuity of task execution, but also enables the robot to operate stably and long-term in dynamic application scenarios such as industrial production lines and service robots, which require extremely high runtime and stability. This significantly reduces manual calibration and parameter adjustment costs, and effectively improves the system's intelligence and environmental adaptability.

[0071] The newly collected operating data is not only used for this round of training, but also forms a long-term optimized data pool with historical data, so as to enable dynamic sampling during the training phase and improve the generalization ability of the model.

[0072] Compared with related technologies, the solution provided in the embodiments of the present application uses the state-action sequence generated by the pre-trained policy model in the real operating environment as structured training data, performs condition-triggered cyclic fine-tuning on the policy model, and combines it with hot deployment of the model, so that the policy model can be updated without interrupting task execution. In the process of cyclic execution of data collection, data preprocessing, policy fine-tuning and model deployment, the three stages are parallelized and data interaction and process decoupling are carried out through message queues, thereby utilizing real operation feedback to achieve continuous optimization of the policy model, improve the stability and generalization ability of policy migration, and reduce the risk of performance degradation of traditional migration methods in actual deployment. Experimental results show that this method can reduce the overall migration error of traditional migration strategies in real deployment from approximately 10% to 20% to less than 5%, achieving more stable control performance and higher task execution reliability.

[0073] In addition, some embodiments of the present application further provide an electronic device. The electronic device may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device may also be various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0074] The electronic device includes: one or more processors; and a memory storing computer program instructions, and when the computer program instructions are executed, the processor executes a robot control strategy migration method provided in the above embodiment. Figure 3An exemplary structural diagram of the electronic device is disclosed. The electronic device includes: one or more processors 1101, a memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, with each device providing some of the necessary operations. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0075] The electronic device may further include: an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means. Figure 3 The bus connection is taken as an example.

[0076] Input device 1103 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. Examples include a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, and other input devices. Output device 1104 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0077] To provide user interaction, the electronic device may be a computer. The computer includes a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse) through which the user can provide input to the computer. Other types of devices may also be used to provide user interaction; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback), and input from the user may be received in any form (e.g., voice input or tactile input).

[0078] In an embodiment of the present application, a computer-readable medium stores a computer program / instructions. When executed by a processor, the computer program / instructions implement a robot control strategy migration method provided in the above embodiment. The computer-readable medium may be included in the electronic device described in the above embodiment, or it may exist independently and not be incorporated into the device. The computer-readable medium carries one or more computer-readable instructions.

[0079] The memory 1102 can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor 1101 executes the non-transitory software programs, instructions, and modules stored in the memory 1102 to execute various functional applications and data processing of the server, thereby implementing the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0080] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely located relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0081] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. Computer-readable media may be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0082] Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technology, compact discs, digital versatile discs or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0083] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network or a wide area network, or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] In the above embodiments, all or part of the steps or functions of the present invention may be implemented using software, hardware, firmware, or any combination thereof. For example, implementation may be achieved using a dedicated integrated circuit, a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of the present application may be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) may be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, a floppy disk, or the like. In addition, some steps or functions of the present application may be implemented using hardware, for example, as a circuit that cooperates with a processor to perform the various steps or functions.

[0085] The computer program product provided in the embodiments of the present application includes one or more computer programs / instructions that, when executed by a processor, fully or partially produce the processes or functions described in accordance with the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0086] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-specific system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0087] The scope of this application is defined by the appended claims rather than the foregoing description and is therefore intended to encompass within this application all changes that come within the meaning and range of equivalents of the claims. Any reference signs in the claims should not be construed as limiting the claims to which they relate. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim may also be implemented by one unit or device through software or hardware. Words such as "first" and "second" are only used to distinguish the description and do not indicate any particular order, nor should they be understood as indicating or implying relative importance.

[0088] The above descriptions are merely specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art may easily propose variations or substitutions within the technical scope disclosed in the present application, and such variations or substitutions shall be encompassed within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims, and the above descriptions shall be regarded as exemplary and non-limiting.

Claims

1. A robot control strategy migration method, characterized in that: include: Collecting the robot's operating data in the real operating environment. The operating data consists of the state-action sequence generated by the pre-trained policy model when running in the real physical environment. Preprocessing the operation data to obtain structured training data including the state-action sequence; Determining whether a preset update condition is satisfied based on the running state of the structured training data, and when the preset update condition is satisfied, fine-tuning the strategy model using the structured training data as an expert strategy to generate an updated strategy model; Hot deploying the updated strategy model to the real operating environment of the robot, so that the robot can run based on the updated strategy model without interrupting task execution and generate new operating data; The new operating data is collected again, and the steps of collecting operating data, preprocessing operating data, fine-tuning the training strategy model, and deploying and updating the strategy model are executed cyclically to drive continuous optimization of the strategy model using real operating feedback. During the cyclic execution, the three stages of data collection, strategy training, and model hot deployment are executed in parallel, and data interaction and process decoupling between the stages are achieved through message queues.

2. The robot control strategy migration method according to claim 1, characterized in that: The step of preprocessing the operating data to obtain structured training data containing the state-action sequence includes: caching the operation data in batches based on a preset sampling period; Performing data validity screening and noise suppression processing on the cached operating data to remove abnormal sampling values ​​that do not conform to the expected distribution; Aligning and synchronizing the operating data after removing abnormal sampling values ​​according to a unified time reference; Extracting characteristic parameters from the synchronized operating data through mean statistical operation to form a characteristic representation for model training; The feature representation is mapped into structured training data that conforms to the training input format to serve as an expert policy for fine-tuning training of the policy model.

3. The robot control strategy migration method according to claim 1, characterized in that: The step of using the structured training data as an expert strategy to fine-tune the strategy model to generate an updated strategy model includes: Mapping the structured training data into ordered pairs of state vectors and action vectors, and training a function approximation model of the expert strategy based on the ordered pairs, wherein the function approximation model is used to generate a reference action vector during fine-tuning to serve as a target output to constrain the update direction of the strategy model; When performing parameter updates on the strategy model, the weights of the feature extraction layer, normalization layer, and non-terminal fully connected layer of the environment perception are fixed based on the layered parameter freezing mechanism, and only the trainable parameters of the terminal output layer are updated by gradient backpropagation using the incremental update mechanism; When optimizing the loss function of the policy model, an imitation learning constraint based on the reference action vector is introduced to encourage the policy model to converge to a distribution consistent with the expert policy in the output space by minimizing the weighted difference measure between the action vector predicted by the policy model and the reference action vector.

4. The robot control strategy migration method according to claim 1, characterized in that: The step of hot deploying the updated strategy model to the real operating environment of the robot, so that the robot operates based on the updated strategy model without interrupting task execution, includes: Determining whether to adopt a full update method or a differential update method based on a parameter difference metric between the updated strategy model and the currently running strategy model; When the parameter difference metric value exceeds a preset percentage threshold of the full model parameter scale, generating a full model package of the updated strategy model; When the parameter difference metric value does not exceed the percentage threshold, generating a binary difference package of the updated policy model relative to the current running policy model based on a binary difference generation algorithm; Compress the binary differential package or the full model package to form a compressed package and store it on the cloud server, and generate a globally unique version identifier for each version of the compressed package based on a timestamp and a random number; Obtain the target version identifier and download the corresponding compressed package through the robot's polling request interface or a network interface based on support for cloud server push; The robot receives the compressed package and performs an integrity check. If the check passes, the updated strategy model is loaded into the backup control channel, and the current operating status parameters of the robot are migrated to the updated strategy model, so as to replace the current operating strategy model with the updated strategy model without interrupting task execution.

5. The robot control strategy migration method according to claim 4, characterized in that: In the step of replacing the currently running policy model with the updated policy model, a smooth transition mechanism is used to perform policy switching, specifically including: After the update strategy model is loaded, a double-buffer strategy pool is used to retain the current running strategy model and the update strategy model respectively; Performing a weighted combination of the output action vectors of the current running strategy model and the updated strategy model based on a preset interpolation function to obtain a strategy output result of the transition phase; During the process of gradually increasing the weight coefficient of the interpolation function from an initial value to a target value, monitoring the difference between the output of the updated strategy model and the output of the current operation strategy model; When the difference does not exceed the preset action mutation threshold, the weight coefficient is continuously increased according to the weight coefficient increasing rule until the model switching is completed; When the difference exceeds the preset action mutation threshold, the weight coefficient is rolled back to the previous safe value, and the increment process of the weight coefficient is restarted after a preset delay until a smooth switch to the updated strategy model is completed.

6. The robot control strategy migration method according to claim 1, characterized in that: The step of hot deploying the updated strategy model to the real operating environment of the robot also includes a continuous operation guarantee mechanism for the robot's real-time control task, specifically including: A dual-process architecture is used to isolate a main control process for executing real-time robot control tasks from a policy loading process for loading the updated policy model, wherein the main control process runs continuously at a fixed frequency and is not interrupted by the policy loading process; When an exception occurs in the policy loading process during the model input and output processing, the policy loading process is restarted within a preset time limit through the watchdog timer and rolled back to the previous version of the policy model to ensure the continuous operation of the main control process.

7. The robot control strategy migration method according to claim 1, characterized in that: After the step of replacing the currently running policy model with the updated policy model, a performance verification mechanism after policy deployment is also included, specifically including: After the deployment of the update strategy model is completed, a lightweight simulator is called to load the update strategy model and run a preset number of steps, and a performance indicator is calculated based on the running result. If the performance indicator is lower than a preset ratio threshold of the historical performance indicator, the update strategy model online process is terminated; When the performance indicator meets the preset ratio threshold, part of the real-time task traffic is switched to the updated strategy model for online verification. Multiple operating indicators including average reward value, trajectory tracking error and action smoothness are collected within the preset test duration. When the performance degradation of any operating indicator exceeds the preset tolerance threshold, it triggers automatic rollback to the previous version of the strategy model.

8. An electronic device, characterized in that: The electronic device comprises: One or more processors; and a memory storing computer program instructions, wherein when the computer program instructions are executed, the processor executes the robot control strategy migration method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that: When the computer program and / or instructions are executed by a processor, the robot control strategy migration method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instruction is executed by a processor, the robot control strategy migration method according to any one of claims 1 to 7 is implemented.

Images