Visual-language-action model online action optimization method, system, device and medium

Abstract

The application discloses a visual-language-action model online action optimization method, system, device and medium, wherein the method comprises the following steps: based on the collected real-time image and task instruction, a visual-language-action model is used to perform action reasoning at a first frequency to output a coarse-grained end effector pose, and a low-frequency slow loop is formed; taking the coarse-grained end effector pose as an initial state, taking a pose offset parameter as a state vector, taking a task-specific energy function as a target function, and running a simulated annealing algorithm at a second frequency higher than the first frequency to perform real-time fine adjustment to form a high-frequency fast loop; based on an environment disturbance signal and an end effector grasping confidence change detected in real time, a preset threshold is dynamically generated, and an event-triggered reheating mechanism is used to determine an optimized SA algorithm. The application solves the problem that a VLA model open-loop action trajectory cannot be adaptively corrected in a dynamic disturbance environment, and realizes robust action output in a physically unstable environment.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and edge computing technology, and specifically relates to online motion optimization methods, systems, devices and media for vision-language-motion models. Background Technology

[0002] Vision-Language-Action (VLA) models are a class of multimodal models that unify visual perception, language understanding, and action generation within a single deep learning framework. VLA models, such as Qwen-VLA, can generate end effector motion parameters based on language commands and visual input after pre-training on wide-area robot datasets, showing broad application prospects in the field of autonomous robot operations.

[0003] However, the motion trajectory output by the VLA model during the inference phase is essentially open-loop: the model generates the complete end effector pose all at once and executes it directly, unable to perceive or respond to instantaneous changes in the environment during execution. In particular, the VLA model has high computational overhead and low frequency (typically 2-5Hz) for inference, which is fundamentally contradictory to the high-frequency response (typically above 100Hz) required for real-time motion control. When the VLA model is deployed on mobile platforms (such as ambulances, agricultural vehicles, etc.), this contradiction is further exacerbated by physical environmental disturbances.

[0004] The existing closed-loop control methods have the following shortcomings: (1) Gradient-based visual servoing methods are prone to getting trapped in local optima under strong disturbances; (2) Learning-based reactive strategies require a large amount of labeled disturbance data for training, which is difficult to collect on actual target platforms, and the trained strategies are difficult to generalize to unseen disturbance patterns.

[0005] Simulated Annealing (SA) is a general probabilistic optimization algorithm that avoids getting trapped in local optima by accepting degraded solutions at high temperatures and then finely converges to a high-quality approximate solution at low temperatures. Traditional simulated annealing employs a monotonically decreasing temperature scheduling strategy, suitable for static or slowly changing environments. However, in dynamically perturbed environments, when abrupt changes cause drastic shifts in the optimization landscape, the monotonically decreasing temperature cannot allow the optimizer to escape the established local optima, causing the optimizer to "freeze" in a suboptimal state.

[0006] In the prior art, Chinese patent CN120480906A discloses a robot adaptive decision-making method based on a combination of reinforcement learning and simulated annealing. This method dynamically adjusts the decay coefficient of the temperature parameter in simulated annealing using Q-value function gradient information and immediate reward values ​​to achieve a balance between exploration and exploitation. However, the temperature adjustment in this method always remains below the initial temperature and follows a monotonically decreasing framework—essentially only regulating the rate of temperature decrease, it cannot proactively raise the temperature to restart global exploration when the environment undergoes drastic changes. This invention relies on the iterative update mechanism of reinforcement learning, requires a large amount of environmental interaction data, and is difficult to directly apply to online action correction scenarios requiring immediate responses.

[0007] Chinese patent CN122200228A discloses a post-training method for VLA models based on reinforcement learning. This method constructs diverse simulation scenarios during the training data generation stage using a dynamic environment perturbation strategy to improve model generalization. However, this method is only applicable to offline training and cannot address the problem of adaptive action correction when the VLA model faces sudden changes in the real physical environment (such as vehicle bumps or dynamic occlusion) during online inference.

[0008] Therefore, the following technical gaps urgently need to be addressed in the existing technology: (i) the lack of an asynchronous dual-loop architecture that effectively connects the low-frequency inference capability and high-frequency online optimization capability of the VLA model; (ii) the lack of an event-triggered scheduling mechanism that can proactively break the monotonically decreasing paradigm of simulated annealing temperature and restart global exploration when the environment changes abruptly; (iii) the lack of an alternative evaluation mechanism that can efficiently evaluate the energy values ​​of candidate poses without actually executing the end effector actions; and (iv) the lack of a cross-module information fusion mechanism that dynamically correlates the inherent uncertainty output of the VLA model with the energy function parameters of the online optimizer. There is an urgent need for an online action optimization method that can perform real-time closed-loop correction of the open-loop actions of the VLA model under dynamic perturbation environments and proactively escape local optima when the environment changes abruptly. Summary of the Invention

[0009] To address the problems existing in the prior art, this invention proposes an online action optimization method, system, device, and medium for the visual-language-action model. By introducing an event-triggered reheating temperature scheduling strategy during the online optimization process of the simulated annealing algorithm, the optimizer can actively escape local optima when the environment changes abruptly, thereby achieving robust action output of the VLA model in physically unstable environments.

[0010] The technical solution of the present invention is as follows: On the one hand, this invention provides an online motion optimization method for a vision-language-action model, comprising the following steps: Based on the acquired real-time images and task commands, a coarse-grained end effector pose is output using a vision-language-action model at the first frequency to perform action reasoning, thus forming a low-frequency slow loop. Using the coarse-grained end effector pose as the initial state, the pose offset parameter as the state vector, and the task-specific energy function as the objective function, the coarse-grained end effector pose is finely adjusted in real time using a simulated annealing algorithm at a second frequency higher than the first frequency, and the fine-grained end effector pose is output, forming a high-frequency fast loop, which forms an asynchronous dual-loop architecture with the low-frequency slow loop. Based on real-time detected environmental disturbance signals and changes in the end effector's grasp confidence, a preset threshold is dynamically generated to determine the event-triggered reheating mechanism: if the preset threshold is exceeded, the annealing temperature of the simulated annealing algorithm is increased to the reheating temperature, and the simulated annealing algorithm is restored to its descent according to a preset asymmetric accelerated cooling strategy, converging and freezing to output a refined end effector pose.

[0011] Preferably, the working mechanism of the asynchronous dual-ring architecture is as follows: The low-frequency slow loop uses the first frequency f VLA The algorithm runs, completes single-action reasoning, and outputs the coarse-grained end effector pose; the simulated annealing algorithm operates at a second frequency f. SA Run, where f SA >f VLA Multiple simulated annealing iterations are performed between two adjacent action inferences; When a new round of motion reasoning is completed and a new coarse-grained end effector pose is output, the simulated annealing algorithm resets the current state to the new coarse-grained end effector pose, reinitializes the annealing temperature, and starts a new round of online optimization to achieve asynchronous handover between low-frequency slow loop and high-frequency fast loop. The simulated annealing algorithm determines the target pose for the next cycle based on sensor data from the most recent execution cycle. The optimization iteration of the simulated annealing algorithm is executed asynchronously with the physical motion of the robotic arm. The new refined end effector pose is sent to the motion controller only after the current robotic arm motion is completed, so that the update frequency of the simulated annealing algorithm optimization results matches the actual motion bandwidth of the robotic arm.

[0012] Preferably, in image acquisition scenarios under vibration, the task-specific energy function is defined by the negative value of the image sharpness metric function; In target grasping scenarios under occlusion, a task-specific energy function is constructed using a weighted combination of grasping confidence and collision penalty negative values. The weight coefficients of the task-specific energy function are dynamically correlated with the distribution characteristics of the original logarithmic probability values ​​of the action tokens output by the visual-language-action model action decoder. The correlation method is as follows: obtain the original logarithmic probability values ​​of the action tokens output by the visual-language-action model action decoder, calculate the softmax normalized probability distribution of the original logarithmic probability values ​​of each action dimension and calculate its information entropy, and take the mean or maximum value of the information entropy of all action dimensions as the uncertainty measure; when the uncertainty measure is lower than a preset entropy threshold, increase the ratio of grasping confidence weight to collision penalty weight to strengthen the task optimization tendency; when the uncertainty measure is higher than the preset entropy threshold, decrease the ratio of grasping confidence weight to collision penalty weight to strengthen the collision avoidance safety tendency.

[0013] Preferably, the task-specific energy function is calculated based on an alternative model, and the specific implementation of the alternative model is as follows: In the image acquisition scenario under vibration, the alternative model is an analytical point spread function model based on inertial measurement unit data and camera model. The task-specific energy function value is obtained by analytical calculation of the Frobenius norm of the analytical point spread function model. The data acquisition of the inertial measurement unit is strictly synchronized with the camera exposure time, and the sampling timestamp covers the window interval of the exposure period. In target grasping scenarios under occlusion, the alternative model is a geometric projection model based on the current frame RGB-D image, instance segmentation results, and depth map. The grasping confidence is calculated by the intersection-union ratio of the target bounding box projection area and the visible pixel area under the candidate pose. The collision penalty is calculated by the shortest Euclidean distance between the candidate pose and the obstacle point cloud cluster.

[0014] Preferably, the determination of the event-triggered reheating mechanism based on dynamically generating a preset threshold according to the real-time detected environmental disturbance signal and the change in the end effector's grasp confidence is as follows: Based on the statistical characteristics of the environmental disturbance signal within the preset monitoring period and the change in the end effector's grasp confidence, a first preset threshold and a second preset threshold are dynamically generated, and the reheating temperature is dynamically calculated based on the current environmental disturbance signal and the environmental energy landscape roughness. When the intensity of environmental disturbance exceeds the first preset threshold or the rate of decrease in end effector grasp confidence exceeds the second preset threshold, and the time interval since the last reheating event exceeds the preset reheating cooling interval, the current annealing temperature of the simulated annealing algorithm is raised to the reheating temperature, and then the descent is resumed according to the preset asymmetric accelerated cooling strategy, converging and freezing to output the refined end effector pose.

[0015] Preferably, the first preset threshold is generated as follows: the mean and standard deviation of the environmental disturbance signal are calculated within a preset sliding time window to construct a dynamic threshold T1=μ+α·σ, where μ is the mean of the environmental disturbance signal within the sliding window, σ is the standard deviation of the environmental disturbance signal within the sliding window, and α is a preset sensitivity coefficient; the second preset threshold is obtained by calculating the expected rate baseline of the decrease in the end effector grasping confidence using the exponential weighted moving average method and taking a preset multiple.

[0016] Preferably, the reheating temperature T reheat The generation method is: T reheat =T0·f(D,R), where T0 is the initial annealing temperature, D is the normalized value of the currently detected environmental disturbance signal, R is the current environmental energy landscape roughness, and f(·) is a preset monotonically increasing mapping function; the environmental energy landscape roughness R is obtained by calculating the local variance of the task-specific energy function value in the most recent preset number of iterations of the simulated annealing algorithm; The preset reheating and cooling interval is environmentally adaptive: preset reheating and cooling interval T cool =T base ·g(dD / dt filtered ), where T base The basic cooling interval, g(·) is a preset monotonically increasing function, t filtered dD / dt is the time variable after low-pass filtering. filtered This represents the rate of change of the environmental disturbance signal after low-pass filtering.

[0017] Preferably, the preset asymmetric accelerated cooling strategy specifically includes: Update annealing temperature T k+1 =T k ·λ reheat , where λ reheat To accelerate the cooling coefficient, and λ reheat <λ initial , λ initial T is the cooling coefficient of the simulated annealing algorithm during the initial annealing stage as the temperature decreases from the initial temperature T0. k Let T be the annealing temperature for the kth time. k+1 The annealing temperature is the (k+1)th annealing temperature. When the preset asymmetric accelerated cooling strategy reduces the annealing temperature to below the preset termination temperature and the number of iterations completed since the most recent reheating event is not less than the preset minimum number of freeze iterations, the fine-grained end effector pose is frozen and output.

[0018] On the other hand, the present invention provides an online action optimization system for a vision-language-action model, including an image acquisition module, a language input module, an action reasoning module, a simulated annealing online optimization module, a disturbance detection module, and an event-triggered reheating scheduling module; The image acquisition module is used to acquire real-time images; The language input module is used to input task commands; The action reasoning module is used to perform action reasoning based on the acquired real-time images and task instructions at a first frequency using a vision-language-action model to output coarse-grained end effector pose, forming a low-frequency slow loop; The simulated annealing online optimization module is used to perform real-time fine-tuning of the coarse-grained end effector pose at a second frequency higher than the first frequency using the simulated annealing algorithm, with the pose offset parameter as the state vector and the task-specific energy function as the objective function, taking the coarse-grained end effector pose as the initial state, the pose offset parameter as the state vector, and the task-specific energy function as the objective function. The module outputs the fine-tuned end effector pose, forming a high-frequency fast loop, which forms an asynchronous dual-loop architecture with the low-frequency slow loop. The amplitude of the random disturbance applied in the simulated annealing algorithm is adaptively adjusted with the annealing temperature. The disturbance detection module is used to detect environmental disturbance signals and changes in the end effector's grasp confidence level in real time. The event-triggered reheating scheduling module is used to dynamically generate a preset threshold based on real-time detected environmental disturbance signals and changes in the confidence level of the end effector, and to determine the event-triggered reheating mechanism: if the preset threshold is exceeded, the annealing temperature of the simulated annealing algorithm is increased to the reheating temperature, and the simulated annealing algorithm is restored to its descent according to a preset asymmetric accelerated cooling strategy, converged and frozen to output a refined end effector pose.

[0019] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the online motion optimization method for the vision-language-action model as described in any embodiment of the present invention.

[0020] Compared with the prior art, the present invention has the following technical effects: 1. Adaptive Action Correction under Dynamic Perturbation: The event-triggered reheating mechanism proposed in this invention enables the simulated annealing algorithm to actively escape local optima during sudden environmental changes, rather than being "frozen" in a suboptimal state. Compared to the stagnation of traditional monotonic annealing and the limitations of existing continuous temperature adjustment methods that can only slow down convergence but not restart the search, this invention can continue to search for the optimal action even in strongly perturbed environments. Asymmetric accelerated cooling significantly shortens the recovery time after escaping local optima.

[0021] 2. No need to perturb training data or perform actual actions for evaluation: This invention is based on the zero-order optimization characteristics of the SA algorithm, requiring only the definition of a task-specific energy function to function, without relying on any perturbation dataset. Crucially, the task-specific energy function is evaluated using an IMU-camera analytical model or a depth map geometric projection model, eliminating the need to actually drive the end effector to move to the candidate pose for testing during the optimization iteration process. Therefore, the optimization process does not consume mechanical execution time and does not introduce the risk of physical collisions.

[0022] 3. Low computational overhead suitable for edge deployment: The SA algorithm described in this invention is performed in a 4 to 7-dimensional low-dimensional space, and a single iteration only involves state perturbation and scalar energy evaluation (microsecond to millisecond level), which is suitable for edge scenarios with limited computing power.

[0023] 4. Asynchronous dual-loop architecture decouples large model inference from real-time optimization: In the method described in this invention, the VLA low-frequency inference loop and the SA high-frequency optimization loop run asynchronously and independently, and complete the state reset and handover when the new VLA inference result arrives, thus solving the fundamental contradiction between large model inference latency and real-time control requirements.

[0024] 5. The linkage between the task-specific energy function and the uncertainty output of the VLA model enables cross-module information fusion: The SA optimizer perceives decision uncertainty through the distribution entropy value of the action token logits of the VLA model (with zero additional computational overhead) and adaptively adjusts the optimization strategy between task accuracy and safety. Attached Figure Description

[0025] Figure 1 This is a schematic diagram showing the connection relationship between the vision-language-motion model online motion optimization system and the end effector described in this invention; Figure 2 This is a schematic diagram of the simulated annealing algorithm flow of the simulated annealing online optimization module; Figure 3 This is a schematic diagram of the workflow of the event-triggered reheating scheduling module. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0027] Example 1 This embodiment provides an online motion optimization method for a vision-language-motion model, including the following steps: Based on the acquired real-time images and task commands, a coarse-grained end effector pose is output using a vision-language-action model at a first frequency, forming a low-frequency slow loop. The coarse-grained end effector pose includes pose parameters with 4 to 7 degrees of freedom, such as the spatial pose parameters of the end effector.

[0028] Using the coarse-grained end effector pose as the initial state x0, 4- to 7-dimensional pose offset parameters as the state vector, and the task-specific energy function E(x) as the objective function, the coarse-grained end effector pose is finely adjusted in real time using a simulated annealing algorithm at a second frequency higher than the first frequency. This outputs a fine-grained end effector pose, forming a high-frequency fast loop, which, together with the low-frequency slow loop, creates an asynchronous dual-loop architecture. The amplitude of the random perturbation applied in the simulated annealing algorithm is adaptively adjusted according to the annealing temperature.

[0029] Based on real-time detected environmental disturbance signals and changes in the end effector's grasp confidence, a preset threshold is dynamically generated to determine the event-triggered reheating mechanism: if the preset threshold is exceeded, the annealing temperature of the simulated annealing algorithm is increased to the reheating temperature, and the simulated annealing algorithm is restored to its descent according to a preset asymmetric accelerated cooling strategy, converging and freezing to output a refined end effector pose.

[0030] As a preferred embodiment of this invention, the working mechanism of the asynchronous dual-ring architecture is as follows: The low-frequency slow loop uses the first frequency f VLA The algorithm runs, completes single-action reasoning, and outputs the coarse-grained end effector pose; the simulated annealing algorithm operates at a second frequency f. SA Run, where f SA >f VLA Multiple simulated annealing iterations are performed between two adjacent action inferences.

[0031] When a new round of motion reasoning is completed and a new coarse-grained end effector pose is output, the simulated annealing algorithm resets the current state to the new coarse-grained end effector pose, reinitializes the annealing temperature, and starts a new round of online optimization to achieve asynchronous handover between low-frequency slow loops and high-frequency fast loops.

[0032] The simulated annealing algorithm determines the target pose for the next cycle based on sensor data from the most recent execution cycle. The simulated annealing optimization iteration is executed asynchronously with the physical motion of the robotic arm; the new refined end effector pose is only sent to the motion controller after the current robotic arm movement is completed, ensuring that the update frequency of the simulated annealing optimization results matches the actual motion bandwidth of the robotic arm. It should be noted that SA optimization is essentially inter-frame optimization, rather than intra-frame real-time compensation during a single exposure or action execution. This architecture resolves the inherent contradiction between the high computational cost and low frequency of VLA large-model inference and the need for high-frequency iterations in SA online optimization to track environmental changes.

[0033] Furthermore, the termination conditions of the simulated annealing algorithm include a temperature freeze condition and a forced termination condition: the temperature freeze condition is that the current annealing temperature drops below the preset termination temperature and the number of iterations completed since the most recent reheating event is not less than the preset minimum number of freeze iterations; the forced termination condition is that the total number of iterations reaches the preset maximum number of iterations.

[0034] As a preferred embodiment of this example, in the image acquisition scenario under vibration, the task-specific energy function E(x) is defined by the negative value of the image sharpness measurement function S(x), i.e., E(x) = -S(x), and the optimization objective is to minimize the energy function to maximize image sharpness.

[0035] In target grasping scenarios under occlusion, the task-specific energy function E(x) is constructed using the weighted negative combination of grasping confidence and collision penalty, i.e., E(x) = -(w1·C(x) - w2·P(x)), where C(x) is the grasping confidence score, P(x) is the collision penalty term, and w1 and w2 are preset weight coefficients. The collision penalty term P(x) increases with the proximity of the end effector to the obstacle.

[0036] The weight coefficients w1 and w2 of the task-specific energy function are dynamically correlated with the distribution characteristics of the raw log probability values ​​of the action tokens output by the action decoder in the vision-language-action model. The correlation method is as follows: obtain the raw log probability values ​​of the action tokens output by the action decoder of the vision-language-action model (action token logits, i.e., the raw scores of the output layer before softmax, which are naturally generated in the forward inference of the model without additional computational overhead), calculate the softmax normalized probability distribution of the raw log probability values ​​(logits) of each action dimension and calculate its information entropy, and take the mean or maximum value of the information entropy of all action dimensions as the uncertainty measure H. uncert When the uncertainty measure H uncertWhen the entropy level is below a preset threshold (output distribution is concentrated, and the model is relatively certain about its output), the ratio w1 / w2 of the grasp confidence weight to the collision penalty weight is increased (e.g., 3:1) to strengthen the task optimization tendency. When the uncertainty measure is above the preset threshold (output distribution is dispersed, and the model is relatively uncertain about its output), the ratio of the grasp confidence weight to the collision penalty weight is decreased (e.g., 1:1 or lower) to strengthen the collision avoidance safety tendency. This correlation mechanism enables the SA optimizer's energy function to perceive the decision uncertainty of the VLA model and adaptively balance the trade-off between achieving the task objective and safe collision avoidance during the optimization process.

[0037] In a preferred embodiment, the task-specific energy function is numerically implemented based on an alternative model, independent of the actual execution of the end effector. This allows the SA algorithm to evaluate a large number of candidate states with extremely low computational overhead in each iteration. The specific implementation of the alternative model is as follows: In image acquisition scenarios under vibration (such as inside an ambulance cabin), the alternative model is a resolution point spread function (PSF) model based on inertial measurement unit (IMU) data and a camera model. The median depth of the target region is extracted from the current frame depth map as the dominant depth plane. The angular velocity and linear velocity sequences sampled by the IMU within the current exposure cycle are obtained (IMU data acquisition is strictly synchronized with camera exposure time, and the sampling timestamps cover the window interval of the exposure cycle). For the candidate attitude offset Δθ generated during the SA iteration, the motion blur point spread function H(Δθ) = (1 / N)·Σ is calculated on the dominant depth plane. i δ(p(ω(t i )+Δθ / τ,v(t i ))), where ω(t) i ) represents the IMU at sampling time t i The collected angular velocity, v(t) i ) represents the IMU at sampling time t i The linear velocity is collected, δ(·) is the discrete impact function on the two-dimensional image plane, p(·) is the perspective projection of the three-dimensional motion onto the image plane, τ is the exposure time, N is the number of sampling points, and i is the sampling point index; the image sharpness metric is defined as S=1 / ||H(Δθ)||² F The energy function is the negative of the sharpness, E(Δθ) = -S(Δθ). This alternative evaluation is based entirely on analytical calculations using IMU data and the camera model, with a single evaluation taking on the order of microseconds. The approximation error introduced by PSF estimation based on the dominant depth plane is typically on the order of 10-15%, and the Metropolis probability acceptance criterion of the SA algorithm naturally tolerates this level of noise.

[0038] In target grasping scenarios under occlusion (such as agricultural harvesting), the alternative model is a geometric projection model based on the current frame RGB-D image, instance segmentation results, and depth map. The grasping confidence is obtained by projecting the target object's bounding box onto the current depth map after rigid body transformation according to the candidate pose offset, calculating the intersection-union ratio (IUU) of the projected region and the visible pixel region, and multiplying it by the grasping quality heuristic function. The formula is expressed as: C approx (Δp)=IoU(Proj(B box ,Δp),Mask visible )·Q(Δp), where C approx (Δp) represents the capture confidence level, Δp is the candidate pose offset vector to be evaluated, and B box The bounding box is the 3D bounding box obtained by instance segmentation of the target object, Proj is the projection after rigid body transformation, and Mask is the bounding box. visible For the visible region based on depth continuity, Q(Δp) is the grasping quality function based on the angle between the normal vectors; the collision penalty is calculated by the shortest Euclidean distance between the end effector and each obstacle point cloud cluster under the candidate pose, expressed by the formula: P approx (Δp)=Σ i max(0,d safe -Dist(Δp,Obstacle i )), where P approx (Δp) is the collision penalty, Obstacle i Let d be the point cloud cluster obtained by point cloud segmentation of the i-th obstacle, and Dist(·,·) be the shortest Euclidean distance between the end effector and the obstacle point cloud cluster in the candidate pose. safe A preset safe distance threshold is set. All calculations in this alternative assessment are based on the current frame sensor data and geometric projection. A single assessment takes milliseconds. The geometric approximation ignores physical factors such as the object's mass distribution. The Metropolis criterion of SA can also tolerate the systematic bias introduced by the geometric approximation.

[0039] In a preferred embodiment of this practice, the step of dynamically generating a preset threshold based on real-time detected environmental disturbance signals and changes in the end effector's grasping confidence level, and determining the event-triggered reheating mechanism, specifically involves: Based on the statistical characteristics of the environmental disturbance signal within the preset monitoring period and the change in the confidence level of the end effector, a first preset threshold and a second preset threshold are dynamically generated, and the reheating temperature is dynamically calculated based on the current environmental disturbance signal and the environmental energy landscape roughness.

[0040] When the intensity of environmental disturbance exceeds the first preset threshold or the rate of decrease in the end effector's grasp confidence exceeds the second preset threshold, and the time interval since the last reheating event exceeds the preset reheating-cooling interval, the current annealing temperature of the simulated annealing algorithm is increased to the reheating temperature. This allows the simulated annealing online optimization module to accept temporarily degraded candidate states with a preset probability to escape the local optimum caused by the environmental disturbance. Subsequently, it resumes descent according to a preset asymmetric accelerated cooling strategy, converges, and freezes the output of the refined end effector pose. The cooling rate of the preset asymmetric accelerated cooling strategy is higher than the cooling rate of the simulated annealing online optimization module during the initial annealing stage.

[0041] Unlike the monotonically decreasing temperature scheduling method in traditional simulated annealing, and the continuous temperature regulation method in existing technologies that only adjusts the cooling rate within the decay framework, the event-triggered reheating mechanism described in this invention continuously monitors environmental disturbance signals and action execution status during the operation of the SA online optimizer. When a disturbance event such as severe vibration or a sharp drop in capture confidence is detected, the annealing temperature is proactively and briefly increased (reheated), allowing the optimizer to accept temporarily degraded candidate states with a higher probability, thereby escaping the local optimum caused by sudden environmental changes. After reheating, the temperature decreases according to an asymmetric accelerated cooling strategy—the cooling rate is higher than the initial cooling stage—allowing the optimizer to quickly converge to the optimal action and freeze in the new landscape. This mechanism enables the SA online optimizer to have adaptive relocation capabilities in dynamic environments without requiring additional labeled disturbance training data.

[0042] In a preferred embodiment of this invention, the dynamic generation mechanism of the first and second preset thresholds does not employ fixed static thresholds. Instead, it adaptively calculates based on the statistical characteristics of the disturbance signal within a sliding time window, enabling the thresholds to automatically adjust in response to changes in the environmental baseline. Specifically, the first preset threshold is generated as follows: within a preset sliding time window, the mean and standard deviation of the environmental disturbance signal are calculated to construct a dynamic threshold T1 = μ + α·σ, where μ is the mean of the environmental disturbance signal within the sliding window, σ is the standard deviation of the environmental disturbance signal within the sliding window, and α is a preset sensitivity coefficient. The second preset threshold is obtained by calculating the expected rate baseline of the decrease in the end effector's grasping confidence using an exponentially weighted moving average method and taking a preset multiple.

[0043] As a preferred embodiment of this invention, the reheating temperature T reheat The generation method is: T reheat=T0·f(D,R), where T0 is the initial annealing temperature, D is the normalized value of the currently detected environmental disturbance signal, R is the current environmental energy landscape roughness, and f(·) is a preset monotonically increasing mapping function; the environmental energy landscape roughness R is obtained by calculating the local variance of the task-specific energy function value in the most recent preset number of iterations of the simulated annealing algorithm, reflecting the non-convexity of the solution space of the current optimization problem. The stronger the disturbance and the rougher the energy landscape, the higher the reheating temperature, in order to provide a corresponding strength of global exploration capability.

[0044] The preset reheating and cooling interval is environmentally adaptive: preset reheating and cooling interval T cool =T base ·g(dD / dt filtered ), where T base The basic cooling interval, g(·) is a preset monotonically increasing function, t filtered dD / dt is the time variable after low-pass filtering. filtered This represents the rate of change of the environmental disturbance signal after low-pass filtering. The cooling interval is shortened when the disturbance subsides rapidly, and remains longer when the disturbance persists or intensifies to prevent oscillation optimization.

[0045] As a preferred embodiment of this invention, the preset asymmetric accelerated cooling strategy is specifically as follows: The cooling process follows T k+1 =T k ·λ reheat T k Let T be the annealing temperature for the kth time. k+1 λ is the annealing temperature for the (k+1)th annealing stage; reheat To accelerate the cooling coefficient, the cooling coefficient λ after reheating is... reheat The cooling coefficient λ is less than that of the initial annealing stage. initial , that is, λ reheat <λ initial , λ initial The cooling coefficient of the simulated annealing algorithm during the initial annealing stage, as it decreases from the initial temperature T0. Before reheating, the SA optimizer has already completed a considerable search within the current energy landscape, accumulating knowledge about the landscape structure. After reheating, it does not need to repeat the slow global search of the initial annealing; instead, it should quickly move through the high-temperature random exploration stage and then rapidly enter the low-temperature fine convergence stage. When the preset asymmetric accelerated cooling strategy reduces the annealing temperature below the preset termination temperature, and the number of iterations since the most recent reheating event is not less than the preset minimum number of freeze iterations, the fine-grained end effector pose is frozen and output.

[0046] To verify the effectiveness and superiority of the method provided in this embodiment, some specific examples are provided below: Example 1: Robust Image Acquisition in an Ambulance Scene In ambulance scenarios, random vibrations from vehicle movement cause motion blur in the camera at the end of a robotic arm mounted inside the vehicle, severely degrading the image quality of specific patient body parts (such as trauma areas). The VLA model infers and outputs the initial camera pose at approximately 3 Hz, while the SA algorithm optimizes the pose bias at approximately 200 Hz. The task-specific energy function is evaluated using an analytical PSF alternative model: the median depth of the patient's target region is extracted from the current frame's depth map as the dominant depth plane; the IMU is strictly synchronized with the camera shutter, and the SA algorithm searches for the camera pose bias that minimizes the PSF norm based on vibration data from the most recent exposure period. SA optimization is essentially inter-frame optimization—optimizing the target pose for the next frame based on the vibration data of the current frame. During quasi-stationary vehicle vibrations, the vibration statistics of adjacent frames are highly similar, making inter-frame optimization effective; when the vehicle experiences a sudden, severe jolt, the vibration statistics change abruptly, triggering a reheating mechanism. The reheating temperature is dynamically calculated based on the jolt intensity and energy landscape roughness, and SA re-explores a pose bias adapted to the new vibration mode, subsequently optimizing the pose bias with λ. reheat (e.g., 0.85, compared to λ) initial =0.95) to accelerate cooling, and restore to the termination temperature within about 15-20 iterations and freeze the output.

[0047] Example 2: Anti-occlusion target capture in agricultural scenarios In agricultural harvesting scenarios, dynamic occlusion caused by wind blowing leaves results in instantaneous changes in the position of the target fruit. The VLA model infers and outputs the initial grasping pose at a frequency of approximately 2-3 Hz. The SA algorithm uses grasping translation and rotation offset parameters as the state vector (4 to 7 dimensions) and a task-specific energy function using the negative weighted combination of grasping confidence and collision penalty. It evaluates candidate poses through a geometric projection substitution model: extracting the 3D bounding box of the target fruit and point cloud clusters of branches and leaves obstacles based on the current frame's RGB-D image and instance segmentation results, and calculating the projection intersection-over-union ratio and collision distance under the candidate poses. A single evaluation is on the order of milliseconds. SA runs at approximately 200 Hz and can complete approximately 66-100 iterations between two VLA inferences. When wind blowing leaves causes the target to be occluded and the rate of decrease in grasping confidence exceeds a dynamically generated second preset threshold, the event-triggered reheating scheduling module dynamically calculates T based on the confidence decrease magnitude and energy landscape roughness. reheatSA (Search Engine Optimization) breaks away from the current locally optimal grasping position and explores new, unoccluded candidate regions. Furthermore, when the VLA (Vacuum-Altered Latitude) model has a high entropy value in its action token logits distribution (indicating uncertainty about the initial grasping pose, such as when the target and background fruit appear similar), the collision penalty weight in the energy function automatically increases. SA then tends to select conservative poses away from obstacles during the search, reducing collision risk. When a new round of VLA inference results arrives, SA resets its state and begins a new round of optimization, seamlessly connecting the two asynchronous loops.

[0048] Example 2 Accordingly, this embodiment provides an online motion optimization system for a vision-language-motion model, used to implement the method described in any embodiment of the present invention, such as... Figure 1 As shown, it includes an image acquisition module, a language input module, an action reasoning module, a simulated annealing online optimization module, a disturbance detection module, and an event-triggered reheating scheduling module.

[0049] The image acquisition module is used to acquire real-time images.

[0050] The language input module is used to input task instructions.

[0051] The action inference module is used to perform action inference based on the acquired real-time images and task commands at a first frequency using a vision-language-action model, outputting a coarse-grained end effector pose, forming a low-frequency slow loop. Furthermore, the action inference module is deployed on an edge computing platform. The edge computing platform employs an embedded AI computing module. In one specific implementation, the edge computing platform uses an embedded AI computing module with a computing power of 275 TOPS, equipped with at least 64GB of memory, a processor including a multi-core CPU and GPU, supports multiple camera inputs and vehicle communication interfaces, adopts a wide-voltage DC power supply and a fanless compact design, and is suitable for deployment environments such as mobile vehicles and agricultural sites.

[0052] The simulated annealing online optimization module is used to perform real-time fine-tuning of the coarse-grained end effector pose using a simulated annealing algorithm at a second frequency higher than the first frequency, taking the pose offset parameter as the state vector and the task-specific energy function as the objective function, with the coarse-grained end effector pose as the initial state. This outputs a fine-tuned end effector pose, forming a high-frequency fast loop, which, together with the low-frequency slow loop, forms an asynchronous dual-loop architecture. The amplitude of the random perturbation applied in the simulated annealing algorithm is adaptively adjusted according to the annealing temperature. On a typical platform f SAThe frequency can reach 100-500Hz, and 20-250 SA iterations can be completed between two adjacent VLA inferences. When the motion inference module completes a new round of VLA inference and outputs a new coarse-grained pose, the SA optimizer resets the current state vector to the new pose, reinitializes the annealing temperature T0, and starts a new round of online optimization. The SA optimization iteration is executed asynchronously with the physical motion of the robotic arm, and the new optimal pose is only sent to the motion controller after the current robotic arm motion is completed.

[0053] See Figure 2 As shown, the workflow of the simulated annealing online optimization module is as follows: Step S1: Using the coarse-grained end effector pose as the initial state x0, calculate the initial task-specific energy function value E(x0), and set the initial annealing temperature T0 according to the preset initial acceptance probability.

[0054] Step S2: Apply a random perturbation to the current state x to generate a candidate state x' = x + Δx, where the magnitude of Δx is related to the current annealing temperature T. online Positive correlation.

[0055] Step S3: Calculate the energy difference ΔE = E(x') - E(x) using the substitution model.

[0056] Step S4: If ΔE < 0, accept candidate state x' (i.e., x ← x'); if ΔE ≥ 0, accept candidate state x' with probability p = exp(-ΔE / T). online Accept candidate states, where T online This is the current annealing temperature.

[0057] Step S5: Determine if the termination condition is met: when the current annealing temperature T online Reduced to the preset termination temperature T min If the number of iterations since the most recent reheat event is not less than the preset minimum number of freeze iterations, the system will freeze normally and output the current state as the refined end effector pose; or if the total number of iterations reaches the preset maximum number of iterations, the system will forcibly terminate and output the current state.

[0058] Step S6: Update T according to the temperature command from the event-triggered reheating scheduling module. online Press λ in normal cooling mode initial When the temperature drops and a reheat command is received, it is set to T. reheat Press λ afterward reheat (λ) reheat <λ initial (Accelerate cooling)

[0059] The disturbance detection module is used to detect environmental disturbance signals and changes in the end effector's grasping confidence level in real time. The disturbance detection module detects the environmental disturbance status (carrier vibration intensity collected by the IMU) and the end effector's action execution status (current grasping confidence score) in real time, and continuously outputs the above information to the event-triggered reheating scheduling module.

[0060] The event-triggered reheating scheduling module is connected to the disturbance detection module and the simulated annealing online optimization module, respectively. It is used to dynamically generate a preset threshold based on the real-time detected environmental disturbance signal and the end effector capture confidence change, and to determine the event-triggered reheating mechanism: if the preset threshold is exceeded, the annealing temperature of the simulated annealing algorithm is increased to the reheating temperature, and the simulated annealing algorithm is restored to descent according to the preset asymmetric accelerated cooling strategy, converged and frozen to output the refined end effector pose.

[0061] See Figure 3 As shown, the working mechanism of the event-triggered reheating scheduling module is as follows: continuously receiving disturbance intensity signals and capture confidence signals; evaluating the statistical characteristics (mean μ and standard deviation σ) of the disturbance intensity within a preset time window, and dynamically generating a first preset threshold T1=μ+α·σ; using EWMA to calculate the baseline for the capture confidence decrease rate, and setting the second preset threshold as a preset multiple of the baseline; dynamically calculating T based on the current disturbance intensity D and energy landscape roughness R. reheat =T0·f(D,R); When the detected disturbance intensity exceeds T1 or the confidence rate decreases by more than T2, and the time interval since the last reheating event exceeds the adaptive cooling interval T. cool =T base ·g(dD / dt filtered When ), raise the annealing temperature to T. reheat Then press T k+1 =T k ·λ reheat (λ) reheat <λ initial To accelerate cooling, determine whether the temperature has dropped to T after each iteration. min The following steps will be taken: After the temperature drops to the termination temperature, the current optimal action parameters will be frozen, and continuous monitoring will resume.

[0062] Example 3 This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the online motion optimization method for the vision-language-action model as described in any embodiment of the present invention.

[0063] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0064] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0065] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0066] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An online motion optimization method for a vision-language-action model, characterized in that, Includes the following steps: Based on the acquired real-time images and task commands, a coarse-grained end effector pose is output using a vision-language-action model at the first frequency to form a low-frequency slow loop. Using the coarse-grained end effector pose as the initial state, the pose offset parameter as the state vector, and the task-specific energy function as the objective function, the coarse-grained end effector pose is finely adjusted in real time using a simulated annealing algorithm at a second frequency higher than the first frequency, and the fine-grained end effector pose is output, forming a high-frequency fast loop, which forms an asynchronous dual-loop architecture with the low-frequency slow loop. Based on real-time detected environmental disturbance signals and changes in the end effector's grasp confidence, a preset threshold is dynamically generated to determine the event-triggered reheating mechanism: if the preset threshold is exceeded, the annealing temperature of the simulated annealing algorithm is increased to the reheating temperature, and the simulated annealing algorithm is restored to its descent according to a preset asymmetric accelerated cooling strategy, converging and freezing to output a refined end effector pose.

2. The online motion optimization method for the vision-language-action model according to claim 1, characterized in that, The working mechanism of the asynchronous dual-ring architecture is as follows: The low-frequency slow loop uses the first frequency f VLA The algorithm runs, completes single-action reasoning, and outputs the coarse-grained end effector pose; the simulated annealing algorithm operates at a second frequency f. SA Run, where f SA >f VLA Multiple simulated annealing iterations are performed between two adjacent action inferences; When a new round of motion reasoning is completed and a new coarse-grained end effector pose is output, the simulated annealing algorithm resets the current state to the new coarse-grained end effector pose, reinitializes the annealing temperature, and starts a new round of online optimization to achieve asynchronous handover between low-frequency slow loop and high-frequency fast loop. The simulated annealing algorithm determines the target pose for the next cycle based on sensor data from the most recent execution cycle. The simulated annealing algorithm optimization iteration is executed asynchronously with the physical motion of the robotic arm. The new refined end effector pose is only sent to the motion controller after the current robotic arm motion is completed, so that the update frequency of the simulated annealing algorithm optimization results matches the actual motion bandwidth of the robotic arm.

3. The online motion optimization method for the vision-language-action model according to claim 1, characterized in that, In image acquisition scenarios under vibration, the task-specific energy function is defined by the negative value of the image sharpness metric function; In target grasping scenarios under occlusion, a task-specific energy function is constructed using a weighted combination of grasping confidence and collision penalty negative values. The weight coefficients of the task-specific energy function are dynamically correlated with the distribution characteristics of the original logarithmic probability values ​​of the action tokens output by the visual-language-action model action decoder. The correlation method is as follows: obtain the original logarithmic probability values ​​of the action tokens output by the visual-language-action model action decoder, calculate the softmax normalized probability distribution of the original logarithmic probability values ​​of each action dimension and calculate its information entropy, and take the mean or maximum value of the information entropy of all action dimensions as the uncertainty measure; when the uncertainty measure is lower than a preset entropy threshold, increase the ratio of grasping confidence weight to collision penalty weight to strengthen the task optimization tendency; when the uncertainty measure is higher than the preset entropy threshold, decrease the ratio of grasping confidence weight to collision penalty weight to strengthen the collision avoidance safety tendency.

4. The online motion optimization method for the vision-language-action model according to claim 1, characterized in that, The task-specific energy function is calculated based on an alternative model, the specific implementation of which is as follows: In the image acquisition scenario under vibration, the alternative model is an analytical point spread function model based on inertial measurement unit data and camera model. The task-specific energy function value is obtained by analytical calculation of the Frobenius norm of the analytical point spread function model. The data acquisition of the inertial measurement unit is strictly synchronized with the camera exposure time, and the sampling timestamp covers the window interval of the exposure period. In target grasping scenarios under occlusion, the alternative model is a geometric projection model based on the current frame RGB-D image, instance segmentation results, and depth map. The grasping confidence is calculated by the intersection-union ratio of the target bounding box projection area and the visible pixel area under the candidate pose. The collision penalty is calculated by the shortest Euclidean distance between the candidate pose and the obstacle point cloud cluster.

5. The online motion optimization method for the vision-language-action model according to claim 1, characterized in that, The specific steps for determining the event-triggered reheating mechanism, which dynamically generates a preset threshold based on real-time detected environmental disturbance signals and changes in the end-effector's grasping confidence level, are as follows: Based on the statistical characteristics of the environmental disturbance signal within the preset monitoring period and the change in the end effector's grasp confidence, a first preset threshold and a second preset threshold are dynamically generated, and the reheating temperature is dynamically calculated based on the current environmental disturbance signal and the environmental energy landscape roughness. When the intensity of environmental disturbance exceeds the first preset threshold or the rate of decrease in end effector grasp confidence exceeds the second preset threshold, and the time interval since the last reheating event exceeds the preset reheating cooling interval, the current annealing temperature of the simulated annealing algorithm is raised to the reheating temperature, and then the descent is resumed according to the preset asymmetric accelerated cooling strategy, converging and freezing to output the refined end effector pose.

6. The online motion optimization method for the vision-language-action model according to claim 5, characterized in that, The first preset threshold is generated as follows: the mean and standard deviation of the environmental disturbance signal are calculated within a preset sliding time window to construct a dynamic threshold T1=μ+α·σ, where μ is the mean of the environmental disturbance signal within the sliding window, σ is the standard deviation of the environmental disturbance signal within the sliding window, and α is a preset sensitivity coefficient; the second preset threshold is obtained by calculating the expected rate baseline of the decrease in the end effector grasping confidence using the exponential weighted moving average method and taking a preset multiple.

7. The online motion optimization method for the vision-language-action model according to claim 5, characterized in that, The reheating temperature T reheat The generation method is: T reheat =T0·f(D,R), where T0 is the initial annealing temperature, D is the normalized value of the currently detected environmental disturbance signal, R is the current environmental energy landscape roughness, and f(·) is a preset monotonically increasing mapping function; the environmental energy landscape roughness R is obtained by calculating the local variance of the task-specific energy function value in the most recent preset number of iterations of the simulated annealing algorithm; The preset reheating and cooling interval is environmentally adaptive: preset reheating and cooling interval T cool =T base ·g(dD / dt filtered ), where T base The basic cooling interval, g(·) is a preset monotonically increasing function, t filtered dD / dt is the time variable after low-pass filtering. filtered This represents the rate of change of the environmental disturbance signal after low-pass filtering.

8. The online motion optimization method for the vision-language-action model according to claim 1, characterized in that, The preset asymmetric accelerated cooling strategy is specifically as follows: Update annealing temperature T k+1 =T k ·λ reheat , where λ reheat To accelerate the cooling coefficient, and λ reheat <λ initial , λ initial T is the cooling coefficient of the simulated annealing algorithm during the initial annealing stage as the temperature decreases from the initial temperature T0. k Let T be the annealing temperature for the kth time. k+1 The annealing temperature is the (k+1)th annealing temperature. When the preset asymmetric accelerated cooling strategy reduces the annealing temperature to below the preset termination temperature and the number of iterations completed since the most recent reheating event is not less than the preset minimum number of freeze iterations, the fine-grained end effector pose is frozen and output.

9. An online motion optimization system for a vision-language-motion model, characterized in that, The system is used to implement the online action optimization method for the visual-language-action model as described in any one of claims 1 to 8, including an image acquisition module, a language input module, an action reasoning module, a simulated annealing online optimization module, a disturbance detection module, and an event-triggered reheating scheduling module; The image acquisition module is used to acquire real-time images; The language input module is used to input task commands; The action reasoning module is used to perform action reasoning based on the acquired real-time images and task instructions at a first frequency using a vision-language-action model to output coarse-grained end effector pose, forming a low-frequency slow loop; The simulated annealing online optimization module is used to perform real-time fine-tuning of the coarse-grained end effector pose at a second frequency higher than the first frequency using the simulated annealing algorithm, with the pose offset parameter as the state vector and the task-specific energy function as the objective function, taking the coarse-grained end effector pose as the initial state, the pose offset parameter as the state vector, and the task-specific energy function as the objective function. The module outputs the fine-tuned end effector pose, forming a high-frequency fast loop, which forms an asynchronous dual-loop architecture with the low-frequency slow loop. The amplitude of the random disturbance applied in the simulated annealing algorithm is adaptively adjusted with the annealing temperature. The disturbance detection module is used to detect environmental disturbance signals and changes in the end effector's grasp confidence level in real time. The event-triggered reheating scheduling module is used to dynamically generate a preset threshold based on real-time detected environmental disturbance signals and changes in the confidence level of the end effector, and to determine the event-triggered reheating mechanism: if the preset threshold is exceeded, the annealing temperature of the simulated annealing algorithm is increased to the reheating temperature, and the simulated annealing algorithm is restored to its descent according to a preset asymmetric accelerated cooling strategy, converged and frozen to output a refined end effector pose.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the online motion optimization method for the visual-language-motion model as described in any one of claims 1 to 8.

Images