Prediction control method based on world model and online fine tuning, medium and equipment

Abstract

The invention discloses a prediction control method based on a world model and online fine tuning, a medium and equipment, and belongs to the technical field of control algorithms. The method comprises the following steps: acquiring multi-modal environment observation data, wherein the environment observation data comprises point cloud data, an RGB image and IMU attitude angle information; inputting the environment observation data into a trained variational auto-encoder network, and predicting a potential state; for the predicted potential state and candidate action sequence, predicting a potential state sequence and a decoded occupation grid map of subsequent steps by using a trained cyclic state space model; and for a current action and a plurality of candidate action sequences, taking minimization of a total cost function of each track as an optimization target, and searching an optimal action sequence in an action space. According to the invention, the prediction efficiency and accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of control algorithm technology, and more specifically, to a predictive control method, medium, and device based on a world model and online fine-tuning. Background Technology

[0002] Age-related functional decline and lower limb dysfunction (such as motor degenerative diseases, hip and knee replacement, cerebral palsy, and sarcopenia) increase the risk of falls in the elderly. Designing a highly robust and intelligent adaptive control system for the elderly and those with lower limb dysfunction has become an urgent clinical and community need.

[0003] Currently, predictive control algorithms for complex dynamic scenarios mainly fall into three categories: 1) Traditional Model Predictive Control (MPC) + Simplified Physical Modeling: Based on offline constructed dynamic models (such as a two-degree-of-freedom car model), control commands are generated using rolling time-domain optimization. This is suitable for trajectory tracking in low-dimensional state spaces, but it relies on idealized models and linearization assumptions, making it difficult to be compatible with high-dimensional sensor data (such as IMU and visual fusion data). When dealing with unstructured obstacles, the number of optimization variables increases dramatically, leading to a sharp drop in real-time performance; 2) Hybrid Architecture of Deep Reinforcement Learning (DRL) and MPC: The policy network is trained end-to-end through DRL to predict obstacle avoidance paths or emergency response paths in dynamic environments. Braking commands partially replace the optimization solver in MPC to accelerate computation. However, the DRL strategy is highly sensitive to historical experience data and is prone to prediction accumulation errors in unseen scenarios such as sudden changes in target pose or lighting occlusion. Furthermore, online training requires additional computational resources, exacerbating hardware load conflicts. 3) Online adaptive MPC: This algorithm introduces a dynamic parameter estimator (such as a Kalman filter) to correct model errors or environmental disturbance compensation terms in real time, improving the robustness of the controller. However, there is resource contention between online parameter updates and MPC optimization threads, which can easily lead to control lag or even system oscillation due to computational delays. Moreover, it has limited adaptability to sudden multimodal disturbances (such as the superposition of dense furniture and slipping actions).

[0004] In the prior art, patent application CN118585813A discloses an agent control method based on a world model, including the following steps: acquiring environmental observation data; training a world model using a training dataset; the world model includes a variational autoencoder module, a sequence modeling module, a hidden state prediction module, and an optimization module; the variational autoencoder module is used to encode the observation data sampled from the replay buffer by adding adaptive Gaussian noise, generating a latent vector; the sequence modeling module is used to generate a hidden state based on the latent vector and the action vector generated by the agent; the hidden state prediction module generates a prediction result based on the hidden state; the optimization module optimizes the model parameters based on the prediction result of the hidden state prediction module; the world model is used to generate an imagined trajectory; and the agent confirms the optimal strategy based on the imagined trajectory. However, the imagined trajectory is generated based on the current strategy, which is prone to getting trapped in local optima and makes it difficult to explore the globally optimal path. Patent application CN119270885A discloses an unmanned system control method, device, and medium based on a multi-level world model, which can be widely applied in the field of unmanned control technology. This application trains a large-world model from a cloud-side model using multimodal data including image, video, or business text data. Then, it distills this large-world model to obtain a middle-world model on the edge and a small-world model on the endpoint. Natural language instructions for the target object and the agent's state from the small-world model are input into the middle-world model to predict working instructions. These working instructions are then input into the small-world model to predict adjustments to the unmanned system's working state. The unmanned system is then adjusted according to these adjustments, thus reducing reliance on human experts. However, in this approach, knowledge distillation from the large model to the small model relies on simple loss functions such as mean squared error (MSE), which fails to preserve long-tail patterns in complex environments, and the cloud-edge-end communication link easily becomes a bottleneck.

[0005] Analysis reveals that existing technologies have the following main drawbacks: 1) Existing cross-modal world models rely on high-dimensional parameterized networks (such as multi-branch encoders), resulting in excessive computational load and significantly increased inference latency when deployed at the edge. In multimodal data fusion, sensor noise (such as visual occlusion and IMU zero drift) spreads through the shared latent space, reducing the robustness of environmental modeling. In addition, virtual sample generation lacks physics engine verification, which can easily lead to dynamic out-of-bounds actions (such as joint torque exceeding limits), exacerbating policy security risks.

[0006] 2) In long-term time-domain inference, the sequence prediction module of traditional world models (such as RNN / LSTM) causes the imagined trajectory to deviate from the dynamics of the real environment due to error accumulation (such as trajectory drift). Policy optimization relies on the black-box hidden state space and lacks explicit constraints on physical quantities (such as the stable region of the centroid), which may generate high-risk control instructions. In addition, although adaptive noise injection improves generalization, it can lead to feature loss or prediction failure due to empirical threshold setting.

[0007] 3) The cloud-edge-device layered model suffers from version synchronization delays and communication bottlenecks. In weak network environments, command response timeouts cause unmanned systems to "freeze." Multimodal task command (such as natural language + sensor data) parsing relies on preset templates, lacks open semantic understanding capabilities, and lightweight model distillation ignores long-tail scene features, resulting in a sharp drop in task execution success rate. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a predictive control method, medium, and device based on a world model and online fine-tuning.

[0009] According to a first aspect of the present invention, a predictive control method based on a world model and online fine-tuning is provided. The method includes the following steps: Acquire multimodal environmental observation data, including point cloud data, RGB images, and IMU attitude angle information; The environmental observation data is input into a trained variational autoencoder network to predict the potential state; For the predicted latent states and candidate action sequences, a trained recurrent state space model is used to predict the subsequent latent state sequences and the decoded occupied grid map for multiple steps. Given the current action and multiple candidate action sequences, the optimal action sequence is searched in the action space with the goal of minimizing the total cost function of each trajectory.

[0010] According to a second aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described predictive control method based on a world model and online fine-tuning.

[0011] According to a third aspect of the present invention, a computer device is provided, comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that the processor, when executing the computer program, implements the steps of the above-described predictive control method based on a world model and online fine-tuning.

[0012] Compared with existing technologies, the advantages of this invention lie in its predictive control method based on a world model and online fine-tuning. By constructing a latent space dynamic modeling and lightweight comparative fine-tuning mechanism, it effectively overcomes problems such as high-dimensional control-sensor joint optimization, rapid adaptive modeling under environmental drift, and online learning stability assurance. This invention can simultaneously achieve a technical balance between high-speed environmental adaptation, low resource consumption, and high reliable operation, solving the long-standing triple contradiction problem of response speed, computational overhead, and safety redundancy in dynamic control systems. It provides a new paradigm of intelligent decision-making with real-time evolution capabilities for intelligent systems for elderly care.

[0013] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0015] Figure 1 This is a flowchart of a predictive control method based on a world model and online fine-tuning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a predictive control method based on a world model and online fine-tuning according to an embodiment of the present invention; Figure 3 This is a schematic diagram of an application process for navigation according to an embodiment of the present invention. Detailed Implementation

[0016] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0017] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0018] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0019] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0020] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0021] In summary, the predictive control method based on a world model and online fine-tuning provided by this invention achieves highly adaptive reasoning and low-latency safe control in dynamic environments through latent state contrastive learning and MPC rolling optimization. It reduces model complexity by decoupling coding and corrects environmental perception biases in real time by combining online fine-tuning, ultimately achieving high-precision and robust autonomous decision-making in lightweight deployment at the edge.

[0022] See Figure 1 As shown, the provided predictive control method based on a world model and online fine-tuning includes the following steps: Step S1: Construct a variational autoencoder network for generating latent states and train it using a set loss function.

[0023] The Variational Autoencoder (VAE) network is a variational temporal model that integrates multimodal coding, latent state inference, and temporal dynamic modeling. The model as a whole includes a multimodal encoder for extracting fused features from point clouds, images, and IMU data, generating the mean and variance of latent states, a cyclic state space module, and a multimodal decoder for reconstructing the observations at the next time step and predicting the reward.

[0024] Combination Figure 2 As shown, the overall structure of the variational autoencoder network includes a multimodal sensor input layer, a modal branch processing module, a cyclic state space model, and a decoder (not shown).

[0025] The multimodal sensor input layer is used to input different modal data, such as LiDAR point cloud data (10Hz, about 10,000 points per frame, 360° scan), three-channel RGB images, and IMU attitude angle information (pitch angle, roll angle, sampling frequency 100Hz).

[0026] The modality branching module is used to extract feature data from different modalities, including sub-branches for point cloud processing, image processing, and temporal mapping. Specifically, in the point cloud processing (PointNet+ branch) sub-branch, the input is the original point cloud p∈R. Nx3 (N represents point data), local geometric features are extracted using, for example, a four-layer pyramid structure, ultimately outputting a 128-dimensional data vector. In the image processing sub-branch, the input RGB image I is used for feature extraction via depthwise separable convolution and an attention mechanism, resulting in a 128-dimensional feature vector. In the temporal mapping sub-branch, IMU temporal data... With path point sequence The mapping is performed using, for example, an MLP network to map the 4-dimensional vector at each time step to 64 dimensions, and then a single-layer GRU network is used for temporal fusion to output a 128-dimensional fused feature vector. f Then, the features from each modality are concatenated, and a context vector is generated through a temporal fusion layer (such as GRU) (or multimodal fusion layer): (1) in, This represents the fusion characteristics at time t. This represents the hidden state at time t-1, used to carry information about historical time steps. This represents a gated loop unit, and t represents time or time step.

[0027] Output latent state distribution parameters: (2) in, This represents the contextual hidden state at time t, i.e., the feature representation after multimodal temporal fusion. express t The mean vector of the potential states at time 1. express t The standard deviation vector of the latent state at time t is given by MLP, which stands for Multilayer Perceptron.

[0028] Sampling latent states: (3) The input to the decoder is the latent state. and current action (e.g., robot linear velocity, angular velocity). In one embodiment, the decoder mainly includes a multi-layer deconvolutional network (image reconstruction) and fully connected layers (point cloud and IMU reconstruction). The output reconstructs the observation of the next frame. and predicting reward signals (Used to reinforce learning through reward guidance) express t+1 Predicted point cloud data at any given time (i.e., the reconstruction result of the 3D point cloud of the scene in the next frame). express t+1 The predicted RGB image at time (i.e., the reconstruction result of the visual observation in the next frame). This represents the predicted IMU pitch angle at time t+1. This represents the predicted IMU roll angle at time t+1. This represents the predicted reward signal at time t+1, used for policy updates or value function calculations in reinforcement learning.

[0029] In one embodiment, the overall loss function for training a variational autoencoder network is... It consists of three parts, represented as: (4) In the formula: It is the potential space regularization loss. for The divergence coefficient can be dynamically adjusted, for example, with an initial value of 0.1. To reward prediction weights, It is the loss from multimodal observation reconstruction. It is the prediction loss after adding the reward signal.

[0030] For multimodal observation reconstruction loss It can be decomposed into a weighted sum of three terms: point cloud mode, image mode, and IMU pose mode, expressed as: (5) in, It is the point cloud reconstruction loss. It is the structural similarity index loss. It is the IMU attitude and path reconstruction loss. , and These are the weighting coefficients for the corresponding terms.

[0031] Chamfer Distance Loss Used to measure the reconstructed point cloud Compared with real point clouds The geometric consistency is expressed as: (6) in, It is about reconstructing point clouds. It is a real point cloud, where p represents the set of real point clouds. A three-dimensional point, Represents the reconstructed point cloud set A three-dimensional point.

[0032] The Structural Similarity Index (SSIM) loss is used to evaluate the quality of reconstructed images and is expressed as: (7) in, It is to reconstruct the image. X It is a real image.

[0033] The IMU attitude and path reconstruction (MSE) loss is used to calculate the mean square error of pitch angle, roll angle, and path points, and is expressed as: (8) Where T represents the number of time steps. It is the actual pitch angle at time t. It is the predicted pitch angle at time t. It is the actual roll angle at time t. It is the predicted roll angle at time t. express t The x-coordinate of the actual path point at time [time]. express t The x-coordinate of the predicted path point at time [time]. express t The y-coordinate of the actual path point at time [time]. express t The y-coordinate of the predicted path point at time [time].

[0034] , and It can be set according to actual needs or simulation.

[0035] Latent space regularization loss Used to constrain the distribution of potential states Approximately a standard normal distribution, represented as: (9) in, The Kullback–Leibler divergence measures the degree of difference between two probability distributions. i Represents the th in the latent vector Indexes in 10 dimensions The dimension representing the latent space, i.e., the latent variables. Length, The latent variables of the model in the latent space are inferred by the encoder. This represents the observed inputs to the model, such as point clouds, images, IMU data, etc.

[0036] Considering that the task involves reinforcement learning, a reward signal prediction loss is added for supervision, which is represented as: (10) in, This represents the true reward signal at time t+1. This represents the predicted reward signal at time t+1.

[0037] Step S2: Use the cyclic state-space model to capture the dynamic evolution of the environment and make dynamic predictions.

[0038] Cyclic State-Space Model (RSSM) uses deterministic states With random latent states Joint modeling to capture the dynamic evolution patterns of the environment: State transition equation: (11) Observation generation equation: (12) Reward prediction equation: (13) in, The gated loop unit encodes the temporal dependency between historical states and actions, and the latent states. Follow the mean The standard deviation is Gaussian distribution, action Generated by a policy network or by external input, Represents the potential state at time t-1. This represents the action at time t-1. Indicates a given deterministic state Lower potential state The probability distribution, This represents the predicted observation at time t. Indicates the state of being hidden. and potential state Under these conditions, generate observations The probability distribution, This represents the predicted reward signal at time t. This represents the probability distribution of rewards given the hidden and potential states.

[0039] For training strategies and loss function design of recurrent state-space models, dynamic scenarios (such as robot navigation and obstacle avoidance) can be constructed in robot simulation software (such as Gazebo), and multimodal observation data can be collected through ROS programs. Data augmentation is performed by adding sensor noise (such as Gaussian noise, zero drift) and dynamic disturbances. Multi-step prediction loss is used. During the time window Minimize the difference between predicted and actual observations.

[0040] In one embodiment, multi-step prediction loss Represented as: (14) in, Indicates the previous time step The offset k relative to the window start point represents the maximum number of steps in multi-step prediction. Indicates the actual observation at future time steps The value, including multimodal data, This represents the model's prediction of future observations. This indicates the true reward signal at future time steps. The value, This represents the future reward value predicted by the model. This represents the weighting coefficients of the loss terms, used to balance the contributions of observation error and reward prediction error.

[0041] Step S3: Enhance the model's generalization ability by using online fine-tuning and contrastive learning mechanisms.

[0042] 1. Data Stream Processing and Online Buffer During the input phase, the observation-action-reward triplet is dynamically acquired. The latent state is generated through a pre-trained VAE encoder network. VAE - Encoder Feature extraction is performed. A buffer contains 1000 of the most recent interaction data points, and older data is dynamically overlaid to maintain environmental adaptability. New samples are lightly perturbed (e.g., Gaussian noise, random occlusion) to enhance generalization. Samples with larger prediction errors are assigned higher sampling weights based on historical prediction loss records.

[0043] 2. Contrastive learning loss design By measuring similarity in the latent space, the clustering of positive sample pairs (similar states) is enhanced, thereby improving the model's ability to generalize to new environmental patterns.

[0044] In one embodiment, the contrastive loss formula (modified InfoNCE) is set as follows: (15) Temporally adjacent potential states Construct positive sample pairs and randomly select states from non-adjacent time steps in the buffer. Construct a negative sample pool. B represents the batch size. Indicates the temperature coefficient. A similarity measure function for two states in the latent space. i This represents the index of the positive sample pair in the current batch. j This represents the index of a negative sample in the negative sample pool. This indicates the number of negative samples in the negative sample pool. This represents the time step offset when constructing positive sample pairs, used to select potential states that are close in time.

[0045] 3. Parameter update strategy Core principles: 1) Fix the VAE encoder weights: to avoid destroying the multimodal feature representation capability of pre-trained VAEs. 2) Limit the fine-tuning range: Only update the parameters of the MLP dynamic prediction layer and reward predictor in RSSM.

[0046] 3) Elastic Weight Constraint (EWC): Limits deviations from critical legacy task parameters. EWC regularization term: Add a parameter importance penalty term to the loss function: (16) in, It is the parameter importance matrix, which can be calculated based on the Fisher information matrix from the pre-training phase: (17) In the formula, This represents the elasticity coefficient, used to control the constraint strength on the old task parameters. This indicates the th [model name] in the current training process. One trainable parameter, This indicates the first stage corresponding to the pre-training phase. These parameter values ​​are used as a reference to prevent excessive deviation. This indicates the old task dataset. Take the expected value. ) indicates that the model is at the input Lower prediction output The probability distribution is determined by the parameters. Decide, x y represents the input sample of the old task, and y represents the corresponding label or target output of the old task. Indicates parameters The importance metric, calculated from the Fisher information matrix, reflects the importance of this parameter to the performance of legacy tasks.

[0047] Step S4: Perform model predictive control and real-time planning.

[0048] Step S4 executes Model Predictive Control (MPC) and real-time planning. For example, for predicting potential states... and candidate action sequences Expanded prediction is performed using a pre-trained RSSM model: (18) Output: Predicted Step-by-step potential state sequence and the occupied raster map of decoding . This indicates the RSSM decoder, used to convert the latent state... With hidden state Map back to the observation space.

[0049] Raster graph decoding: A lightweight deconvolutional network is used to map the latent state to... Obstacle probability map, formula: (19) in, This represents a lightweight deconvolutional network used to process latent states. Mapped to the original probability map of occupied grid cells. Represents the corresponding grid point The probability of the existence of obstacles.

[0050] The total cost function is a weighted sum of multiple objectives, and this total cost function needs to be minimized. (20) Weight configuration (ground robot scenario): These represent the weighting coefficients used to balance the importance of different cost objectives. This represents the path tracking error penalty, measuring the deviation of the predicted position from the reference path. This indicates a control penalty, restricting the range of motion. Indicates the proximity to obstacles to avoid collisions. This represents a motion smoothness constraint, limiting the range of variation between adjacent actions. This represents the end-game reward, encouraging the sequence's endpoint to approach the target position.

[0051] (1) Path tracking error Penalty prediction location and reference path Deviation: (twenty one) in, Indicates from predictive observations The robot's current position coordinates are extracted from the data. Indicates the reference path at time step The target location coordinates.

[0052] (2) Cost of Controlling Quantity Suppress large movements (such as sharp turns / acceleration / deceleration): (twenty two) (3) Obstacle proximity Safety margin (distance to the nearest obstacle) is calculated based on the predicted raster map. ): (twenty three) Indicates a safe distance. This indicates the distance between the predicted location and the nearest obstacle. This represents a small constant to prevent division by zero.

[0053] (4) Motion smoothness Constraining adjacent action differences: (twenty four) (5) Endgame reward ( Encourage predictions that the endpoint will be close to the target: (25) In the formula, The negative sign indicates a penalty. The current position coordinates of the robot at the end of the predicted sequence are represented by Goal, and the desired target position is represented by Goal. This indicates the width of the reward distribution at the finish line, controlling the decay of rewards near the target.

[0054] Furthermore, the cross-entropy method (CEM) is used in the action space. The optimal sequence is searched using the following parameters: 500 trajectories sampled per round, 3 iterations, and the top 10% of elite samples. The initial distribution is Gaussian. ,in Zero control,

[0055] In one specific embodiment, the algorithm steps include: Sampling phase: Generate K candidate action sequences from the current action. ; Evaluation phase: Cost of parallel computation for each trajectory ; Elite selection: Choosing the lowest cost Trajectories constitute an elite group ; Distribution Update: Based on Re-estimate the distribution parameters: (26) in, The sample set, typically representing the set of data points in the current batch or cluster, is represented by M, where M represents the number of samples in the set. Represents a set The first in A sample vector, This represents the updated mean vector. This represents the updated covariance matrix.

[0056] Finally, iterative convergence is achieved: steps S1 to S4 are repeated for a total of 3 rounds, and the final mean is output. The first action is used to execute.

[0057] The process of applying this invention to intelligent navigation is described in [reference needed]. Figure 3 As shown, the main components include: initializing the intelligent navigation system; acquiring multimodal sensor data; determining whether the safe distance to obstacles or the posture of elderly users meets the safety threshold; if so, emergency braking or adjustment of speed and direction; if not, planning a new path and updating the global path until the target point is reached.

[0058] It should be noted that, without departing from the spirit and scope of this invention, those skilled in the art can make appropriate changes or modifications to the above embodiments. For example, the environment coding module can use diffusion probabilistic models instead of traditional VAEs to achieve nonlinear mapping from observation data to the latent space, constructing a feature extraction chain through the inverse process of stochastic differential equations, and preserving the latent space representation ability of dynamic physical constraints. Alternatively, a multi-head attention mechanism can be used instead of RSSM for temporal dependency modeling, and a state predictor based on the Transformer architecture can achieve long-range context awareness and uncertainty quantification. Or, a policy gradient reinforcement learning (PRL) module can be used instead of the traditional MPC framework, based on an end-to-end control command generation method using value function estimation and action advantage analysis. All aspects involving the above-mentioned world model construction paradigm, latent state prediction, and online fine-tuning interaction mechanisms, and their engineering applications in predictive control systems, regardless of specific sensor configurations or deployment scenarios, are included within the scope of protection of this invention.

[0059] In summary, the technical advantages of this invention are mainly reflected in the following aspects: 1) A VAE-RSSM-MPC joint architecture was designed, which innovatively integrates variational inference, temporal modeling, and predictive control to form a closed-loop optimization loop. This is achieved by designing a VAE-based environmental observation compression coding and latent space reconstruction method, and by using RSSM to implement dynamic probabilistic modeling techniques for latent state sequence prediction and error propagation.

[0060] 2) A latent space contrastive fine-tuning algorithm was designed, based on an online update strategy of contrastive learning, which can adjust the latent state distribution of the world model in real time without interrupting system operation. All technical paths that drive the following processes through real-time data streams are protected: dynamic comparison criteria between latent state predictions and actual observations, backpropagation model fine-tuning process based on contrastive loss, and collaborative iterative optimization method for model parameters and inference strategies. Through the innovative real-time contrastive learning strategy, the system's adaptation speed to unknown conditions is improved compared to traditional solutions, quickly completing adaptation to new environments (traditional systems based on fixed world models require more than 30 minutes of retraining), greatly enhancing the dynamic response capability in complex scenarios.

[0061] 3) A dynamic compensation prediction mechanism was designed, embedding an environmental disturbance observer into the MPC module. By coupling the VAE's reconstruction error feedback, long-term prediction biases are automatically corrected. The dynamic environment adaptation enhancement scheme employs a collaborative architecture of online world model updates and control strategy replanning, including: an incremental feature extraction method for unsteady-state observation data streams, system-level reconstruction triggering logic driven by model prediction errors, and a multi-objective balancing technique between control stability and learning efficiency. The incremental online fine-tuning technique reduces memory consumption compared to mainstream reinforcement learning schemes, significantly improving data utilization efficiency and reducing hardware deployment costs.

[0062] 4) By utilizing the latent space compression capability of VAE, high-dimensional sensor data is mapped into low-dimensional temporal features, thus achieving efficient representation learning.

[0063] 5) RSSM innovatively introduces attention-guided trajectory sampling, which improves the credibility of multi-step predictions through importance weighting, and realizes an interpretable prediction mechanism.

[0064] 6) The MPC controller integrates an online comparative loss function, which enables 1200 policy fine-tunings per second through this incremental optimization strategy.

[0065] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0066] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0067] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0068] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0069] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0070] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0071] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0073] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A predictive control method based on a world model and online fine-tuning, comprising the following steps: Acquire multimodal environmental observation data, including point cloud data, RGB images, and IMU attitude angle information; The environmental observation data is input into a trained variational autoencoder network to predict the potential state; For the predicted potential states and candidate action sequences, a cyclic state-space model is used to predict the potential state sequence for subsequent multiple steps. Given the current action and multiple candidate action sequences, the optimal action sequence is searched in the action space with the goal of minimizing the total cost function of each trajectory.

2. The method according to claim 1, characterized in that, The overall loss function used to train the variational autoencoder network is set as follows: in, It is the loss from multimodal observation reconstruction. It is the potential space regularization loss. It is the prediction loss after adding the reward signal. for divergence coefficient, The prediction weight is used as a reward.

3. The method according to claim 2, characterized in that, The multimodal observation reconstruction loss Set to: in: in, It is the point cloud reconstruction loss. It is structural similarity loss. It is the IMU attitude and path reconstruction loss. , and These are the weighting coefficients of the corresponding terms. It is about reconstructing point clouds. It is a real point cloud, where p represents the set of real point clouds. A three-dimensional point, Represents the reconstructed point cloud set A three-dimensional point, It is to reconstruct the image. It is a real image. It is the actual IMU pitch angle. It is the predicted IMU pitch angle at time step t. It is the actual IMU roll angle at time t. It is the predicted IMU roll angle at time step t. Indicates the number of time steps. This represents the x-coordinate of the actual path point at time step t. This represents the x-coordinate of the predicted path point at time step t. This represents the y-coordinate of the actual path point at time step t. This represents the y-coordinate of the predicted path point at time step t.

4. The method according to claim 2, characterized in that, The potential space regularization loss is set as follows: in, Let i represent the Kullback–Leibler divergence, where i represents the i-th vector in the latent vector. Indexes in 10 dimensions The dimension of the potential space, This represents the latent variables in the latent space. Indicates the observed input, Indicates time step t The mean vector of the latent states, Indicates time step t The standard deviation vector of the latent states.

5. The method according to claim 2, characterized in that, The prediction loss with the added reward signal is set as follows: in, This represents the true reward signal at time step t+1. This represents the predicted reward signal at time step t+1.

6. The method according to claim 1, characterized in that, The total cost function is set as follows: in: in, It is path tracking error. It is the cost of controlling quantity. It is the proximity to the obstacle. It is the smoothness of motion. It's a reward for the endgame. Indicates from predictive observations The current position coordinates extracted from [the data]. Indicates the reference path at time step The target location coordinates Indicates a safe distance. This indicates the distance between the predicted location and the nearest obstacle. It is a set constant. Indicates the width of the finish line reward distribution. This represents the action at time step t+k-1. Indicates the time step of the prediction Observations.

7. The method according to claim 2, characterized in that, It also includes fine-tuning the variational autoencoder network through online fine-tuning and contrastive learning mechanisms, wherein the contrastive loss is set as follows: in, Temporally adjacent potential states Construct positive sample pairs and randomly select states from non-adjacent time steps in the buffer. Construct a negative sample pool, where B represents the batch size. Indicates the temperature coefficient. Let represent the similarity measure function between two states in the latent space, where i represents the index of the positive sample pair in the current batch, and j represents the index of the negative sample in the negative sample pool. This indicates the number of negative samples in the negative sample pool. This represents the time step offset when constructing positive sample pairs.

8. The method according to claim 7, characterized in that, During the online fine-tuning and contrastive learning mechanism, a parameter importance penalty term is added to the overall loss function. : in: in, Represents the elastic coefficient. Indicates the first One training parameter, This indicates the first stage corresponding to the pre-training phase. Parameter values, This indicates the old task dataset. Take the expected value. ) indicates input Lower prediction output The probability distribution, x Let represent the input sample of the old task, and y represent the corresponding label of the old task. Indicates parameters Importance indicators.

9. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images