A method, apparatus and control system for correcting control signals

Abstract

The application discloses a one-step denoising control signal correction method, device and system based on multi-modal conditions and control memory enhancement. The method aims at the problem of insufficient robustness caused by noise, deviation and working condition changes of the control signal output by the upstream controller. In each control cycle: a fusion representation is generated by multi-modal observation, and global control planning information is generated; a memory abstract is retrieved and generated in the control signal memory based on the fusion representation and the planning information; the short-term intention prior is inferred and the noise intensity parameter is estimated in combination with the noisy input control signal; and the noisy input control signal, the noise intensity parameter and the conditional input are input into a one-step denoising control correction model, and the corrected control signal is directly output after one forward inference. Compared with the prior art, the application improves the self-adaptive ability and safety controllability in a multi-noise environment while maintaining pluggable deployment and real-time performance.

Description

Technical Field

[0001] This invention relates to the fields of intelligent control, robotics and automation, and artificial intelligence generative models, specifically to a method, apparatus, and control system for correcting control signals. The invention also relates to a training method for a one-step denoising correction model used in the above method. Background Technology

[0002] In automatic control systems such as robots, autonomous driving, and industrial equipment, a "perception-planning-control" chain is typically used for control. The system acquires environmental and its own state information through multiple sources of sensors, and forms a control-related representation through state estimation and information fusion. The planning module generates global or local references (such as trajectories, target state sequences, or control references) based on task objectives and constraints. Subsequently, upstream controllers (including but not limited to proportional-integral-derivative (PID) control, robust control, model predictive control (MPC), learning-based control strategies, etc.) output control signals to drive actuators to complete the task.

[0003] In real-world engineering scenarios, the control signals output by upstream controllers are often inevitably affected by various uncertainties, resulting in noisy, biased, or insufficiently robust input control signals. Typical influencing factors include: sensor noise, clock asynchrony between different sensors, communication delays, and obstructions / missing fields of view leading to inaccurate state estimation, causing the controller to output suboptimal control based on the biased state; discrepancies between the real system dynamics and the model, such as load variations, contact friction variations, actuator saturation, mechanical wear, and increased clearance, causing the controller to generate systematic errors under model mismatch conditions; and distributional deviations in the control strategy, such as the controller being designed / trained on simulation or limited operating condition data, while the deployment environment contains uncovered disturbance modes and scenario combinations, resulting in unstable output control signals or signals that do not conform to expert control principles.

[0004] To address the aforementioned issues, existing technologies typically employ several types of compensation or correction measures, such as signal smoothing based on filtering and estimation, robust enhancement based on gain tuning, constraint optimization based on MPC, and compensation strategies based on residual learning / imitation learning. In recent years, generative models have also attracted attention in the generation and correction of control signals: for example, using diffusion-based generative models to model the strategy as a conditional denoising generation process to generate action sequences from observation conditions; and in shared control / assisted control scenarios, treating input actions as "noisy actions" and obtaining action outputs closer to the expert distribution through diffusion denoising mapping.

[0005] While the aforementioned techniques have improved the stability and availability of control systems to some extent, existing solutions still suffer from insufficient robustness and adaptability when facing complex real-world environments and the need for pluggable engineering deployments. Traditional filtering, gain tuning, or fixed compensation strategies typically rely on specific noise models or assumptions about specific operating conditions. When noise intensity varies over time, disturbances originate from diverse sources, and exhibit multimodal characteristics (e.g., the superposition of visual occlusion and actuator wear, environmental changes and task switching), existing methods struggle to maintain stable correction effects under different noise levels, easily leading to under-compensation or over-compensation.

[0006] In addition, existing solutions have other problems such as being unable to be easily integrated into existing control systems; being prone to selecting the wrong control mode in ambiguous scenarios, causing the output action to deviate from the expected target; and being unable to leverage successful experience to achieve adaptive enhancement in repetitive scenarios, similar working conditions, or long-term operation, resulting in limited cross-scenario generalization ability.

[0007] Based on the current state and shortcomings of existing technologies, a new control signal correction scheme is needed. This scheme can perform one-step correction of the noisy control signal output by the upstream controller without changing the main structure of the existing control system, so as to meet the requirements of real-time control and engineering deployment in complex scenarios. Summary of the Invention

[0008] To address the issue of excessive delay in existing control system calibration schemes without modifying the original controller, this invention provides a control system for calibrating noisy input control signals output by the upstream controller at discrete control moments. Compared to the multi-step calibration of hundreds of milliseconds to seconds in existing technologies, this method only requires a forward calculation of a few milliseconds to tens of milliseconds during the deployment phase to complete the calibration, significantly improving real-time performance and meeting the requirements of high-frequency control cycles.

[0009] In one aspect, the present invention provides a method for correcting control signals, comprising the following steps: acquiring multimodal sensor observations; calculating a multimodal fusion representation based on the multimodal sensor observations; generating global control planning information based on the multimodal fusion representation; generating a memory retrieval key based on the multimodal fusion representation and the global control planning information; performing a retrieval in a control signal memory bank based on the memory retrieval key and generating a memory summary; acquiring a noisy input control signal output by an upstream controller; inferring a short-term intent prior based on the multimodal fusion representation, the global control planning information, the memory summary, and the noisy input control signal; constructing correction conditions using the multimodal fusion representation, the global control planning information, the memory summary, and the short-term intent prior; estimating a noise intensity parameter based on the noisy input control signal and the correction conditions; and inputting the noisy input control signal, the noise intensity parameter, and the correction conditions into a one-step denoising correction model to perform a forward inference, thereby directly outputting a corrected control signal.

[0010] Advantageously, the method further includes the following steps: performing security verification or constraint processing on the correction control signal and issuing it for execution; and writing the entries containing the memory retrieval key, the noisy input control signal, the correction control signal, the short-term intent prior, and the feedback quality information back to the control signal memory bank based on the execution feedback, so as to update subsequent retrieval and summary calculation.

[0011] In one specific embodiment, multimodal sensor observations include two or three of the following: language information, visual information, hardware sensor parameters, or attitude information.

[0012] In one specific embodiment, the control signal memory uses a data structure that associates index keys with index content for storage. The index keys are associated with at least the conditional input, control input, and short-term intent fields, and the index content includes at least the control output, control intent, and local state update fields, and is associated with a timestamp field.

[0013] In one specific embodiment, generating a memory summary includes Top-K nearest neighbor retrieval and ranking based on similarity metrics to obtain candidate entries, and quality filtering or reordering of the candidate entries.

[0014] In one specific embodiment, the memory summary is obtained by weighted aggregation of candidate items, wherein the weights are related at least to the similarity score and to the item quality label and / or freshness weight.

[0015] In one specific embodiment, the short-term intent prior is any one or a combination of short-term sub-target encoding, short-term reference direction encoding, short-term state change trend encoding, and control modality labeling, wherein the intent component of the short-term intent prior is selectively or weighted based on contact / collision risk, perception confidence, or planning deviation.

[0016] In one specific embodiment, the noise intensity parameter is a continuous scalar or discrete level value limited to a preset range, and is estimated by a rule function, a learning model, or a combination of both, wherein the rule function is calculated based on at least one of the following: the amplitude of the input control signal change, the planning deviation risk, the stability risk, and the contact / collision risk.

[0017] In one specific embodiment, the safety verification or constraint processing includes any one or a combination of amplitude limiting processing, quadratic programming projection processing, and barrier function filtering processing, so that the issued control signal satisfies the safety constraints associated with speed, torque, rotation angle, acceleration, and collision distance.

[0018] Optionally, the method further includes the following steps: when a perception anomaly is detected, the risk of constraint violation increases, or the memory hit rate decreases, a degradation strategy is triggered. The degradation strategy includes bypassing a one-step denoising correction model and directly outputting the control signal of the upstream controller, or switching to a more conservative set of memory entries to generate a memory summary.

[0019] In another aspect, the present invention provides a training method for a one-step denoising correction model according to the above-described method, comprising the following steps: obtaining a training dataset containing multimodal observations and target control signals; calculating multimodal fusion representations and global control planning information for the training samples, and constructing a memory retrieval key and a memory summary; constructing a noisy input control signal for the target control signal according to a preset noise schedule; training a control correction teacher model to learn to output a target control signal from the noisy input control signal under conditional constraints; generating trajectory sample pairs corresponding to different noise levels based on the teacher model, and performing consistency distillation training on the control correction student model to enable the student model to acquire the ability to output a correction control signal through forward inference; and training a short-term intent inference module and a noise intensity estimation module for use in the online inference stage for conditional construction and adaptive noise intensity estimation.

[0020] Advantageously, during the generation of trajectory sample pairs corresponding to different noise levels based on the teacher model, the generation of trajectory samples by the teacher model introduces prior guidance, wherein the prior guidance includes at least one of planning consistency guidance, memory attraction guidance, and safety constraint guidance.

[0021] In one specific embodiment, the consistency distillation training includes both output consistency constraints across noise levels and supervisory constraints with the target control signal, and only one inference step is performed using the student model during online deployment.

[0022] In another aspect, the present invention provides a control signal correction device, comprising: a multimodal feature extraction and fusion module for generating a multimodal fusion representation; a global control planning module for generating global control planning information; a control memory module for generating a memory retrieval key and performing retrieval and summary calculation to output a memory summary, and for memory write-back updates; a local intent inference module for outputting short-term intent priors and constructing correction conditions; a noise level estimation module for estimating noise intensity parameters; a control denoising correction module for inputting a noisy input control signal, noise intensity parameters, and correction conditions into a one-step denoising correction model and outputting a corrected control signal through one forward inference; and a control output and communication interface for outputting the control signal to be executed, wherein each module cooperates to implement the above-mentioned method for correcting the control signal.

[0023] The control signal correction device also includes a control constraint module, which is used to perform safety verification or constraint processing on the correction control signal.

[0024] The control signal memory module stores historical entries based on a data structure that associates index keys with index content, and generates memory summaries using Top-K nearest neighbor retrieval, quality filtering, and weighted aggregation.

[0025] A control system includes: a multimodal sensor, an upstream controller, an actuator, and the aforementioned control signal correction device, wherein the noisy input control signal output by the upstream controller is processed by the control signal correction device and then output to the actuator for execution, and the execution feedback is used to update the control signal memory.

[0026] Additional features and advantages of the methods, apparatus, and control systems for correcting control signals described herein will be set forth in the detailed description below, and will be recognized by those skilled in the art either by the following description or by practice of the embodiments described herein, including the detailed description below and the accompanying drawings. Attached Figure Description

[0027] In view of the above objectives, the technical features of the present invention are clearly described in the following specific embodiments, and its advantages are apparent from the following detailed description with reference to the accompanying drawings, which illustrate preferred embodiments of the invention by way of example, without limiting the scope of the inventive concept.

[0028] Figure 1 A functional block diagram of a control signal correction device for a control system according to a specific embodiment of the present invention is shown.

[0029] Figure 2 A schematic diagram of the architecture of a control signal correction device for a control system according to a specific embodiment of the present invention is shown.

[0030] Figure 3 A schematic diagram of the data structure and index architecture of the control signal memory of a control signal correction device for a control system according to a specific embodiment of the present invention is shown.

[0031] Figure 4 A flowchart is shown showing a method for correcting control signals using a control signal correction module of a control signal correction device for a control system according to a specific embodiment of the present invention.

[0032] Figure 5 A schematic diagram illustrating the retrieval and update process of the control signal memory bank of the control signal correction device for a control system according to a specific embodiment of the present invention is shown.

[0033] Figure 6 A schematic diagram illustrating the training process of a one-step denoising correction model for a control system according to a specific embodiment of the present invention is shown. Detailed Implementation

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention in any way.

[0035] Terms and Definitions

[0036] This invention addresses the engineering goal of "pluggable integration into existing control systems" by providing a one-step noise reduction control signal correction plug-in. Located between the "upstream controller output" and the "actuator input" in the control system, this plug-in takes a noisy input control signal as input and, by combining multimodal conditions, global planning, and historical prior knowledge, outputs a one-step corrected expert control signal.

[0037] To facilitate understanding of the overall technical solution of the present invention, the following description is based on the system architecture (e.g., the appendix). Figure 1 The functional modules, information flow between modules, and key objects of the present invention are defined and explained as shown in the figure.

[0038] The terms used below are defined as follows: "Short-term intention prior" is not a probabilistic prior, but rather an encoded vector of local target, direction, or mode inferred from the current context, used to constrain the selection of control mode for the correction output.

[0039] The "correction conditions" specifically include multimodal fusion representation, global control planning information, memory summarization, and short-term intent priors (i.e., below). ).

[0040] A "memory summary" is a fixed-dimensional vector obtained by weighted aggregation (such as weighted average, prototype clustering, etc.) of the retrieved candidate items.

[0041] "Freshness weight" should be understood as time freshness, such as exponential decay exp(-Δt / τ) based on timestamps.

[0042] In this invention, "conditional adjustment" refers to conditional branching operations that enhance, suppress, or zero certain components of the intent vector based on risk or confidence signals.

[0043] "Consistent distillation training" refers to using the probability flow ordinary differential equation trajectory generated by the teacher model to train the student model so that the output is consistent for inputs with different noise levels on the same trajectory, thereby achieving one-step inference.

[0044] The "noise intensity parameter" characterizes the degree to which the input control signal deviates from the expert control signal and is used to control the intensity of correction intervention.

[0045] "Local state update" includes end-effector pose increment, stability index changes, peak force, etc.

[0046] The term "expert" is used to describe high-quality, idealized control signals or control strategies.

[0047] To ensure consistency and unambiguity in subsequent descriptions, the key objects and symbols involved in this invention are defined as follows. This invention does not limit the specific names of the symbols, but their physical meanings and correspondences should remain consistent.

[0048] 1. Time Index and Control Dimension

[0049] Let the discrete control time be .

[0050] Control signals (such as joint speed / torque, vehicle steering angle / acceleration, actuator input, etc.) are vectors.

[0051] in d u To control the dimension.

[0052] 2. Noisy input control signal and corrected output control signal

[0053] The noisy / biased input control signal output by the upstream controller is defined as follows:

[0054] The term "noisy / biased" includes, but is not limited to: control errors caused by state estimation errors, systematic deviations caused by dynamic uncertainties, and unstable outputs caused by distribution shifts.

[0055] The correction control signal output by the one-step correction module of this invention is defined as follows:

[0056] Its goal is to statistically approximate the "expert control signal" (see below for details). (Definition), and can satisfy planning consistency and security constraints.

[0057] 3. Expert control signals (used for training / calibration)

[0058] The expert control signal used for training or calibrating a consistency denoising model is defined as follows:

[0059] Expert control signals can originate from manual teaching, remote operation trajectories, or trajectories generated by high-quality controllers, without being limited by the source format.

[0060] 4. Multimodal observation and fusion characteristics

[0061] Multimodal observation set is defined as These correspond to language, vision, and hardware sensor parameters / attitude, respectively.

[0062] Multimodal fusion features are defined as

[0063] in This represents a fusion coding mapping, which can be implemented by neural networks or other information fusion algorithms.

[0064] 5. Global Planning Information

[0065] Global / local planning information is defined as follows

[0066] in Represents a planning mapping. It can represent path, reference trajectory code, target state code, or a combination thereof.

[0067] 6. Control signal memory and retrieval summary

[0068] The control signal memory bank is defined as a set or database.

[0069] in Representing historical context keys (e.g., features fused from multiple modalities) as well as structure), This indicates the performance feedback or evaluation metrics (such as success / failure, cost, degree of constraint violation, etc.).

[0070] Memory retrieval summary is defined as

[0071] in For retrieval and abstract mapping, This is the current search key.

[0072] 7. Prior local / short-term intentions

[0073] Local goals / short-term intentions prior definition

[0074] in This is an intent inference mapping. Local intents can be used to represent short-term sub-objectives, short-term reference directions, or short-term state change trends, and are used to constrain the selection of control modes for the correction output.

[0075] 8. Condition set and noise level / intervention intensity

[0076] The general set of conditions for control correction in this invention is defined as follows:

[0077] Noise level / intervention intensity is defined as a nonnegative scalar

[0078] in This is a mapping or rule function for noise level estimation. Used to characterize the defect strength of the input control signal and to adjust the correction strength (e.g., to achieve continuous adjustment from “weak intervention” to “strong intervention”).

[0079] 9. One-step noise reduction control correction mapping (the innovative output formula of this invention)

[0080] The output of the one-step correction module is defined as follows:

[0081] in This is the trained one-step denoising consistency model / correction model. This definition clarifies the core interface of this invention: the input is... (Noisy control signal) and conditional information, output as (The corrected expert control signal), rather than the output "correction amount".

[0082] 10. Optional safe feasible region and projection operator

[0083] The safe and feasible region is defined as a set. .

[0084] The projection / filtering operator is defined as follows:

[0085] in This is the final control signal issued for execution.

[0086] Through the above overall architecture and key object definition, the present invention clearly and fixes the meaning of variables and interface relationships in subsequent method steps, device structure and training scheme, thereby ensuring that the descriptions in each section of the patent specification are consistent, implementable and easy to extract claims.

[0087] Overall Architecture

[0088] like Figure 1 As shown, the correction system for one-step denoising control signals of the present invention includes: sensor fusion features, global control planning, local control signals, local control correction signals, and a control signal memory. The sensor fusion features are used to generate fusion features from information such as language, vision, and hardware sensors / attitude, and provide condition updates to the global control planning and local control correction modules. The global control planning is used to generate planning conditions based on the fusion features and output them to the local control signal module. The local control signal module outputs the noisy control signal generated by the upstream controller and provides it as a correction input to the local control correction signal module. Under the common conditions of fusion features, planning conditions, and memory summaries, the local control correction signal module performs one-step denoising correction on the noisy input control signal and outputs the corrected control signal. The control signal memory is used to store historical control segments and their context information, and forms a closed loop with the local control correction signal module: on the one hand, it provides memory retrieval and summaries to the module; on the other hand, it receives correction results and feedback information to update the memory, thereby achieving adaptive enhancement as the environment and operating conditions change.

[0089] In a specific embodiment, the present invention provides a control system, which includes, but is not limited to: 1. Multimodal sensor subsystem: Used to acquire observations of speech, vision, and hardware sensor parameters / attitude, etc. .

[0090] 2. Upstream controller subsystem: Used to output the control signal to be calibrated. .

[0091] 3. Control signal correction device: Receiver Output with conditional information or The device will be described in detail below.

[0092] 4. Actuator Subsystem: Receives... And execute the action, outputting execution feedback. .

[0093] 5. External Planning Subsystem: Outputs planning information For use in the device of this invention.

[0094] 6. Safety Supervision Subsystem: Provides safety constraint parameters and handles anomalies (such as degradation strategies, emergency braking, etc.).

[0095] In terms of system connectivity, the calibration device of this invention is deployed between the "upstream controller output end and the actuator input end," forming a pluggable control link: when the calibration device is activated, the actuator receives... When the calibration device is bypassed / closed, the actuator can directly receive... This satisfies the engineering requirements for rollback strategies and security audits (this bypass mechanism can be used as an example, without limiting the scope of protection).

[0096] Control signal correction device

[0097] The “modules / units” described in this section can be implemented by hardware circuits, by a processor executing program instructions stored in a storage medium, or by a combination of hardware and software.

[0098] exist Figure 2 In one specific embodiment shown, the control signal correction device of the control system of the present invention includes at least the following functional modules: 1. Multimodal Feature Extraction and Fusion Module: This module receives multimodal observations from language, vision, and hardware sensor parameters / attitudes (e.g., joint angles / velocities, IMU, force / torque, odometer, environmental sensors, etc.) and outputs a unified fused representation. This fused representation characterizes the current task and environmental state, serving as the foundational information for subsequent intent inference and control correction.

[0099] 2. Global Control Planning Module: This module generates global or local planning information (e.g., global path, reference trajectory, sub-objective sequence, control reference window, etc.) based on fused representations. This module can be implemented using existing planners / predictors, without limitation to any specific algorithm. The planning information serves as input to the intent inference and correction modules as high-level constraints, ensuring that the correction output aligns with the task objectives and the planning.

[0100] 3. Upstream Controller (original control module, not shown): Used to output the input control signal to be corrected. This upstream controller can be a traditional controller (PID, feedforward + feedback, etc.), a model predictive controller (MPC), a learning control strategy, or any other type of control module. One of the core features of this invention is that control signal correction can be achieved simply by using its output control signal as the input to the plug-in of this invention, without modifying the internal structure of the upstream controller.

[0101] 4. Local Intent Inference Module: This module fuses the current fusion representation, global planning information, and memory retrieval summary, and combines this with (optional) input control signals to output a "local goal / short-term intent" prior. This intent prior is used to reduce the risk of incorrect control mode selection under multimodal ambiguity conditions, enabling the corrected output to more stably align with task semantics and planning direction.

[0102] 5. Control Memory Module: This module stores historical control contexts and execution results, and retrieves them based on the current context, outputting a memory summary. The memory can store associated data across "context—input control—correction output—execution feedback," and supports retrieval by similarity, task tags, or success metrics. This module enables the correction process to have reusable experience and cross-scenario enhancement capabilities.

[0103] Reference Figure 3 The control signal memory is used to store historical control segments and supports fast index-based retrieval and updates. Its structure consists of two parts: an index key and associated index content. The index key is used to establish retrieval entries and form associations with multiple types of fields, while the index content is used to store historical information that can be retrieved and reused.

[0104] Specifically, the index key is associated with at least the following fields: conditional input, control input, and short-term intent; wherein, the conditional input is used to characterize the contextual conditions required to generate or correct the control signal, and may further include at least one of perceptual features, global planning (optional), and scene description (optional); the control input is used to record the noisy control signal output by the upstream controller; the short-term intent is used to record local target / intent information inferred from the conditional input and control input, so as to distinguish different control modes and improve matching accuracy during retrieval. Figure 3 The hierarchical relationship shown on the left indicates that the index key can be generated by or in combination of conditional input, control input, and short-term intent. The conditional input can be further composed of perceptual features and optional global planning and scene description.

[0105] The index content includes at least: control output, control intent, and local state update, and can be associated with timestamp information; wherein, control output is used to record the control signal obtained after one step of denoising correction or the corresponding high-quality / expert control signal; control intent is used to record the intent information or control mode summary corresponding to the control output; local state update is used to record the local state changes or effect feedback information after control execution, so as to support subsequent quality assessment and adaptive updates; timestamp is used to record the time sequence information of the entries, so as to facilitate filtering and updating by freshness, operating condition or version. Figure 4 The hierarchical relationship shown on the right indicates that the index content can be composed of control outputs, control intentions, and local state updates, and can be supplemented with timestamps for management and backtracking.

[0106] With the above structure, the control signal memory can generate an index key based on the current condition input and control input during online inference and retrieve the matching index content to form a memory summary for short-term intent inference and one-step denoising correction. At the same time, after control execution, the control output and feedback information are written back to the index content to realize the closed loop of the memory.

[0107] 6. Noise Level Estimation Module (Adaptive Intervention Module): This module estimates the noise level or defect severity based on the current context and input control signal, and determines the correction intervention strength accordingly. It achieves a controllable trade-off between "faithful input control" and "close to expert control": intervention is reduced when the input signal quality is good, and intervention is increased when the input signal quality is poor or under abnormal disturbance.

[0108] 7. Control Signal Correction Module (Main Innovation of this Invention): This module receives noisy input control signals and the aforementioned conditional information (fusion features, planning, memory summary, intent prior, and noise level), and outputs a corrected control signal through a single forward inference. Based on the concept of consistent denoising / probabilistic flow trajectory distillation, this module achieves one-step generation of a single function evaluation, thereby meeting the low latency requirements of real-time control loops.

[0109] 8. (Optional) Control constraint module

[0110] This module is used to perform safety limiting, feasible region projection, or barrier filtering on the correction output, ensuring that the control signals ultimately sent to the actuators meet system safety constraints (such as speed / torque limits, collision constraints, road safety distance constraints, etc.). This module can be deployed as an independent hardware / software safety layer, decoupled from the one-step correction module.

[0111] 9. (Optional) Closed-loop update module

[0112] It is used to write the control signals and execution feedback after execution back to the memory bank, and can be used to update the parameters / hyperparameters (such as thresholds, retrieval indexes, memory eviction strategies, etc.) of the intent inference module, noise level estimation module, or correction module to achieve experience iteration and performance maintenance during long-term operation.

[0113] Through the synergy of the above modules, this invention forms a closed-loop structure at the system level of "multimodal conditions - planning constraints - memory priors - one-step correction - security filtering - memory write-back", realizing pluggable one-step expert control signal correction.

[0114] Correction method

[0115] Reference Figure 4 This invention provides a one-step control signal correction method based on generative denoising. This method maps the noisy input control signal output by the existing upstream controller into a corrected control signal that more closely approximates the distribution of expert control signals, without altering the existing upstream controller structure. It integrates multimodal conditions, global planning constraints, and historical control memory to achieve controllable intervention and closed-loop self-updating. The following is a structured description of the method's flow in conjunction with the discrete control time k. This method can be implemented by a processor executing instructions stored in a storage medium, or it can be implemented using a combination of hardware and software.

[0116] Step S1: Acquire multimodal observations and generate fused representations

[0117] At any moment Obtain the system's multimodal observation set

[0118] in Represents language instructions or text descriptions. Represents a visual image / video or its features. This indicates the parameters / attitude of hardware sensors (including but not limited to joint angles / velocities, IMU, odometer, force / torque, environmental sensors, etc.).

[0119] Through multimodal fusion mapping Generate fusion representation:

[0120] in It can be implemented as a neural network encoder, feature splicing and projection, attention fusion or other information fusion methods, and the present invention does not limit its specific implementation form.

[0121] Step S2: Obtain global control planning information

[0122] Based on fusion representation Mapped by planning Output global or local planning information:

[0123] in It can represent path / trajectory reference coding, sub-target sequence coding, target state coding, or a combination thereof. The planning module in this step can be implemented using existing planners or predictors. The key to this invention is to use the planning information as a constraint input for subsequent intent inference and control correction. It can be generated internally by the device or input from an external planner.

[0124] Step S3: Construct memory retrieval keys and generate memory summaries

[0125] Construct a memory retrieval key based on the current fusion representation and planning information:

[0126] in Generate a mapping for the search key, which can be concatenation, projection, or other mapping methods.

[0127] In the control signal memory bank Perform the search and summary generation in the middle to obtain the memory summary:

[0128] in This represents the mapping between retrieval and summary, which can be implemented as similarity-based nearest neighbor retrieval, clustering prototype retrieval, hash index retrieval, etc. This step enables historical successful control experience to participate in current control correction decisions in a searchable form.

[0129] Step S4: Obtain the noisy input control signal output by the upstream controller.

[0130] The upstream controller outputs the input control signal to be corrected:

[0131] in It may contain noise and bias, the sources of which include, but are not limited to, state estimation errors, dynamic uncertainties, environmental disturbances, or distribution shifts. This invention does not require changes to the internal structure of the upstream controller; it merely uses its output as the input to the correction module of this invention.

[0132] Step S5: Infer local target / short-term intention prior

[0133] By integrating current fusion representations, planning information, and memory summaries, and combining (optionally) input control signals, calculate the local target / short-term intent prior:

[0134] in To infer the mapping for the intention, It can be used to represent short-term sub-targets, short-term reference directions, short-term state change trends, or combinations thereof.

[0135] Further, a general set of conditions for controlling the correction is formed:

[0136] The purpose of this step is to: when multimodal inputs and planning constraints cause the control solution space to exhibit multimodal characteristics, explicitly introduce... As a priori condition, the control mode selection of the correction output is constrained, reducing the risk of erroneous actions in ambiguous scenarios.

[0137] Step S6: Estimate the noise level / intervention intensity and determine the correction magnitude

[0138] Estimating noise level or intervention intensity based on noisy input control signal and condition set:

[0139] in It can be a rule function or a learning model, used to characterize the defect strength of the input control signal and determine the degree of correction intervention accordingly. Through With adaptive estimation, this invention can achieve a continuously adjustable trade-off between the following two types of cases: when When the value is small, it maintains high fidelity to the input control signal, and the correction intervention is weak; when When the value is large, strengthen the correction of the input control signal to make the output closer to the expert control law.

[0140] It can be calculated by rules, output by a learning model, or a combination of both (e.g., rules provide upper and lower bounds, and learning provides refined values).

[0141] Furthermore, when the control dimension is multi-channel (e.g., multi-joint / multi-wheel / multi-actuator), the device can estimate conditions based on channel sharing or separately for each channel. It also outputs channel correction control signals.

[0142] Step S7: One-step noise reduction correction outputs expert control signal

[0143] The noisy input control signal is modeled as an expert control signal at a noise level. The following perturbation observations indicate the presence of expert control signals. Make:

[0144] Based on this, the mapping is corrected through one-step denoising control. Output correction control signal:

[0145] in The output of the trained one-step denoising consistency correction model is... Instead of outputting a "correction amount," the corrected expert control signal is directly used as the input control signal. Through this step, the present invention achieves one-step inference and one-step correction of the input control signal to meet the low latency requirements of the real-time control loop.

[0146] Step S8: Projection and Control of Safety Constraints / Feasible Region

[0147] When the system has safety constraints or feasible region limitations, the correction output is mapped to the safe feasible region. :

[0148] in This is a projection / filtering operator, which can be implemented as a limiting, constraint-optimized projection, or barrier filtering operator. Ultimately, it will... Issue to the implementing agency; if this step is not enabled, it can be issued directly. .

[0149] Step S9: Close the loop, write back to memory, and update the index.

[0150] Obtain execution feedback or evaluation metrics (For example, task success markers, costs, constraint violation levels, trajectory errors, etc.), and write the current context key, input control, output control, and feedback into the memory bank to achieve experience accumulation and subsequent retrieval and reuse:

[0151] In an optional embodiment, it can be based on Memory entries are filtered, weighted, eliminated, or clustered and updated to improve memory retrieval quality and maintain long-term operational stability. It can be deployed on local storage (edge), vehicle storage, control cabinet storage, or cloud storage and synchronized through caching; retrieval can be completed locally or on the edge server.

[0152] like Figure 5 As shown, the control signal memory retrieval and update process of the present invention includes four stages: retrieval key calculation, library retrieval sorting, output aggregation, and library retrieval writing.

[0153] In the retrieval key calculation stage, the condition information at the current moment is first embedded with features. The condition information includes at least perceptual feature embedding and optionally further includes global planning matching and scene description embedding, thereby obtaining the retrieval key for accessing the memory bank.

[0154] During the database retrieval and sorting stage, a set of candidate entries is obtained by matching the retrieval key in the memory based on the retrieval key, and the candidate entries are retrieved and sorted. After sorting, the input control signal is located to determine the range of candidate entries or candidate segments associated with the current input control signal.

[0155] In the output aggregation stage, the sorted candidate items are aggregated to generate a memory summary; the aggregation process can be further combined with the feature embedding of local state update records to enhance the matching of the current working condition; at the same time, the short-term intentions and control outputs of the candidate items can be sampled or selected to form a memory summary output for subsequent short-term intention inference and one-step denoising control correction.

[0156] During the database retrieval and writing phase, the local state update corresponding to the current execution feedback is written to the database, and the short-term actual intention and control output are written to the database to complete the entry update or new entry writing, so that the subsequent retrieval phase can utilize the latest control experience and form a closed-loop self-update.

[0157] Step S10 (optional): Apply planning / memory / safety constraints to the correction process in a priori form.

[0158] In an alternative embodiment, planning information can be used as a basis. Memory Summary Constructing a priori energy function with safety constraints During the training phase, this prior knowledge is used to constrain or guide the learning of the correction model, ensuring that the one-step correction output internalizes planning consistency, empirical preferences, and safety feasibility requirements at the generation mechanism level. The unit executing safety constraints can be deployed inside the control signal correction device or implemented by the system's existing safety monitoring module.

[0159] Through the above steps, the present invention forms a complete, implementable and easily integrated engineering method: taking a noisy input control signal as the core input object, integrating multimodal representation, planning constraints and historical memory, inferring local intention priors and adaptively estimating noise / intervention intensity, and finally outputting an expert-level control signal by a one-step denoising correction model, and optionally combining safety filtering and memory closed-loop updates, thereby solving the key problems of existing technologies in terms of real-time performance, robustness, deployability and long-term adaptability.

[0160] Training methods

[0161] Reference Figure 6 The present invention also provides a training method for training a one-step correction module for control signals. The model obtained through this training method can be deployed under a given set of conditions. and noise level / intervention intensity At the same time, for noisy input control signals Perform one forward inference and directly output the correction control signal. ,satisfy:

[0162] in This is the one-step denoising control correction model trained in this invention.

[0163] In one embodiment, the training method includes the following steps (T1) to (T6).

[0164] (T1) Obtain training data (expert control dataset)

[0165] Obtain or build an expert control dataset:

[0166] in For multimodal observation, For planning information, This refers to expert control signals. These signals may originate from teaching data, offline optimized trajectories, or validated high-quality control strategy outputs (the source format is not limited).

[0167] (T2) Generate the condition information required for training

[0168] For each training sample, generate the condition set according to the link generation conditions consistent with the deployment phase, excluding... The outer part: 1) The fusion representation is obtained from the fusion mapping:

[0169] 2) If the training samples are not directly provided Then it is generated by the planning mapping:

[0170] 3) Construct search keys and obtain memory summaries:

[0171] (T3) Constructing noisy training samples

[0172] To simulate the noisy control signal output by the upstream controller during the deployment phase, this invention modifies the expert control signal during the training phase. Add noise disturbance to construct a noisy input control signal : 1) Sampling noise scale:

[0173] 2) Sampling noise vector:

[0174] 3) Construct a noisy input control signal:

[0175] 4) After obtaining Then, local intent priors are generated and a complete set of conditions is formed: .

[0176] (T4) Training the teacher denoising model

[0177] This invention constructs and trains a teacher denoising model during the training phase. It is used to provide a denoising reference from noisy control signals to expert control signals, so that "sample points with different noise levels on the same trajectory" can be generated subsequently based on the probability flow trajectory.

[0178] In one embodiment, the teacher denoising model is trained by minimizing the following objective function:

[0179] The desired outcome is to average the training data samples and the noise sampling process described in (T3).

[0180] (T5) Generate teacher trajectory samples based on PF-ODE and train a one-step model using consistency distillation.

[0181] This step is the key to the training method of this invention: using PF-ODE to generate sample points on the same denoised trajectory between different noise levels, and training a one-step model based on this to output consistent results under different noise inputs on the same trajectory, thereby achieving one-step inference in the deployment stage.

[0182] (T5.1) Set the noise level sequence and trajectory generation interval

[0183] Define a monotonically decreasing noise level sequence:

[0184] in Located near the upper bound of the training noise, The noise level is close to 0 or 0.

[0185] (T5.2) Define the PF-ODE trajectory generation operator and generate sample points of the same trajectory.

[0186] Given a teacher model With conditions In this case, a pre-defined numerical integrator for probability flow ordinary differential equations (which can be Euler, Heun, or other single-step / multi-step integrators; the implementation form is not limited) is used to input... Along the noise level from Integrating to a lower noise level yields a series of intermediate samples along the same trajectory:

[0187] in Can be generated from (T3) , This represents a single-step integral update operator based on the probability flow ordinary differential equation.

[0188] (T5.3) Introducing prior energy to guide the trajectory

[0189] In an alternative embodiment, a priori energy function can be constructed to make the trajectory more consistent with planning, memory preference, or safety requirements. and in trajectory generation operator A guiding term is introduced to make the trajectory respond to prior preferences during numerical integration. The prior energy function can be derived from... Parameterization is used to characterize the feasibility, smoothness, planning consistency, or risk cost of control signals. This guidance is a subordinate implementation and does not change the main flow of this invention: "trajectory sample → consistency distillation → one-step inference".

[0190] (T5.4) Uniform distillation training one-step model

[0191] Two points are selected from the same trajectory (these can be adjacent noise levels or non-adjacent noise levels) to obtain sample pairs:

[0192] Using the same trajectory sample pair as training input, impose at least the following two types of constraints on the one-step model (G(\cdot)): 1) Consistency constraint: Requires the model to maintain consistent output for different noise inputs on the same trajectory in one step.

[0193] 2) Expert alignment constraint: requires a one-step output that approximates the expert control signal. and / or

[0194] In one embodiment, a weighted combination loss is used to train a one-step model:

[0195] in These are weighting coefficients, which can be entered manually or adjusted based on the training model.

[0196] (T6) Training the noise level estimator With Intent Inferrer

[0197] (T6.1) Noise Level Estimator

[0198] Due to the training phase The data sampled in (T3) can be used as a supervisory signal for training. so that it can be made by and Estimated noise level / intervention intensity:

[0199] After training is completed, the deployment phase is handled by

[0200] To achieve adaptive intervention intensity.

[0201] (T6.2) Intent Inferrer

[0202] The intent inferrer is used to output local target / short-term intent prior. The training can be implemented using any of the following methods or combinations thereof (without limitation on specific forms): Utilizing planning information Construct supervision / weak supervision signals related to local sub-objectives, so that Consistent with the planning of local windows; By utilizing the temporal consistency of multimodal observation sequences, the intention of adjacent time points can be smoothly changed within a reasonable range; Using successful entries retrieved from the memory, Successful experiences are distinguishable.

[0203] Furthermore, a phased or alternating training strategy can be adopted: first train... and Retraining or joint fine-tuning and This is to ensure training stability and final performance.

[0204] Using the training method described above, only the student model G (and optional noise estimator and intent inferrer) is retained during the deployment phase, enabling training at each control time step. based on and Perform one forward inference to output a correction control signal This enables real-time one-step correction control.

[0205] Compared with existing technologies, this invention achieves one-step expert correction of noisy control signals from upstream controllers through an overall technical solution of "multimodal conditions, planning constraints, memory priors, noise adaptation, one-step denoising correction, optional security filtering, and closed-loop write-back," thereby achieving comprehensive improvements in real-time performance, deployability, robustness, task consistency, and security.

[0206] (1) Real-time performance and robustness

[0207] This invention employs a one-step denoising control and correction model to directly output correction control signals during the inference phase. This avoids the accumulation of inference latency caused by multi-step iterative optimization or multiple rounds of generation. Because this invention uses trajectory-based consistency distillation during the training phase, enabling the model to generate stable outputs in one step under different noise levels, correction can be completed with only one forward computation during deployment. This makes it suitable for scenarios sensitive to control cycles, such as robot and vehicle control, reducing the impact of the correction process on the stability of the control closed loop. Furthermore, it provides output correction control signals. The calibration device is deployed between the upstream controller and the actuator without changing the original controller's internal structure, interface protocol, or control logic. This facilitates the rapid integration and engineering verification of industrial control systems, vehicle control systems, and robot control systems, reducing deployment barriers and maintenance costs.

[0208] This invention uses multimodal fusion characterization Global planning information Memory Summary and local intention prior Formation condition set and use it as a one-step correction model The conditional input allows the corrected output to not only depend on the instantaneous control signal itself, but also to select a more reasonable control mode under the combined influence of language commands, visual semantics, attitude / sensor parameters, and planning constraints. This reduces the probability of erroneous action selection under multimodal ambiguity conditions, making the corrected control output more consistent with task semantics and planning intent, and improving control stability and interpretability in complex scenarios.

[0209] This invention also introduces noise level / intervention intensity estimation. And allow it to be estimated by the estimator Adaptive determination enables the correction intensity to automatically adjust according to the degree of defect in the input control signal: weak intervention when the input signal quality is high to maintain the continuity of the original controller strategy; strong intervention when the input signal is strongly disturbed or has significant deviations to more closely approximate expert control laws. This mechanism achieves continuously adjustable control of the correction intensity under different noise intensities and operating conditions, thereby improving the system's robustness to environmental changes, execution deviations, and distribution offsets.

[0210] (2) Experience closed loop and safety assurance

[0211] This invention constructs a control signal memory library During the reasoning stage, a memory summary is obtained through retrieval. Participation Conditions Set The design allows correction decisions to reference successful control experiences from similar historical contexts. After execution, a closed-loop write-back mechanism updates the context key, input / output control, and execution feedback to the memory, forming a continuously iterative experience loop. This design enables the system to gradually accumulate effective experience and improve generalization ability in repetitive tasks, similar scenarios, or long-term operation, reducing instantaneous decision fluctuations caused by single observation noise or local occlusion, thereby improving long-term stability and cross-scenario robustness.

[0212] In an optional embodiment, the present invention introduces a safety constraint projection / filtering module to map the correction output to a safe feasible region. Alternatively, prior guidance during the training phase can be used to internalize planning consistency and safety preferences in a one-step model. This can reduce the probability of control signals exceeding limits, violating feasible regions, or triggering dangerous states, making the output control signals not only meet task objectives but also better comply with engineering safety requirements, thereby improving the system's safety and availability under complex environments and uncertain disturbances.

[0213] In summary, among the modules mentioned above, the one-step denoising correction model solves the real-time problem, the noise adaptive estimation solves the working condition adaptability problem, and the memory closed loop solves the cross-scenario generalization problem. The three work together under a unified framework to achieve a comprehensive effect.

[0214] The correction system provided by this invention can be deployed in an integrated manner within existing control links. Under the combined effects of multimodal perception, global planning constraints, and prior knowledge of historical memory, it performs one-step correction on the noisy control signal output by the upstream controller. It is applicable to various control scenarios with high requirements for real-time performance, robustness, and safety, including gripping, assembly, and contact operation control of robotic arms; trajectory tracking and motion control of autonomous vehicles and mobile robots; drift correction in industrial process control and equipment operation control; stable control of medical robots and remote-assisted operation; and adaptive maintenance and long-term operation drift compensation for complex equipment. It should be understood that the above application scenarios do not constitute a limitation on the scope of application of this invention.

[0215] Specific application examples

[0216] 1. Applications of robotic arms

[0217] (1) Implementation environment and hardware configuration

[0218] In this embodiment, the system is deployed on a 7-DOF industrial collaborative robotic arm, with a two-finger gripper at the end effector, and the worktable includes a material box and an assembly fixture. Control cycle That is, 100Hz downlink control.

[0219] The sensor configuration includes: Vision: RGB-D camera at 30Hz (used for object detection, pose estimation, and scene semantics); Body: Joint encoder 1kHz (joint angle) Joint velocity ); Force sensing: 6-axis force / torque sensor at the end of the device, 250Hz ( ); Task instructions: Language / structured instructions (e.g., "grab part A → insert into the fixture hole → press firmly into place").

[0220] The upstream controller is an existing control module that outputs joint speed commands:

[0221] The module of this invention outputs the corrected joint velocity command:

[0222] (2) Construction of Multimodal Fusion Features and Planning Conditions

[0223] In this embodiment, multimodal observation It is composed of the following information: Visual semantics: target category, 6D pose, gripping point, jig hole pose, obstacle position; Ontology status: And its sliding window statistics (mean / variance of the most recent 10 frames); Force perception state: and its derivative approximation ; Instruction coding: Task stage (grab / transfer / align / insert / press) and parameters (target hole ID, insertion depth, etc.).

[0224] Output of the fusion feature module:

[0225] The global control planning module generates planning conditions (example: reference end trajectory code + stage status code for the next second):

[0226] The planning can come from existing trajectory planners (such as RRT). ( / spline / interpolator) or MPC reference trajectory; this invention does not limit the planning implementation, but only requires the output of coded planning conditions.

[0227] (3) Memory entry structure, index keys and retrieval strategies

[0228] This embodiment uses a control signal memory library. Each memory unit storage:

[0229] in: For index keys; Update the summary of local states (such as end-effector pose increment, peak force, and insertion success marker); Quality label (combined score of success rate / safety margin); For timestamps.

[0230] Index key generation employs an achievable linear projection plus normalization:

[0231] Online searches are performed using the Top-(K) similarity metric (e.g., K=16), and cosine similarity is used for the overall similarity.

[0232] Example of a quality filtering rule: Only retain quality scores And entries from the last 30 days; if insufficient The rules were relaxed to .

[0233] The output aggregation uses a weighted average:

[0234] ( ).

[0235] This is a mapping that projects entry fields into a fixed-dimensional summary vector (e.g., 128-dimensional).

[0236] (4) Short-term intentions Specific definitions and inferences

[0237] In assembly tasks, the "short-term intent" must be able to distinguish between alignment and insertion, as well as axial advancement and fine-tuning during insertion. This embodiment defines intent as "short-term target pose incremental encoding + pattern label":

[0238] in For the terminal target location increment, For attitude increment, This is an 8-dimensional one-hot process (grabbing / transporting / aligning / inserting / pressing / failure recovery, etc.).

[0239] The intent inference module is implemented as follows:

[0240] And introduce force gating (condition): if (Example) If the phase is biased towards the "insertion / contact adjustment" related component, the distinguishability of the contact state will be improved.

[0241] (5) Noise intensity / intervention intensity Operational implementation

[0242] This embodiment uses a "rule-based + learning-based optional" approach, and the default rule-based approach is sufficient for deployment: Regular :

[0243] in To normalize the planning deviation risk (e.g., the lateral deviation between the end point and the reference trajectory) to When its input control undergoes sudden changes, planning deviations are large, or contact occurs, it improves... This makes the calibration more biased towards "expert distribution".

[0244] (6) One-step noise reduction correction and constraint distribution

[0245] Construction conditions:

[0246] One-step calibration output:

[0247] The constraint module employs two levels of protection: Speed ​​limit: ; Acceleration limit: .

[0248] If a collision risk is detected ( (or visual collision prediction), then degradation is triggered: direct output. Or revert to security policy control.

[0249] Final issuance:

[0250] (7) Memory write-back and quality label Specific definition

[0251] Quality labels are generated after each execution:

[0252] in The distance of the endpoint from the reference trajectory. This is the normalization constant. And write it back:

[0253] (8) Example of complete task process

[0254] Take "peg-in-hole" assembly as an example: During the alignment phase, there is a 2–3 mm deviation in visual positioning, and the upstream controller outputs... Directional jitter occurs in some frames; this invention is based on Improvement of deviation from planning ,make The output is smoother and consistent with historical successful entries. Reduce lateral sway; Entering the contact phase, when Exceeding the threshold, intention Gated as "contact fine-tuning / axial advance," and successfully retrieved from the memory bank a control segment with similar hole positions and similar contact force modes, enabling... Automatically reduces lateral components and enhances axial propulsion stability; If frictional variations cause an abnormal peak force and a decrease in the quality label, the relevant entry will be written back but will be missed in subsequent searches. Filtering or low weighting enables "experience self-cleaning".

[0255] 2. Typical Application Examples of Autonomous Driving

[0256] (1) Implementation environment and signal interface

[0257] In this embodiment, the module of the present invention is deployed on the vehicle domain controller (or on-board computing platform), and the control cycle is... (50Hz).

[0258] The upstream controller is an existing horizontal and vertical control module (such as an MPC+PID combination or a learning strategy), which outputs a noisy control quantity:

[0259] These are the front wheel steering angle, drive acceleration, and braking command, respectively.

[0260] Output of this invention:

[0261] (2) Multimodal observation, fusion characteristics and planning conditions

[0262] Observation Include: Visual semantics (camera 20Hz): lane lines, drivable area, front vehicle / obstacle detection box and relative speed; Location and vehicle status (IMU / wheel speed / GNSS): Vehicle speed yaw rate Lateral acceleration Heading angle ; Environmental conditions: Rainfall / Temperature / Road adhesion estimation (Optional); High-level instructions: navigation intent (go straight / change lanes / exit), and, if necessary, language instruction mapping ("keep to the right and keep a safe distance").

[0263] Fusion characteristics:

[0264] Planning conditions Output from the planning module (e.g., encoding of trajectory point sequence for the next 2 seconds):

[0265] It may contain a reference curvature sequence. Target velocity sequence Lane changing phase, etc.

[0266] (3) Memory entry structure and retrieval

[0267] Memory entries are defined as follows:

[0268] in Record local state update summaries (e.g., changes in lateral deviation, jerk, and stability indices). It can be labeled with operating conditions (rainy day / night / highway / curve, etc.).

[0269] The index key example uses a "curvature-velocity-attachment-intent" sensitive design:

[0270] Retrieve Top-(K) Example of filtering rules: like (Slippery), prioritize keeping Entries; If vehicle speed (High-speed) Priority will be given to retaining high-speed entries; Quality threshold And remove entries marked with "yaw instability / triggered ESC".

[0271] Aggregation yields memory summary This is for use in subsequent intent inference and one-step correction.

[0272] (4) Short-term intentions Specific definition

[0273] In driving control, "modal" is crucial: maintaining lane position, following other vehicles, changing lanes, and avoiding obstacles will produce completely different control distributions. This embodiment defines short-term intent as:

[0274] in The data consists of 4-dimensional one-hot encoding (lane keeping / following / left lane change / right lane change), with the remainder representing short-term future reference changes (extracted from the planning sequence). Intent inference:

[0275] It can also introduce a "previous vehicle cutting in / lane line confidence decrease" trigger bias, making Switch to the "Follow / Yield" category promptly.

[0276] (5) Noise intensity Adaptive estimation

[0277] The "noise / defects" in vehicle scenarios mainly manifest as: large lateral deviation, jittery steering commands, increased stability risks, and decreased road adhesion. This embodiment provides a rule-based estimation that can be directly applied to the vehicle:

[0278] in: For lateral bias normalization (e.g.) Cut off to ); For stability risks (e.g.) )and Comprehensive results ); For adhesion estimation (from wheel speed difference / ABS trigger statistics or road surface identification).

[0279] In real-world scenarios, the more unstable, the further off-plan, and the more slippery, The larger the size, the more "experts" are needed for calibration.

[0280] (6) One-step noise reduction and vehicle safety constraints

[0281] Construction conditions:

[0282] One-step correction:

[0283] The constraint module must contain at least the following executable constraints (example values, which can be calibrated by vehicle model): Corner limiter: ; Rate of change of angle: (Every 20ms); Longitudinal acceleration limit: Braking has higher priority than driving. Traction circle approximation (stability constraint): If Then reduce And suppress accelerated output; Degradation strategy: If lane line confidence is low or perception is abnormal, revert to normal. For larger values, enable more conservative memory entries. (High and the tag is the same working condition).

[0284] Output:

[0285] And then send it to the vehicle actuator.

[0286] (7) Memory write-back and quality label

[0287] Generate quality labels during write-back (example):

[0288] in For vertical jerk If ABS / ESC is triggered or there is a risk of crossing the line, then immediately set... They are then tagged as "risk samples" for subsequent filtering.

[0289] (8) Complete scene process example

[0290] Rainy days caused The speed is reduced, and the upstream controller's rotation angle exhibits jitter. Increase; Memory retrieval prioritizes returning high-quality entries under the "slippery curve" condition. This makes the correction output smoother and more stable; The vehicle in front cutting in triggers the intention to switch to "follow". The mode in the middle changes, resulting in The output changes from the "lane keeping mode" to the "following mode" expert control distribution; If the risk of yaw increases, the constraint module further suppresses the rate of change of the rotation angle and limits the accelerated output to ensure safe execution.

[0291] 3. Training Example

[0292] (1) Training data acquisition and sample construction

[0293] T1: Obtain training data.

[0294] Data collection Each sample contains multimodal observations. Global planning information or task annotations (optional), and expert / high-quality control signals. Example: A robotic arm task collects 50 trajectories, each with 4,000 frames, for a total of approximately 200,000 steps.

[0295] T2: Input conditions for the computational model.

[0296] Calculate fusion features for each sample And calculate the planning code. (If a planning module exists, use its output; otherwise, it can be generated from task annotations or reference trajectories.) Simultaneously, initialize or build the memory bank offline for retrieving summaries during the training phase. .

[0297] T3: Construct noisy training pairs.

[0298] Sampling from noise distribution and sampling Construct a noisy input:

[0299] Example: Take To be a piecewise logarithmic uniform distribution, The interval is discretized into (L=32) noise levels for batch training.

[0300] (2) Training control modified teacher model

[0301] Teacher model denoted as ,in The training objective is to enable the teacher to recover the expert control signal under a given noise level.

[0302] Example training configuration: batch=1024, learning rate Training 200ksteps.

[0303] (3) PF-ODE prior guided trajectory sampling and uniform distillation training of student model

[0304] The student model is a one-step consistency model. The goal is to output correction control in one step during online inference.

[0305] T5.1 Noise schedule and trajectory edge interval.

[0306] Select a pair of noise levels Example: from sampling, from sampling.

[0307] T5.2 Probability Flow Trajectory Sampling Point Calculation (Teacher Trajectory).

[0308] Based on the teacher model, construct the probability flow trajectory sampling points from (\sigma_a) to (\sigma_b):

[0309] Example: Use 4 Euler steps for integration (or Heun 2nd order), and the time taken for each step is controllable.

[0310] T5.3 Prior Energy Guidance (Optional Enhancement).

[0311] Constructing the prior energy function This is used to inject planning consistency, memory preference, and safety preference into teacher trajectory sampling. Example: Planning Consistency Energy: ( To map control onto a predictor of short-term trajectories, (For reference only) Memory attracts energy: ( (From Top-(K) memory output); Safety Energy: ( (For safety constraints).

[0312] comprehensive: Example weights ( ).

[0313] Weighting or guiding the sampling during trajectory integration (e.g., adding a parameter to each update step) (Related correction terms for _u E) make the trajectory more consistent with prior preferences.

[0314] T5.4 Consistent distillation training.

[0315] Using two points u_(σ_a) and u_(σ_b) on the same trajectory as training pairs, the student outputs are constrained to be consistent, and pairs of u^ can be superimposed. Supervision:

[0316] In this example, β=0.1. After training, the online phase can be directly used... One-step output, no iterative integration required.

[0317] (4) Training the auxiliary estimation module

[0318] T6.1 Training Intent Inference Module I ( ).

[0319] Training is performed using "short-term goal / direction labels" or "next state change encoding" as supervised signals. Example: If next state features exist... Target direction can be constructed , and minimize .

[0320] T6.2 Training noise estimation module (S(\cdot)).

[0321] When constructing noisy samples, σ is known and can be used as a supervised label to train S( Regress σ and constrain its output range. .

[0322] While the structure of the present invention has been described above with reference to preferred embodiments, those skilled in the art should recognize that the above examples are merely illustrative and should not be construed as limiting the invention. Therefore, modifications and variations can be made to the present invention, all of which will fall within the scope defined by the appended claims.

Claims

1. A method for correcting a control signal, used to correct a noisy input control signal output by an upstream controller at discrete control times, the method comprising the following steps: Acquire multimodal sensor observations; Multimodal fusion characterization is calculated based on observations from the aforementioned multimodal sensor; Global control planning information is generated based on the multimodal fusion representation; A memory retrieval key is generated based on the multimodal fusion representation and the global control planning information; Based on the memory retrieval key, a retrieval is performed in the control signal memory bank and a memory summary is generated; Obtain the noisy input control signal output from the upstream controller; Short-term intent priors are inferred based on the multimodal fusion representation, the global control planning information, the memory summary, and the noisy input control signal; The correction conditions are constructed using the multimodal fusion representation, the global control planning information, the memory summary, and the short-term intent prior. The noise intensity parameters are estimated based on the noisy input control signal and the correction conditions. as well as The noisy input control signal, the noise intensity parameter, and the correction conditions are input into a one-step denoising correction model to perform one forward inference, thereby directly outputting the correction control signal.

2. The method according to claim 1, characterized in that, After outputting the correction control signal, the method further includes the following steps: Perform security verification or constraint processing on the correction control signal and then issue it for execution; and Based on the execution feedback, the entries containing the memory retrieval key, the noisy input control signal, the correction control signal, the short-term intent prior, and the feedback quality information are written back to the control signal memory to update subsequent retrieval and summary calculation.

3. The method according to claim 1, characterized in that, The multimodal sensor observations include two or three of the following: language information, visual information, hardware sensor parameters, or attitude information.

4. The method according to claim 1, characterized in that, The control signal memory uses a data structure that associates index keys with index content for storage. The index keys are associated with at least the conditional input, control input, and short-term intent fields. The index content includes at least the control output, control intent, and local state update fields, and is associated with a timestamp field.

5. The method according to claim 1 or 4, characterized in that, Generating the memory summary includes Top-K nearest neighbor retrieval and ranking based on similarity metrics to obtain candidate entries, and quality filtering or reordering of the candidate entries.

6. The method according to claim 5, characterized in that, The memory summary is obtained by weighted aggregation of the candidate items, wherein the weights are at least related to the similarity score and to the item quality label and / or freshness weight.

7. The method according to claim 1, characterized in that, The short-term intent prior is any one or a combination of short-term sub-target encoding, short-term reference direction encoding, short-term state change trend encoding, and control modality labeling, wherein the intent component of the short-term intent prior is selectively or weighted based on contact / collision risk, perception confidence, or planning deviation.

8. The method according to claim 1, characterized in that, The noise intensity parameter is a continuous scalar or discrete level value limited to a preset range, and is estimated by a rule function, a learning model, or a combination of both. The rule function is calculated based on at least one of the following: the amplitude of the input control signal change, the planning deviation risk, the stability risk, and the contact / collision risk.

9. The method according to claim 2, characterized in that, The safety verification or constraint processing includes any one or a combination of amplitude limiting processing, quadratic programming projection processing, and barrier function filtering processing, so that the issued control signal satisfies the safety constraints associated with speed, torque, rotation angle, acceleration, and collision distance.

10. The method according to claim 2, characterized in that, It also includes the following steps: When a perception anomaly is detected, the risk of constraint violation increases, or the memory hit rate decreases, a degradation strategy is triggered. The degradation strategy includes bypassing the one-step denoising correction model and directly outputting the control signal of the upstream controller, or switching to a more conservative set of memory entries to generate a memory summary.

11. A method for training a one-step denoising correction model according to claim 1, comprising the following steps: Obtain a training dataset, which includes multimodal observations and target control signals; Calculate multimodal fusion representation and global control planning information for training samples, and construct memory retrieval keys and memory summaries; A noisy input control signal is constructed from the target control signal based on a preset noise schedule; The training control correction teacher model is made to learn to output the target control signal under conditional constraints from the noisy input control signal; Based on the teacher model, trajectory sample pairs corresponding to different noise levels are generated, and consistency distillation training is performed on the control correction student model to enable the student model to obtain the ability to output a correction control signal through forward inference. The short-term intent inference module and noise intensity estimation module are trained for condition construction and adaptive noise intensity estimation during the online inference phase.

12. The training method according to claim 11, characterized in that, During the generation of trajectory sample pairs corresponding to different noise levels based on the teacher model, the generation of trajectory samples by the teacher model introduces prior guidance, wherein the prior guidance includes at least one of planning consistency guidance, memory attraction guidance, and safety constraint guidance.

13. The training method according to claim 11, characterized in that, The consistency distillation training includes both output consistency constraints across noise levels and supervisory constraints with the target control signal, and only one inference step is performed using the student model during online deployment.

14. A control signal correction device, comprising: A multimodal feature extraction and fusion module is used to generate multimodal fusion representations; The global control planning module is used to generate global control planning information; The control memory module is used to generate memory retrieval keys and perform retrieval and summary calculation to output memory summaries, and is used for memory write-back updates; The short-term intent inference module is used to output short-term intent priors and construct correction conditions; The noise intensity estimation module is used to estimate noise intensity parameters; The control denoising correction module is used to input the noisy input control signal, the noise intensity parameter, and the correction conditions into a one-step denoising correction model and output a correction control signal through one forward inference; and The control output and communication interface is used to output control signals for execution. The modules work together to implement the method described in claim 1.

15. The control signal correction device according to claim 14, characterized in that, The one-step correction device for the control signal further includes: The control constraint module is used to perform safety verification or constraint processing on the correction control signal.

16. The control signal correction device according to claim 14, characterized in that, The control signal memory module stores historical entries based on a structure that associates index keys with index content, and generates memory summaries using Top-K nearest neighbor retrieval, quality filtering, and weighted aggregation.

17. A control system, comprising: The multimodal sensor, the upstream controller, the actuator, and the control signal correction device according to any one of claims 14 to 16, wherein the noisy input control signal output by the upstream controller is processed by the control signal correction device and then output to the actuator for execution, and the execution feedback is used to update the control signal memory.

18. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for correcting control signals as described in any one of claims 1-10, or the training method as described in any one of claims 11-13.

Images