A Self-Healing Control Method for Industrial Control Systems Based on Multimodal Temporal Residual Driven by AI
By constructing a multimodal time-series residual-driven industrial control system, the problems of nondeterministic divergence and lack of dynamic physical feedback in industrial control systems are solved. A closed-loop system from natural language instructions to safety control logic is realized, ensuring the robust stability and adaptive correction of the system.
Patent Information
- Application Number
- CN202611350221.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-02
- Publication Date
- 2026-11-10
AI Technical Summary
In existing technologies, industrial control systems lack underlying deterministic constraints, which makes the system state prone to divergence. Furthermore, the lack of dynamic physical feedback makes it impossible to perceive continuous timing deviations of actuators, resulting in the inability to adaptively correct control logic deviations.
A multimodal temporal residual-driven industrial control system is constructed. Through an adaptive semantic configuration layer and a hardware-in-the-loop verification layer, high-frequency synchronous sampling is performed using a hard real-time industrial network to calculate the temporal residual matrix, extract the time-domain transition hysteresis and state amplitude distortion integral, and use the residual-multimodal cue word mapping operator to convert physical deviations into natural language cue words, driving the large language model to perform autoregressive topology correction.
It realizes a closed-loop system from natural language commands to safety control logic, eliminates the risk of nondeterministic divergence, ensures the robust stability of the system, and enables the AI controller to perceive and self-regressively correct the underlying physical dynamic deviations.
Smart Images

Figure CN123067283A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence multimodal technology, specifically to an AI self-healing control method for industrial control systems based on multimodal temporal residual driving. Background Technology
[0002] In the field of industrial automation and robot control, using large language models (LLM) to directly convert natural language instructions into programmable logic controller (PLC) control logic or robot motion scripts is an important technical path to realize flexible manufacturing and smart factories.
[0003] Current technologies primarily employ an open-loop or semi-open-loop assisted programming approach: "Natural Language Input -> LLM Code Generation -> Compiler Static Checking -> Direct Deployment to Physical Devices." For example, in existing commercial industrial AI development environments (such as Beckhoff's TwinCAT Chat / TwinCAT 3 CoAgent or Siemens IndustrialCopilot tools) or related literature (such as "LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems" (published in 2024, indexed by arXiv / IEEE)), the typical implementation process is as follows: the system receives natural language instructions from the operator and directly utilizes the autoregressive generation capability of the Large Language Model (LLM) to translate them into underlying PLC structured text (ST) or robot motion scripts. Subsequently, the generated code undergoes static syntax checking using a traditional industrial development environment. If no basic syntax errors are found, the control logic is directly issued and deployed to the underlying physical hardware for open-loop execution.
[0004] Chinese Patent Publication No. CN 117991647 B discloses a method and apparatus for performance monitoring and self-healing control of an industrial control system. The method includes: acquiring real-time operating data of the industrial control system to be controlled; obtaining real-time performance evaluation results of the industrial control system based on the real-time operating data and a performance degradation monitoring module based on the Bellman equation; obtaining performance degradation monitoring results of the industrial control system based on the real-time performance evaluation results; and obtaining self-healing control results of the industrial control system based on the performance degradation monitoring results and a self-optimizing feedback supplementary self-healing control module based on a reinforcement learning algorithm. This invention achieves real-time performance monitoring using data from the industrial operation process without relying on the mathematical model of the controlled object and the control system; it does not change the original controller structure parameters, but only uses data and reinforcement learning methods to design feedback compensation to recover performance degradation, thus achieving self-healing control.
[0005] When the above and existing technologies are applied to actual industrial control, the following two fundamental technical defects still exist: 1. Lack of underlying deterministic constraints leads to easy divergence in system state: Existing technical solutions employ open-loop code generation and distribution mechanisms. Large Language Models (LLMs), due to their autoregressive generation based on probability distributions, inherently possess non-deterministic outputs. Directly distributing the topological logic generated by an LLM without any physical boundary constraints easily leads to temporal logic conflicts or deadlocks. From a control theory perspective, such open-loop systems cannot guarantee the convergence of state variables, easily causing the control system's execution output to diverge at boundaries (e.g., position or velocity variables exceeding the safety envelope), resulting in extremely poor system robustness.
[0006] 2. The lack of dynamic physical feedback leads to modal discontinuity between the observation space and the action space: In existing technologies, static code verification relies solely on discrete text error logs provided by traditional compilers, which falls under the category of static analysis in the digital space. However, the underlying physical world operates on continuous or quasi-continuous time-domain physical signals (such as bus levels and servo physical trajectories). Existing systems lack a mechanism to collect and map continuous temporal residuals from the physical layer back to the AI controller. This missing feedback loop leaves the AI controller in a "physical blind spot," unable to perceive time-varying lags or amplitude distortions in the actuators. Consequently, it cannot adaptively and dynamically correct deviations in the control logic, leading to numerous inconveniences.
[0007] Therefore, this invention requires the design of an AI self-healing control method for industrial control systems based on multimodal temporal residual driving to solve the aforementioned problems. Summary of the Invention
[0008] The purpose of this invention is to provide an AI self-healing control method for industrial control systems based on multimodal temporal residual driving in order to solve the above-mentioned problems. This method addresses the issue mentioned in the background art of how to establish physical dynamic feedback between a nondeterministic large language model and deterministic industrial physical equipment, transforming the continuous temporal physical deviation of the execution layer into discrete semantic features that the large language model can perceive, thereby eliminating the risk of nondeterministic divergence in control logic and realizing the self-closing and self-healing problems of the control system.
[0009] To address the above problems, the present invention provides a technical solution: A self-healing control method for industrial control systems based on multimodal temporal residuals includes the following specific steps: S1. Construct the system topology architecture, receive natural language control commands, extract intent using a large language model, and discretize and map it into an N×M-dimensional discrete target state trajectory matrix in the control time domain. Where N is the number of physical observation channels and M is the total number of sampling frames; S2. Through deterministic assembly using control laws, the large language model is constrained to not directly output code text, and is forced to output a topology description file in a limited format, which is only allowed to contain configuration parameters and topology connection relationships, thereby calling industrial control operators in a preset deterministic standard function library; if the middleware compares the whitelist and finds that the large language model is trying to call an illegal function, it will be blocked. S3. Observe the hardware in the loop state. Load the combined control logic that has passed the inspection into the hardware-in-the-loop verification layer for operation. At this time, the physical control device is in a truncated state. Use the hard real-time industrial network to perform high-frequency synchronous sampling and obtain the N×M-dimensional discrete actual response trajectory matrix of the heterogeneous actuators. ; S4. Perform temporal residual deconstruction and feature extraction, and calculate the discrete temporal residual matrix of the reference trajectory and the actual response trajectory. The time-domain transition hysteresis of each physical channel is extracted using the residual deconstruction operator. and state amplitude distortion integral splicing and assembling into a control deviation feature vector ; S5. Multimodal self-healing iteration and gated delivery processing: Utilizing the residual-multimodal prompt word mapping operator, the numerical feature vector is transformed... Based on a pre-defined alarm template library, the alarms are converted into natural language prompts containing physical deviation characteristics, which are then fed back into the context window of the large language model to drive its autoregressive topology correction of the control logic; simultaneously, the Frobenius norm of the residuals is continuously monitored. If and only if the convergence criterion is satisfied. At that time, the hardware communication release locking mechanism is activated, and the control logic for verifying security is transmitted to the field backbone network.
[0010] In a preferred embodiment of the present invention, the preset deterministic standard function library in step S2 includes function blocks FB / FC conforming to the IEC 61131-3 standard; the topology description file is in JSON format; the middleware compares the whitelist, and if it finds that the large language model is trying to call an illegal function, it directly blocks it at the digital layer.
[0011] In a preferred embodiment of the present invention, the state amplitude distortion integral in step S4 The calculation method is as follows: Using the trapezoidal integral numerical algorithm, the absolute values of the row vectors corresponding to the physical channels in the discrete-time residual matrix E are accumulated and summed in the time domain, i.e. .
[0012] In a preferred embodiment of the present invention, the method for calculating the time-domain transition hysteresis Δτi in step S4 is as follows: capture the first frame in which the reference signal undergoes a level transition. The difference between the first frame kact that actually captures the jump in the same direction as the underlying bus, multiplied by the scan sampling period of the industrial bus. ,Right now .
[0013] In a preferred embodiment of the present invention, in step S5 The preset minimum system tolerance threshold is defined when the Frobenius norm of the residual matrix ( ) hour( When the control law reaches asymptotic stability, the system sends an unlock command to the hardware communication interception gateway connected to the industrial backbone network. The physical relay inside the gateway closes, and the physical layer switches from cutoff to conduction.
[0014] In a preferred embodiment of the present invention, the residual-multimodal cue word mapping operator in step S5 incorporates a threshold judgment and semantic conversion mechanism. When the time-domain jump of any physical observation channel is along the hysteresis... Or state amplitude distortion integral When the self-healing prompt word concatenation logic is triggered, the specific physical deviation algebraic calculation results and the corresponding channel hardware topology mapping name are combined and assembled into a natural language description autoregressive prompt word text alarm, which is then fed back to the context window of the large language model.
[0015] In a preferred embodiment of the present invention, the system topology includes an adaptive semantic configuration layer and a hardware-in-the-loop verification layer, wherein the adaptive semantic configuration layer and the hardware-in-the-loop verification layer are bidirectionally connected. The adaptive semantic configuration layer is used to receive natural language instructions, extract intent using a large language model and generate a discrete target state trajectory matrix, and output a topology description file in a defined format to call industrial control operators in a preset deterministic standard function library. The hardware-in-the-loop verification layer is used to connect logically with the upper-layer adaptive semantic configuration layer through the built-in HIL interception gateway, load the combined control logic to perform hardware-in-the-loop verification, use the hard real-time industrial network to perform high-frequency synchronous sampling to obtain the actual response trajectory, calculate the time-series residuals and perform feature extraction, and control the truncation and conduction of the physical control equipment mechanism according to the residual norm convergence judgment result.
[0016] In a preferred embodiment of the present invention, the adaptive semantic configuration layer includes a natural language input module and an LLM semantic parser and planner module, wherein the output of the natural language input module is communicatively connected to the input of the LLM semantic parser and planner module. The natural language input module is used to receive natural language control commands input by staff and transmit them to the LLM semantic parser and planner module through a logical link; The LLM semantic parser and planner module is used to extract the intent of natural language instructions, discretize and map them into a discrete target state trajectory matrix in the control time domain, and generate a topology description file by calling a preset deterministic standard function library.
[0017] In a preferred embodiment of the present invention, the hardware-in-the-loop verification layer includes a timing residual deconstruction chip and a multimodal mapping operator. The output of the timing residual deconstruction chip is communicatively connected to the input of the multimodal mapping operator. The timing residual deconstruction chip is bidirectionally communicatively connected to the underlying physical actuator through an industrial bus. The time-series residual deconstruction chip is used to calculate the discrete time-series residual matrix between the reference trajectory and the actual response trajectory. It calls the residual deconstruction operator to extract the time-domain transition hysteresis and state amplitude distortion integral of each physical channel, and splices them together to form a control deviation feature vector. The multimodal mapping operator is used to convert the numerical control deviation feature vector into natural language prompts containing physical deviation features according to a preset alarm template library, and feed them back to the context window of the upper-level large language model.
[0018] In a preferred embodiment of the present invention, the hardware-in-the-loop verification layer further includes a physical control device gateway, which is connected across the industrial backbone network. Its default state is a truncated state, which isolates the control logic from the physical actuators. When the system determines that the residual norm converges to below a preset minimum system tolerance threshold, the physical control device gateway receives an unlocking command, closes its internal physical relay, switches from the truncated state to the on state, and sends the verified secure control logic down to the field backbone network.
[0019] The beneficial effects of this invention are as follows: By setting an adaptive semantic configuration layer and a hardware-in-the-loop verification layer, this invention constructs a complete closed-loop system from natural language commands to the issuance of safety control logic. In actual operation, it receives natural language control commands from staff, extracts intent using a large language model, and generates a discrete target state trajectory matrix. A middleware interceptor forces the LLM to output a topology description file in a defined format to call industrial control operators in a preset deterministic standard function library. The checked combined control logic is loaded into the hardware-in-the-loop verification layer for execution. High-frequency synchronous sampling is performed using a hard real-time industrial network to obtain the actual response trajectory. The timing residual matrix is calculated, and the time-domain transition hysteresis and state amplitude distortion integral of each physical channel are extracted and assembled into a complete system. The control deviation feature vector is transformed into natural language prompts containing physical deviation features using a residual-multimodal prompt mapping operator. These prompts are then fed back into the LLM context window to drive autoregressive topology correction of the control logic. Simultaneously, the Frobenius norm of the residuals is continuously monitored. Hardware communication release mechanism is activated only when convergence criteria are met, transmitting the verified and safe control logic to the field backbone. Through LLM semantic parsing and multimodal reference trajectory construction by the planner module, nondeterministic natural language commands are transformed into deterministic mathematical trajectory matrices, providing a quantitative benchmark for subsequent physical residual comparison. A middleware whitelist mechanism strictly constrains the LLM to only call IEC-compliant... The configuration layer of the FB / FC function block in the 61131-3 standard fundamentally eliminates the risk of nondeterministic divergence caused by AI directly generating low-level code. Through the timing residual deconstruction chip of the hardware-in-the-loop verification layer, two deterministic mathematical operators, namely the time-domain transition hysteresis and the state amplitude distortion integral, are used to accurately quantize the physical deviation in the continuous time domain into an algebraic feature vector, realizing a precise mapping between the physical world and the digital space. Through the threshold triggering and semantic conversion mechanism of the residual-multimodal prompt word mapping operator, the algebraic deviation vector is automatically converted into LLM-aware natural language alarms, enabling the large language model to perceive the underlying physical dynamic deviation and realize the autoregressive topology correction of the control logic in a negative feedback manner. Through the convergence judgment based on the Frobenius norm and the gating and distribution mechanism of physical control equipment, it is ensured that only control logic that meets the asymptotic stability condition can be conducted and distributed to the real production line, thus guaranteeing the robust stability of the system from the perspective of control theory. Attached Figure Description
[0020] For ease of explanation, the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.
[0021] Figure 1 This is a flowchart of the overall method of the AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to the present invention. Figure 2This is a diagram showing the overall system topology and hardware architecture of an AI self-healing control method for industrial control systems based on multimodal temporal residual driving, as described in this invention. Figure 3 This invention relates to a time-series residual matrix (E) discretization reconstruction and feature extraction operator for an AI self-healing control method for industrial control systems based on multimodal time-series residuals. Flowchart of the self-healing algorithm; Figure 4 This is a waveform comparison diagram showing the time-domain phase lag and amplitude distortion generated by the reference trajectory and the actual response trajectory in an AI self-healing control method for industrial control systems based on multimodal time-sequential residual drive according to the present invention. Figure 5 This is a state switching diagram of the hardware communication release locking mechanism of the AI self-healing control method for industrial control systems based on multimodal timing residual driving, in the default cut-off state and the conduction state. Detailed Implementation
[0022] like Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown, the specific implementation adopts the following technical solution: A self-healing control method for industrial control systems based on multimodal temporal residuals includes the following specific steps: S1. Construct the system topology architecture, receive natural language control commands, extract intent using a large language model, and discretize and map it into an N×M-dimensional discrete target state trajectory matrix in the control time domain. Where N is the number of physical observation channels and M is the total number of sampling frames; S2. Through deterministic assembly using control laws, the large language model is constrained to not directly output code text, and is forced to output a topology description file in a limited format, which is only allowed to contain configuration parameters and topology connection relationships, thereby calling industrial control operators in a preset deterministic standard function library; if the middleware compares the whitelist and finds that the large language model is trying to call an illegal function, it will be blocked. The preset deterministic standard function library contains function blocks FB / FC that conform to the IEC 61131-3 standard; the topology description file is in JSON format; the middleware compares the whitelist, and if it finds that a large language model is trying to call an illegal function, it will directly block it at the digital layer; S3. Observe the hardware in the loop state. Load the combined control logic that has passed the inspection into the hardware-in-the-loop verification layer for operation. At this time, the physical control device is in a truncated state. Use the hard real-time industrial network to perform high-frequency synchronous sampling and obtain the N×M-dimensional discrete actual response trajectory matrix of the heterogeneous actuators. ; S4. Perform temporal residual deconstruction and feature extraction, and calculate the discrete temporal residual matrix of the reference trajectory and the actual response trajectory. The time-domain transition hysteresis of each physical channel is extracted using the residual deconstruction operator. and state amplitude distortion integral splicing and assembling into a control deviation feature vector ; State amplitude distortion integral The calculation method is as follows: Using the trapezoidal integral numerical algorithm, the absolute values of the row vectors corresponding to the physical channels in the discrete-time residual matrix E are accumulated and summed in the time domain, i.e. ; The method for calculating the time-domain transition hysteresis Δτi is as follows: capture the first frame in which the reference signal undergoes a level transition. The difference between the first frame kact that actually captures the jump in the same direction as the underlying bus, multiplied by the scan sampling period of the industrial bus. ,Right now ; S5. Multimodal self-healing iteration and gated delivery processing: Utilizing the residual-multimodal prompt word mapping operator, the numerical feature vector is transformed... Based on a pre-defined alarm template library, the alarms are converted into natural language prompts containing physical deviation characteristics, which are then fed back into the context window of the large language model to drive its autoregressive topology correction of the control logic; simultaneously, the Frobenius norm of the residuals is continuously monitored. If and only if the convergence criterion is satisfied. At that time, the hardware communication release lock mechanism is activated, and the control logic for verifying security is sent to the field backbone network. The preset minimum system tolerance threshold is defined when the Frobenius norm of the residual matrix ( ) hour( When the control law reaches asymptotic stability, the system sends an unlock command to the hardware communication interception gateway connected to the industrial backbone network. The physical relay inside the gateway closes, and the physical layer switches from cutoff to conduction. The residual-multimodal cue word mapping operator has a built-in threshold judgment and semantic transformation mechanism. When the time domain jump of any physical observation channel is along the hysteresis... Or state amplitude distortion integral When the self-healing prompt word concatenation logic is triggered, the specific physical deviation algebraic calculation results and the corresponding channel hardware topology mapping name are combined and assembled into a natural language description autoregressive prompt word text alarm, which is then fed back to the context window of the large language model.
[0023] In a preferred embodiment of the present invention, the system topology includes an adaptive semantic configuration layer and a hardware-in-the-loop verification layer, which are bidirectionally connected. The adaptive semantic configuration layer receives natural language instructions, extracts intent using a large language model, generates a discrete target state trajectory matrix, and outputs a topology description file in a defined format to call industrial control operators in a preset deterministic standard function library. The hardware-in-the-loop verification layer is logically connected to the upper-layer adaptive semantic configuration layer through a built-in HIL interception gateway, loads combined control logic for hardware-in-the-loop verification, uses a hard real-time industrial network for high-frequency synchronous sampling to obtain the actual response trajectory, calculates the time-series residuals and extracts features, and controls the truncation and conduction of the physical control equipment mechanism based on the residual norm convergence determination result.
[0024] The adaptive semantic configuration layer includes a natural language input module and an LLM semantic parsing and planning module. The output of the natural language input module is communicatively connected to the input of the LLM semantic parsing and planning module. The natural language input module receives natural language control commands input by staff and transmits them to the LLM semantic parsing and planning module through a logical link. The LLM semantic parsing and planning module extracts the intent of the natural language commands, discretizes and maps them into a discrete target state trajectory matrix in the control time domain, and generates a topology description file by calling a preset deterministic standard function library.
[0025] The hardware-in-the-loop verification layer includes a timing residual deconstruction chip and a multimodal mapping operator. The output of the timing residual deconstruction chip is communicatively connected to the input of the multimodal mapping operator. The timing residual deconstruction chip is bidirectionally communicatively connected to the underlying physical actuator via an industrial bus. The timing residual deconstruction chip is used to calculate the discrete timing residual matrix of the reference trajectory and the actual response trajectory, and calls the residual deconstruction operator to extract the time-domain transition hysteresis and state amplitude distortion integral of each physical channel, and splices them together to form a control deviation feature vector. The multimodal mapping operator is used to convert the numerical control deviation feature vector into natural language prompts containing physical deviation features according to a preset alarm template library, and feeds them back to the context window of the upper-layer large language model.
[0026] The hardware-in-the-loop verification layer also includes a physical control device gateway, which is connected to the industrial backbone network. Its default state is a truncated state, which isolates the control logic from the physical actuators. When the system determines that the residual norm has converged to below the preset minimum system tolerance threshold, the physical control device gateway receives an unlock command, closes its internal physical relay, switches from the truncated state to the on state, and sends the control logic for verifying security to the field backbone network.
[0027] Example: Detailed implementation and mathematical derivation of the algorithm: The following describes in detail the mathematical calculation logic and self-healing closed-loop execution process of the present invention, taking into account a specific control scenario of a "conveyor belt with pneumatic feeding mechanism and a 6-DOF collaborative robot handling production line".
[0028] 1. Mathematical Reconstruction of State Space and Temporal Matrix Assume control time domain Industrial bus sampling cycle Then the total number of sampling frames Frame. System joint observation dimension : Channel 1 ( ): PLC cylinder drive solenoid valve switch output channel (Q0.0), high level 1 means the cylinder extends, low level 0 means the cylinder retracts.
[0029] Channel 2 ( ): The continuous downward displacement of the collaborative robot along the Z-axis (unit: mm).
[0030] Human staff input control commands "When the sensor is triggered, the conveyor belt stops, the cylinder immediately extends to push the material, and the robot falls to grab it. The robot must not fall before the cylinder is fully pushed into place." After receiving the instruction, the upper-layer LLM first constructs the target state trajectory matrix. In the ideal design of AI, the cylinder is... (i.e., frame 1000) The extension is triggered, and the robot is delayed until... (i.e., frame 1500) The fall begins:
[0031] 2. Hardware-in-the-loop testing and timing residual matrix Calculation After the LLM programmes the parameters of the underlying function blocks (FB / FC), it loads the control logic into the hardware-in-the-loop verification layer. Because the large model lacks an understanding of the mechanical inertia of physical entities, it does not connect the cylinder position sensor (I0.2) in series along the robot's startup path for interlocking.
[0032] During the operation of the test equipment, the actual response trajectory matrix was obtained by high-frequency sampling through the Profinet and Modbus TCP buses as state observers. Due to the response delay of the mechanical valve, the cylinder did not immediately reach its position after receiving the command in frame 1000, but did not reach its position until frame 1250 (i.e., It was only physically pushed into place at frame 1500. However, because the robot executes high-speed electrical signal drive, it only reaches its physical position at frame 1500. At that time, the descent had strictly followed the flawed logic, resulting in... Spatial collisions and conflicts occurred within a certain time period.
[0033] The system algorithm calculates the discretized time-series residual matrix in real time. :
[0034] 3. Residual deconstruction operator feature extraction and dimensionality reduction For matrices The system's built-in residual deconstruction chip uses the following two deterministic mathematical operators to extract physical self-healing features and reduces their dimensionality to a control deviation feature vector. : Operator A: Calculates the cylinder passage ( The time-domain transition along the hysteresis The algorithm captures the first frame of the reference signal transition. Capture the first frame of the actual signal transition. The time-delay residual is then:
[0035] Operator B: Computational robot channel ( State amplitude distortion integral Because the robot falls prematurely, its actual position in the time domain deviates from the ideal safe trajectory. The algorithm uses trapezoidal numerical integration to calculate the area of absolute error over the entire period:
[0036] The two residual features are assembled into an feature vector: .
[0037] 4. Residual-Discrete Semantic Mapping and Control Law State Self-Healing Mapping operators within the system Detected and This triggers a threshold alarm. The operator automatically retrieves the underlying hardware topology mapping table and converts the algebraic deviation vector... Translated into the following multimodal discrete semantic self-healing prompt (keyword): "Physical residual closed-loop negative feedback alarm: Detected in control channel 1 (Q0.0 cylinder drive valve) generating..." The time-domain phase lag causes control channel 2 (collaborative robot Z-axis) to generate [something] before the safety boundary is met. Spatial lead amplitude distortion. The system experiences timing deadlock and collision conflict. Please reconstruct the control law topology and forcibly interlock the enable terminal (EN) of the robot start function block (FB201) with the cylinder position sensor channel (I0.2). The prompt is sent to the context window of the upper-level LLM. The LLM performs autoregressive inference, detects the negative feedback constraints in the physical time domain, automatically corrects the assembly topology of the control operator, and forcibly attaches the robot enable signal after the physical feedback of the PLC switch, thus completing the algorithm self-healing.
[0038] 5. Stability determination and physical lock release The self-healing control logic undergoes a second hardware-in-the-loop retest. The actual waveform captured by the bus at this time... Strictly aligned with the ideal reference waveform. The algorithm calculates the Frobenius norm of the current residual matrix:
[0039] Since the system error norm is less than the set minimum tolerance threshold, the system is determined to have reached asymptotic stability in a cybernetics sense. At this point, the system sends an unlock command to the hardware communication interception gateway connected to the industrial backbone network. The physical relay inside the gateway closes (the physical layer switches from cutoff to conduction), the communication lock is released, and the self-healing control logic that verifies security is officially deployed and sent to the actual main production line in the factory.
[0040] Compared with the prior art, the present application has the following significant technical advantages: 1. Effectively avoids the risk of unpredictable divergence in AI control logic, significantly improving the operational safety of physical equipment. Technical Approach: This invention constructs a two-layer architecture of upper-layer adaptive semantic configuration and lower-layer execution, strictly constraining the Large Language Model (LLM) to only call preset deterministic function blocks conforming to the IEC61131-3 standard for topology assembly. Simultaneously, a Hardware-in-the-Loop (HIL) verification and gating mechanism are introduced. Control logic must first run in the HIL layer if and only if the time-order residual norm converges to a preset minimum tolerance threshold value (…). Only when the physical communication lock mechanism is released will the system be able to send out the communication.
[0041] The resulting technical advantages are: This method fundamentally intercepts timing conflicts and deadlocks caused by AI illusions, and avoids equipment damage accidents such as spatial collisions caused by directly deploying nondeterministic code to the real production line.
[0042] 2. It bridges the modal gap between the digital virtual world and the physical world, achieving dynamic closed-loop self-healing of the AI controller. Technical approach employed: This invention utilizes high-frequency synchronous sampling in a hard real-time industrial network to acquire the actual response trajectories of heterogeneous actuators and extract continuous physical features (such as time-domain transition hysteresis and state amplitude distortion integral). Subsequently, using the residual-multimodal prompt word mapping operator, these algebraic-dimensional physical deviations are precisely converted into discrete natural language semantic features (Prompt warnings), which are then fed back into the LLM.
[0043] The resulting technical advantages are: it changes the status quo that traditional development environments can only provide feedback on static text syntax errors, enabling large models to truly "perceive" deviations in underlying continuous physical dynamics, and use this as negative feedback to achieve self-regressive topology correction, forming a true AI self-healing control closed loop.
[0044] 3. Possesses high-precision multi-channel joint observation and control deviation positioning capabilities. Technical approach employed: This invention uses a mathematical reconstruction method based on state space and time-series matrices to transform the discretized control objective of the LLM (Limited Linear Modulation) and the actual high-frequency sampled physical response into matrices of the same dimension for alignment and comparison. Phase lag is calculated by finely capturing the difference in the first frame of the signal jump, and the error trajectory is then integrally time-domain using absolute values.
[0045] The resulting technical advantages are: In the joint control scenarios of multi-channel heterogeneous devices (such as cylinders and collaborative robots), this approach enables the system to accurately locate and quantify which specific underlying channel's response delay caused the global security breach, thereby providing the most accurate repair guidance for LLM.
[0046] In practical applications, industrial control systems with multiple physical actuators, such as pneumatic cylinders, servo motors, and hydraulic cylinders, are used in conjunction with an adaptive semantic configuration layer, a hardware-in-the-loop verification layer, and the underlying physical actuators. These multiple physical actuators have different dynamic response characteristics and accuracy levels. This invention constructs a complete closed-loop system from natural language commands to the issuance of safety control logic by setting up an adaptive semantic configuration layer and a hardware-in-the-loop verification layer. In actual operation, it receives natural language control commands from the operator, extracts intent using a large language model, generates a discrete target state trajectory matrix, and forces the LLM to output a topology description file in a defined format through a middleware interceptor. By invoking industrial control operators from a pre-defined deterministic standard function library, the checked combined control logic is loaded into the hardware-in-the-loop verification layer for execution. High-frequency synchronous sampling is performed using a hard real-time industrial network to acquire the actual response trajectory. The time-series residual matrix is calculated, and the time-domain transition hysteresis and state amplitude distortion integral of each physical channel are extracted and concatenated to form a control deviation feature vector. The residual-multimodal cue word mapping operator is used to transform the numerical feature vector into natural language cue words containing physical deviation features, which are then fed back into the LLM's context window to drive autoregressive topology correction of the control logic. Simultaneously, the Frobenius norm of the residuals is continuously monitored, and the algorithm executes the control logic if and only if the following conditions are met. When convergence conditions are met, the hardware communication release mechanism is activated, and the control logic for verifying security is sent to the field backbone network. Through LLM semantic parsing and multimodal reference trajectory construction by the planner module, nondeterministic natural language instructions are transformed into deterministic mathematical trajectory matrices, providing a quantitative benchmark for subsequent physical residual comparison. A middleware whitelist mechanism strictly constrains LLM to only call the configuration layer of function blocks FB / FC conforming to the IEC61131-3 standard, fundamentally eliminating the risk of nondeterministic divergence caused by AI directly generating low-level code. Through timing residual deconstruction of the chip in the hardware-in-the-loop verification layer, the hysteresis along time-domain transitions and state amplitude distortion are utilized. By integrating two deterministic mathematical operators, the physical deviations in the continuous time domain are precisely quantified into algebraic feature vectors, achieving a precise mapping between the physical world and the digital space. Through the threshold triggering and semantic conversion mechanism of the residual-multimodal cue word mapping operator, the algebraic deviation vector is automatically converted into LLM-aware natural language alarms, enabling the large language model to perceive the underlying physical dynamic deviations and achieve autoregressive topology correction of the control logic through negative feedback. Through convergence determination based on the Frobenius norm and the gating mechanism for physical control equipment, it is ensured that only control logic that meets the asymptotic stability condition can be sent to the real production line, guaranteeing the robust stability of the system from a control theory perspective.
[0047] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0048] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, equipment, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0049] The adaptive semantic configuration layer and hardware-in-the-loop verification layer may or may not be physically separate. The components displayed as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of these units can be selected to achieve the purpose of this embodiment according to actual needs.
[0050] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0051] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A self-healing control method for industrial control systems based on multimodal temporal residual driving, characterized in that, The specific steps include the following: S1. Construct the system topology architecture, receive natural language control commands, extract intent using a large language model, and discretize and map it into an N×M-dimensional discrete target state trajectory matrix in the control time domain. Where N is the number of physical observation channels and M is the total number of sampling frames; S2. Through deterministic assembly using control laws, the large language model is constrained to not directly output code text, and is forced to output a topology description file in a limited format, which is only allowed to contain configuration parameters and topology connection relationships, thereby calling industrial control operators in a preset deterministic standard function library; if the middleware compares the whitelist and finds that the large language model is trying to call an illegal function, it will be blocked. S3. Observe the hardware in loop mode. Load the checked combinational control logic into the hardware loop verification layer for operation. At this time, the locking pin is in the cut-off state. Use the hard real-time industrial network to perform high-frequency synchronous sampling to obtain the N×M-dimensional discrete actual response trajectory matrix of the heterogeneous actuator. ; S4. Perform temporal residual deconstruction and feature extraction, and calculate the discrete temporal residual matrix of the reference trajectory and the actual response trajectory. The time-domain transition hysteresis of each physical channel is extracted using the residual deconstruction operator. and state amplitude distortion integral splicing and assembling into a control deviation feature vector ; S5. Multimodal self-healing iteration and gated delivery processing: Utilizing the residual-multimodal prompt word mapping operator, the numerical feature vector is transformed... Based on a pre-defined alarm template library, the alarms are converted into natural language prompts containing physical deviation characteristics, which are then fed back into the context window of the large language model to drive its autoregressive topology correction of the control logic; simultaneously, the Frobenius norm of the residuals is continuously monitored. If and only if the convergence criterion is satisfied. At that time, the hardware communication release locking mechanism is activated, and the control logic for verifying security is transmitted to the field backbone network.
2. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 1, characterized in that: The preset deterministic standard function library in step S2 includes function blocks FB / FC that conform to the IEC 61131-3 standard; the topology description file is in JSON format; the middleware compares the whitelist, and if it finds that the large language model is trying to call an illegal function, it will directly block it at the digital layer.
3. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 1, characterized in that: State amplitude distortion integral in step S4 The calculation method is as follows: Using the trapezoidal integral numerical algorithm, the absolute values of the row vectors corresponding to the physical channels in the discrete-time residual matrix E are accumulated and summed in the time domain, i.e. .
4. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 1, characterized in that: In step S4, the method for calculating the time-domain transition hysteresis Δτi is as follows: capture the first frame in which the reference signal undergoes a level transition. The difference between the first frame kact that actually captures the jump in the same direction as the underlying bus, multiplied by the scan sampling period of the industrial bus. ,Right now .
5. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 1, characterized in that: In step S5 The preset minimum system tolerance threshold is defined when the Frobenius norm of the residual matrix ( ) hour( When the control law reaches asymptotic stability, the system sends an unlock command to the hardware communication interception gateway connected to the industrial backbone network. The physical relay inside the gateway closes, and the physical layer switches from cutoff to conduction.
6. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 1, characterized in that: The residual-multimodal cue word mapping operator in step S5 has a built-in threshold judgment and semantic transformation mechanism. When the time domain jump of any physical observation channel is along the hysteresis... Or state amplitude distortion integral When the self-healing prompt word concatenation logic is triggered, the specific physical deviation algebraic calculation results and the corresponding channel hardware topology mapping name are combined and assembled into a natural language description autoregressive prompt word text alarm, which is then fed back to the context window of the large language model.
7. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 1, characterized in that: The system topology includes an adaptive semantic configuration layer and a hardware-in-the-loop verification layer, which are bidirectionally connected. The adaptive semantic configuration layer is used to receive natural language instructions, extract intent using a large language model and generate a discrete target state trajectory matrix, and output a topology description file in a defined format to call industrial control operators in a preset deterministic standard function library. The hardware-in-the-loop verification layer is used to connect logically with the upper-layer adaptive semantic configuration layer through the built-in HIL interception gateway, load the combined control logic to perform hardware-in-the-loop verification, use the hard real-time industrial network to perform high-frequency synchronous sampling to obtain the actual response trajectory, calculate the time-series residuals and perform feature extraction, and control the truncation and conduction of the physical control equipment mechanism according to the residual norm convergence judgment result.
8. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 7, characterized in that: The adaptive semantic configuration layer includes a natural language input module and an LLM semantic parser and planner module, and the output of the natural language input module is communicatively connected to the input of the LLM semantic parser and planner module. The natural language input module is used to receive natural language control commands input by staff and transmit them to the LLM semantic parser and planner module through a logical link; The LLM semantic parser and planner module is used to extract the intent of natural language instructions, discretize and map them into a discrete target state trajectory matrix in the control time domain, and generate a topology description file by calling a preset deterministic standard function library.
9. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 7, characterized in that: The hardware-in-the-loop verification layer includes a timing residual deconstruction chip and a multimodal mapping operator. The output of the timing residual deconstruction chip is communicatively connected to the input of the multimodal mapping operator. The timing residual deconstruction chip is bidirectionally communicatively connected to the underlying physical actuator through an industrial bus. The time-series residual deconstruction chip is used to calculate the discrete time-series residual matrix between the reference trajectory and the actual response trajectory. It calls the residual deconstruction operator to extract the time-domain transition hysteresis and state amplitude distortion integral of each physical channel, and splices them together to form a control deviation feature vector. The multimodal mapping operator is used to convert the numerical control deviation feature vector into natural language prompts containing physical deviation features according to a preset alarm template library, and feed them back to the context window of the upper-level large language model.
10. The AI self-healing control method for industrial control systems based on multimodal temporal residual driving according to claim 7, characterized in that: The hardware-in-the-loop verification layer also includes a physical control device gateway, which is connected to the industrial backbone network and is in a truncated state by default, thus isolating the control logic from the physical actuators.
Citation Information
Patent Citations
Industrial control system performance monitoring and self-healing control method and device
CN117991647B
