Multi-mode adaptive control method and system for industrial process

Abstract

The invention provides an industrial process multi-modal adaptive control method and system, and the method comprises the steps: collecting technological parameters, equipment states and environmental factors in real time through a multi-source heterogeneous sensor, carrying out the feature extraction and fusion through a deep learning model, and generating multi-modal fusion features. The dynamic memory module combines short-term and long-term memory, stores historical experience in a structured manner and supports dynamic retrieval. The cognitive decision module integrates multi-source information by using an attention mechanism, and generates an adaptive control action in combination with reinforcement learning. The adaptive learning mechanism optimizes decision and knowledge management through online and offline learning. The performance evaluation module establishes a multi-dimensional index system and guides the system to be self-optimized. According to the method, the sensing precision, decision-making flexibility and long-term stability of the control system are remarkably improved, and the intelligence and robustness of the industrial process are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of industrial process control technology, and in particular relates to a multimodal adaptive control method and system for industrial processes. Background Technology

[0002] Industrial process control technology is widely used in chemical, manufacturing, and energy industries, aiming to optimize and stabilize production processes through sensing, decision-making, and execution. Existing industrial control systems typically rely on single-modal sensor data, static models, or manually designed control strategies. However, with the increasing complexity of industrial processes, traditional technologies are gradually showing limitations in dealing with multi-source heterogeneous data, dynamic environmental changes, and long-term optimization requirements. Current industrial system control schemes have at least the following shortcomings:

[0003] (1) It usually relies on simple historical data records, lacks a structured memory mechanism, and cannot effectively store and utilize historical experience, making it difficult to optimize real-time decision-making in dynamic environments;

[0004] (2) Most of the decision-making methods are static or based on fixed rules, which are difficult to adapt to complex and ever-changing environments, resulting in poor control performance;

[0005] (3) Lacking learning ability or relying solely on real-time adjustments or periodic offline updates, they are unable to balance short-term response and long-term optimization.

[0006] Therefore, we need to develop a multimodal adaptive control method and system for industrial processes, which adopts a dual memory structure and dynamic retrieval technology, integrates attention mechanism and reinforcement learning, and combines online and offline learning to effectively improve the decision-making flexibility and long-term optimization capability of the control system in complex environments. Summary of the Invention

[0007] The purpose of this invention is to provide a multimodal adaptive control method and system for industrial processes, in order to solve the problems mentioned in the background art, such as the lack of an effective memory mechanism, poor flexibility of the control decision process, and difficulty in balancing short-term response and long-term optimization in existing industrial system control schemes.

[0008] To achieve the above objectives, on the one hand, the present invention provides a multimodal adaptive control method for industrial processes, the method being as follows:

[0009] The multimodal sensing module utilizes multi-source heterogeneous sensors to collect real-time state information of the industrial process, including process parameters, equipment status, and environmental factors. Feature extraction and fusion are performed on the state information to generate multimodal fusion features that characterize the global state of the industrial process.

[0010] Based on the aforementioned multimodal fusion features, the short-term memory unit and long-term memory unit of the dynamic memory module are used to realize the temporal modeling of industrial process state and control behavior and the structured storage of historical experience data, resulting in state memory sequence, action memory sequence and experience representation vector.

[0011] The cognitive decision-making module integrates the state memory sequence, action memory sequence, and experience representation vector using an attention mechanism, and generates the optimal control action through a policy network.

[0012] The optimal control action is applied to the actual industrial process through the control action execution and feedback acquisition module, and the system response and performance feedback are obtained.

[0013] Online learning based on real-time interactive data optimizes the dynamic decision-making capability of the cognitive decision-making module through an adaptive learning module, including: At any given time, the observed state based on the policy network The control actions that are actually executed are obtained. Industrial process response And migrate to the new observation state Generates real-time feedback data Offline learning based on historical experience data optimizes the knowledge management capabilities of the long-term memory unit.

[0014] Through the performance evaluation and optimization module, a multi-dimensional performance evaluation framework is established to quantitatively evaluate the control performance of the control system, and a hierarchical optimization mechanism is established to adaptively adjust the key components and parameters of the control system.

[0015] Based on the aforementioned scheme, the temporal modeling of industrial process state and control behavior specifically involves the following: the short-term memory unit encodes state memory from multimodal fusion features and action memory from control action sequence through a gated recurrent network, and obtains state memory sequence and action memory sequence respectively.

[0016] Based on the aforementioned scheme, the structured storage of the historical experience data includes:

[0017] The long-term memory unit enables the storage, updating, and efficient utilization of historical experience data of industrial processes by constructing an experience knowledge base and a dynamic experience retrieval mechanism, in order to support long-term optimization control decisions.

[0018] Based on the aforementioned scheme, the process of obtaining the state memory sequence includes:

[0019] Based on the multimodal fusion features at the current moment and one's own state at the previous moment The multimodal fusion features are encoded using a gated recurrent network to generate a state memory sequence.

[0020] ,in The mapping function representing the GRU network, Here are the parameters of the GRU network, and D is the dimension of the state memory vector and the action memory vector.

[0021] Based on the aforementioned scheme, the process of acquiring the action memory sequence includes:

[0022] remember ,express The control actions actually performed on the industrial process at all times, including To control the number of variables;

[0023] A GRU network is used to encode the control action sequence into action memory, generating the action memory sequence: ,in For the parameters of the GRU network, action memory sequence It contains the pattern of control action adjustment over a period of time, guiding the system to produce a smooth and coordinated action sequence.

[0024] Based on the aforementioned scheme, the experience knowledge base is... ,in , indicating the first A number of empirical data, The quantity of existing experience data in the experience knowledge base. For historical multimodal fusion features, To control the action vector, To provide real-time feedback data for quantifying the control effect;

[0025] The knowledge base content is dynamically maintained through an adaptive update mechanism. The update of the experience knowledge base is represented as follows:

[0026]

[0027]

[0028] in The maximum capacity of the experience knowledge base is set; when the knowledge base is not full, new experience data is directly added to the knowledge base; when the knowledge base is full, it is necessary to evaluate the importance of each experience data and eliminate the data with the lowest value to make room for new data.

[0029] Based on the aforementioned scheme, the dynamic experience retrieval mechanism is based on multimodal fusion features. Through nearest neighbor search, in the experience knowledge base We select a set of empirical data that is most relevant to the current state at time k as the relevant empirical subset. ;

[0030] Based on the aforementioned relevant subset of experience Through attention mechanisms By assigning importance weights to different empirical data, the most valuable knowledge is extracted to obtain an empirical representation vector. :

[0031]

[0032]

[0033]

[0034] in This represents a multilayer perceptron network. For the Kth in the experience knowledge base The attention weight of each piece of empirical data, i.e., the importance weight. , for the first Feature representation of empirical data.

[0035] Based on the aforementioned scheme, the integration of the state memory sequence, action memory sequence, and experience representation vector using an attention mechanism includes:

[0036] Multimodal fusion features and state memory sequence By concatenating the data, we obtain the state memory representation. ;

[0037] The state memory representation As input, a query-key-value attention mechanism is used to compute the corresponding state-focused representation. Specifically, this involves fusing multimodal features. Mapped to query vector, state memory representation Mapping to key and value vectors, calculating the similarity between the query vector and the key vector, and then... The attention weights are obtained by function normalization, applied to the value vector, and mapped through a perceptron to obtain the state-focused representation. ;

[0038] For empirical representation vectors The corresponding empirical focus representation is calculated using a query-key-value attention mechanism. .

[0039] Based on the aforementioned scheme, the real-time feedback data is designed based on factors including process parameter deviations, energy consumption levels, and safety indicators. The multi-objective reward function is as follows:

[0040]

[0041] in The number of process parameters being adjusted. To control the number of actions, To constrain the number of indicators, they correspond to the dimensions of each sub-item in the multi-objective reward function; Let i be the actual value of the i-th process parameter at time k. For process parameters The set value, To control the action The change For the first One constraint index function; These are the weighting coefficients for the three items.

[0042] On the other hand, the present invention provides a multimodal adaptive control system for an industrial process, used to execute the multimodal adaptive control method for an industrial process described in the foregoing scheme, the system comprising:

[0043] Multimodal sensing module: The multimodal sensing unit uses multi-source heterogeneous sensors to collect the state information of the industrial process in real time. The state information includes process parameters, equipment status and environmental factors. The feature extraction and fusion unit extracts and fuses the state information to generate multimodal fusion features that represent the global state of the industrial process.

[0044] Dynamic memory module: Based on the multimodal fusion features, through short-term memory units and long-term memory units, it realizes the temporal modeling of industrial process state and control behavior and the structured storage of historical experience data, and obtains state memory sequence, action memory sequence and experience representation vector;

[0045] Cognitive decision-making module: Integrates the state memory sequence, action memory sequence and experience representation vector through an attention mechanism, and generates the optimal control action through a policy network;

[0046] Control action execution and feedback acquisition module: applies the optimal control action to the actual industrial process and acquires the system response and performance feedback;

[0047] Adaptive learning module: online learning based on real-time interactive data to optimize the dynamic decision-making ability of the cognitive decision-making module; offline learning based on historical experience data to optimize the knowledge management ability of the long-term memory unit;

[0048] Performance evaluation and optimization module: Establish a multi-dimensional performance evaluation framework to quantitatively evaluate the control performance of the control system, establish a hierarchical optimization mechanism, and adaptively adjust the key components and parameters of the control system.

[0049] The present invention has the following advantages and effects compared with the prior art:

[0050] (1) Introduce a dynamic memory module, adopt a dual memory structure combined with dynamic retrieval technology, construct a collaborative system of short-term memory and long-term memory, realize the structured storage and dynamic retrieval of historical data, improve the system's efficiency in utilizing historical experience, and enhance the accuracy of decision-making and environmental adaptability;

[0051] (2) By applying cognitive decision-making mechanisms, integrating attention mechanisms and reinforcement learning techniques, dynamically fusing multi-source information and generating adaptive control commands, the flexibility and intelligence of decision-making are greatly improved, ensuring the system’s excellent control performance in complex scenarios.

[0052] (3) Implement an adaptive learning mechanism, combine online and offline learning technologies, and continuously optimize the control strategy through real-time feedback and historical data analysis, which effectively enhances the system's adaptability and robustness, and achieves the unity of short-term rapid response and long-term performance improvement. Attached Figure Description

[0053] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0054] Figure 1 This is a flowchart of a multimodal adaptive control method for industrial processes provided in an embodiment of the present invention;

[0055] Figure 2 This is a structural diagram of a multimodal adaptive control system for industrial processes provided in an embodiment of the present invention. Detailed Implementation

[0056] To more clearly explain the purpose, technical solutions, and advantages of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein. On the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0057] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0058] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0059] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0060] The present invention will now be described in detail with reference to specific embodiments:

[0061] Example 1

[0062] As attached Figure 1 As shown, Embodiment 1 of the present invention provides a multimodal adaptive control method for industrial processes, which can be implemented in an industrial control system. The method includes steps S101 to S105, and the specific implementation is as follows:

[0063] Step S101, Multimodal perception and feature extraction fusion: In the multimodal perception module, the multimodal perception unit collects the state information of the industrial process in real time through multi-source heterogeneous sensors, and the feature extraction fusion unit extracts and fuses the state information to generate multimodal fusion features that characterize the global state of the industrial process, so as to achieve comprehensive perception of the industrial process.

[0064] Preferably, the status information includes process parameters, equipment status, and environmental factors;

[0065] Specifically, the process parameters are collected by process parameter sensors such as temperature, pressure, flow rate, and liquid level, and are used as signal data reflecting the physicochemical parameters of the production process, uniformly represented as time-series data. , recorded as process parameter data, where This represents the number of measurement points for process parameters.

[0066] Sensing function based on process parameters Regarding the process parameter data Perform feature transformation to obtain a normalized process state feature vector. :

[0067]

[0068] Among them, It is a process state feature vector The dimension, process parameter perception function It can be constructed based on mechanism analysis, including but not limited to thermodynamic models and fluid dynamic models, to map multi-parameter coupled physical processes into independent characteristic indicators.

[0069] Specifically, the equipment status refers to an assessment of the health of the production equipment. This is achieved by collecting status signals during equipment operation through status monitoring sensors such as vibration, noise, and temperature sensors, and then uniformly representing these signals as time-series data. Recorded as device status data, among which This represents the number of equipment status measurement points.

[0070] The device status data Input to device state awareness function This enables the mapping from raw acquired status signals to device health indicators.

[0071]

[0072] in This is a normalized device state feature vector. This represents the dimension of the device state feature vector. Device state perception function. Machine learning methods, such as support vector machines and convolutional neural networks, can be used to build the system. By training with a large amount of historical operating data, the system can learn the potential patterns of equipment degradation and thus accurately assess the health level of the equipment based on the current state signals.

[0073] Specifically, the environmental factors are collected to monitor the dynamic changes in the production environment, including but not limited to ambient temperature and humidity, air composition, vibration, and noise, and are uniformly represented as environmental factor data. , ,in This represents the number of environmental factor monitoring points.

[0074] Perception function based on environmental factors Environmental factor data Mapped to normalized environment state feature vector :

[0075]

[0076] in, The environmental state feature vector dimension, and the environmental factor perception function. Statistical modeling or machine learning methods can be used to uncover the intrinsic relationship between environmental factors and industrial processes, and to quantitatively characterize the impact patterns of environmental factors.

[0077] It should be noted that the feature vectors of process status, equipment status, and environmental status obtained through multimodal sensing While reflecting the real-time state of industrial processes from different perspectives, the following problems still exist: the physical meanings and numerical scales of features from different modalities differ significantly, making direct fusion difficult; and features within the same modality also suffer from information redundancy and the curse of dimensionality. Therefore, this embodiment uses a feature extraction and fusion unit to perform deep feature extraction and feature fusion on multimodal sensing data.

[0078] Preferably, the feature extraction and fusion includes:

[0079] Specifically, a deep learning model is used to analyze the process state feature vector. Device status feature vector and environmental state feature vector Perform feature transformations separately to extract deep features, i.e.:

[0080]

[0081]

[0082]

[0083] in Deep feature extraction models representing process parameters, equipment status, and environmental factors can employ various deep learning structures, including but not limited to convolutional neural networks (CNN), long short-term memory networks (LSTM), and graph convolutional networks (GCN). , , These are the dimensions of process parameter depth features, equipment status depth features, and environmental status depth features, respectively. For the parameters of the corresponding depth feature extraction model, These represent the extracted deep features, which have a significantly lower dimensionality than the original perceptual data.

[0084] The deep feature extraction model can be trained through unsupervised learning such as autoencoders or supervised learning such as classification tasks, from industrial data, i.e., process state feature vectors. Device status feature vector and environmental state feature vector It automatically learns the high-order features that best reflect the inherent laws of industrial processes.

[0085] Furthermore, a feature fusion mechanism is employed to map the deep features to a unified process state representation space, resulting in multimodal fused features. :

[0086]

[0087] in For multimodal fusion features Dimensions The feature fusion function can employ simple feature concatenation and weighted fusion, or it can use an attention mechanism to dynamically adjust the importance of different features. Furthermore, it can utilize feature interaction and autoencoder structures to mine the correlations between different features. The multimodal fusion feature... It integrates the multifaceted influences of process parameters, equipment status, and environmental factors on industrial processes, becoming a key characterization of the real-time state of complex industrial systems.

[0088] In summary, after the above processing, the industrial system is obtained in... Multimodal fusion features at different times The massive amounts of industrial data acquired by multimodal heterogeneous sensors are processed through hierarchical perception, learning, and fusion to form a highly generalized representation of the system state. This representation significantly reduces the data dimensionality, alleviating the computational burden of subsequent analysis and decision-making. On the other hand, it retains the key state information of the industrial process to the greatest extent possible, laying a data foundation for achieving accurate perception.

[0089] It should be noted that the core of the multimodal sensing unit is to construct mechanistic or data-driven sensing models for different physical processes, thereby transforming and improving the data from raw data into state features. The feature extraction and fusion unit focuses on fully utilizing the powerful representational and data mining capabilities of deep learning models to learn global high-order features from local features, ultimately forming a state representation that balances comprehensiveness and simplicity. These two units work together to maximize the discovery of state patterns behind massive amounts of industrial process data, reduce reliance on manual feature engineering, and improve the adaptability and robustness of the sensing module.

[0090] Step S102, Dynamic Memory: Based on the multimodal fusion features By using the short-term memory unit and long-term memory unit of the dynamic memory module, the temporal modeling of industrial process state and control behavior and the structured storage of historical experience data can be realized.

[0091] Preferably, the short-term memory unit achieves feature fusion from multiple modalities through a gated recurrent network. The encoding from state memory and from control action sequence to action memory yields state memory sequence and action memory sequence, respectively.

[0092] Specifically, based on the multimodal fusion features at the current moment and one's own state at the previous moment The multimodal fusion features are encoded using a gated recurrent network (GRU) to generate a state memory sequence.

[0093] ,in The mapping function representing the GRU network, Here, D represents the parameters of the GRU network, and D is the dimension of the state memory vector and action memory vector, both being D-dimensional real-valued vectors. GRU adaptively determines the forgetting, updating, and output of memory information through a gating mechanism, effectively capturing long-short-term dependencies in time-series data. State memory sequence. The key information about the state evolution of an industrial system over a period of time is stored in the form of a memory vector, reflecting the dynamic characteristics of the system.

[0094] It should be noted that, in addition to remembering the system state, it is also necessary to have short-term memory of the executed control actions in order to establish "motor memory" of state-action mapping;

[0095] Specifically, record Let represent the control action actually performed on the industrial process at any given time k. In this embodiment, this action is generated by the cognitive decision-making module (described later) based on the principle of maximizing the expected cumulative reward evaluated by the state-action value function Q(o,u). Here, u(k) serves as historical information to construct a short-term memory of the control behavior, where... To control the number of variables, a GRU network is used to encode the control action sequence into action memory, generating the action memory sequence:

[0096]

[0097] in, For the parameters of the GRU network, action memory sequence It contains the pattern of control action adjustment over a period of time, guiding the system to produce a smooth and coordinated action sequence.

[0098] It should be noted that while short-term memory units track the state changes and action adjustments of industrial system processes in real time through an encoding mechanism, their memory capacity is limited and cannot cope with long-term optimization control problems in industrial systems. Therefore, long-term memory units are further introduced to learn and store process control knowledge from historical experience data, providing long-term knowledge support for the cognitive decision-making part in optimization step S103.

[0099] Preferably, the long-term memory unit enables the storage, updating, and efficient utilization of historical experience in industrial processes by constructing an experience knowledge base and a dynamic experience retrieval mechanism, thereby supporting long-term optimization control decisions.

[0100] Specifically, the experience knowledge base is ,in , indicating the first A number of empirical data, The quantity of existing experience data in the experience knowledge base. For historical multimodal fusion features, To control the action vector, This provides real-time feedback data to quantify control effectiveness. The experience knowledge base K records the evaluation of the effects of specific control actions taken by the industrial system under different states in the form of discrete experience data, containing a wealth of valuable process control experience. The knowledge base content is dynamically maintained through an adaptive update mechanism to ensure the timeliness and richness of the knowledge. New experience data is continuously generated as the industrial system operates, triggering updates to the experience knowledge base, thus continuously improving its capacity and richness. The update of the experience knowledge base can be represented as:

[0101]

[0102]

[0103] in To the maximum capacity of the experience knowledge base, The historical experience data to be phased out is indexed in knowledge base K. When the knowledge base is not full, new experience data is directly added to the knowledge base; when the knowledge base is full, some experience data needs to be phased out to make room for new data. The phase-out of experience data can be based on the first-in, first-out rule, or a dynamic experience retrieval mechanism can be designed as follows: a value assessment function evaluates the importance of each piece of experience data and phases out the data with the lowest value.

[0104] For example, to fully utilize the experience knowledge base Historical experience knowledge requires the design of a dynamic experience retrieval mechanism that is relevant to the current state.

[0105] Specifically, the dynamic experience retrieval mechanism is based on multimodal fusion features. Through nearest neighbor search, in the experience knowledge base We select a set of empirical data that is most relevant to the current state at time k as the relevant empirical subset. :

[0106]

[0107] in For the distance metric function of multimodal fusion features, This represents the distance threshold for relevant experience. The nearest neighbor search is a local search; the local retrieval strategy can quickly identify key experiences highly relevant to the current decision, avoiding the computational overhead of a full database search.

[0108] Based on the aforementioned relevant subset of experience Through attention mechanisms By assigning importance weights to different empirical data, the most valuable knowledge is extracted to obtain an empirical representation vector. :

[0109]

[0110]

[0111] in This represents a multilayer perceptron network. For the Kth in the experience knowledge base The attention weight of each piece of empirical data, i.e., the importance weight. , for the first Feature representation of empirical data. The attention mechanism can adaptively adjust the activation level of different experiences, dynamically adapting to the needs of the decision-making task.

[0112] In summary, the working memory module is based on two complementary learning systems: short-term memory and long-term memory. These systems are responsible for the continuous encoding of immediate states and the discrete storage of historical experiences, respectively. When faced with new decision-making tasks, the short-term memory unit provides the decision system with current and recent state information, while the long-term memory unit retrieves the knowledge and experience most relevant to the current task. The two memory units work together to provide comprehensive, accurate, and efficient memory information support for the decision system, enhancing the cognitive capabilities of complex industrial control systems. The short-term memory unit employs a neural network-driven end-to-end memory encoding method, adaptively optimizing memory representation through data-driven approaches. The long-term memory unit uses an instance-based discrete memory method, continuously absorbing new experiences and adjusting the knowledge structure according to task requirements, demonstrating strong knowledge learning capabilities. This hybrid paradigm, integrating continuous and discrete memory, to some extent mimics the structured organization and flexible application mechanisms of human brain memory, making it a high-performance and generalizable working memory model for industrial processes.

[0113] Step S103: Cognitive Decision-Making. Based on multimodal fusion features and dynamic memory information, intelligent control decisions for industrial processes are achieved through a cognitive decision-making module. This involves integrating multi-source information using an attention mechanism and generating optimal control actions through a policy network to achieve adaptive process control. This is the core of the industrial process adaptive control system.

[0114] It should be noted that the multimodal fusion features representing the instantaneous state of the industrial process obtained in step S101 The short-term memory sequence encoding recent state changes obtained in step S102 is called the state memory sequence. Long-term memory that condenses historical experience and knowledge, i.e., experience representation vectors. These information are usually very high in dimensionality, and the importance of different information to decision-making varies greatly. In order to focus on key information related to decision-making, this embodiment introduces an attention mechanism to assign importance weights and focus attention on these information.

[0115] Preferably, the integration of multi-source information using an attention mechanism includes:

[0116] Multimodal fusion features and state memory sequence By concatenating the data, we obtain the state memory representation. ;

[0117] The state memory representation As input, a query-key-value attention mechanism is used to compute the corresponding state-focused representation. :

[0118]

[0119]

[0120]

[0121]

[0122]

[0123] in , , These are query, key, and value vectors, respectively. These are the weights of the query, key, and value matrices, respectively. For the corresponding bias term, For attention weights, This is the scaling factor. This represents a multilayer perceptron mapping for state attention. In the above process, multimodal fusion features are first... Mapped to query vector, state memory representation Mapping to key and value vectors, calculating the similarity between the query vector and the key vector, and then... The attention weights are obtained by function normalization, applied to the value vector, and mapped through a perceptron to obtain the state-focused representation. ;

[0124] For long-term memory, i.e., empirical representation vectors We employ a query-key-value attention mechanism consistent with the state information for weighted aggregation to obtain an empirically focused representation. :

[0125]

[0126]

[0127]

[0128]

[0129]

[0130] Preferably, the generation of optimal control actions through the policy network is based on the state-focused representation. and experience focus Combined with action memory sequences The stored historical control action information is used to generate the optimal control action at time k through a policy network. The details are as follows:

[0131] Policy Network A deep neural network structure is adopted, and the network parameters are: According to the state-action value function The optimal control action is obtained by approximating the optimal strategy. :

[0132]

[0133]

[0134] Wherein the observed state of the system at time k , Expressing expectations, The expected cumulative return starting from the current time t is calculated as follows: , φ is the discount factor; since the true state-action value function Q(o,u) cannot be obtained directly, this invention uses a deep neural network (i.e., a value network) composed of parameters φ to approximate it, denoted as φ. The policy network employs a deep learning approach. Value Network (DQN) Use parameters The neural network structure approximates the state-action value function. DQN transforms the optimal control policy learning problem into an approximation and optimization problem of the state-action value function. It uses a deep neural network for function approximation and combines reinforcement learning algorithms to iteratively update parameters to learn the optimal control policy and obtain the optimal control action. .

[0135] In summary, the cognitive decision-making module integrates immediate state perception, short-term memory, and long-term knowledge and experience to construct a multi-layered perception foundation. Based on a combination of attention mechanisms and deep reinforcement learning, it achieves adaptive optimization control of industrial processes. During the decision-making process, the system extracts key features relevant to the control objective from current state information and the experience knowledge base through the attention mechanism, forming a focused representation that is input into the DQN policy network. Then, the optimal control action is obtained through function approximation and reinforcement learning algorithm iteration. This module utilizes the dynamic feedback of the perception system on the one hand, and mines long-term experience information on the other, achieving adaptive control and policy updates throughout the entire process. The overall mechanism demonstrates the brain-like system's ability to focus on key information and optimize complex tasks, providing a reference path for the design of industrial control systems oriented towards biological intelligence.

[0136] Step S104, Control Action Execution and Feedback Acquisition: Through the control action execution and feedback acquisition module, the optimal control action generated by the cognitive decision-making module is processed. It is applied to actual industrial processes and obtains system response and performance feedback;

[0137] Specifically, the optimal control action = [ (k), (k), ..., (k)]ᵀ By acting on the corresponding control loop of the industrial process through the actuator, the operating variables such as valve opening, pump speed, and heating power are adjusted, so that the state of the industrial process changes accordingly;

[0138] In optimal control action At the next moment k+1 after execution, the state information of the industrial process is re-acquired through the multimodal sensing module to obtain a new observed state o(k+1), and the real-time feedback data r(k) is calculated based on the control objective to evaluate the optimal control action. The control effect.

[0139] Step S105, Adaptive Learning Mechanism: Online learning based on real-time interactive data, optimizing the dynamic decision-making ability of the cognitive decision-making module through the adaptive learning module, and offline learning based on historical experience data to optimize the long-term knowledge management ability of the long-term memory unit.

[0140] Preferably, the online learning is based on real-time feedback data generated during the interaction between the control system and the industrial process, which is used to adjust and optimize the cognitive decision-making module in real time.

[0141] Specifically, in At any given moment, based on the observed state Obtain the optimal control action The industrial process responds to this control action and migrates to the new observed state. At the same time, it generates real-time feedback data. , Used to reflect control actions Impact on industrial processes, evaluation of control actions The quality of the process can be assessed by designing real-time feedback data based on factors such as process parameter deviations, energy consumption levels, and safety indicators. Multi-objective reward function:

[0142]

[0143] in The number of process parameters being adjusted. To control the number of actions, To constrain the number of indicators, they correspond to the dimensions of each sub-item in the multi-objective reward function; Let i be the actual value of the i-th process parameter at time k. For process parameters The set value, To control the action The change For the first Each constraint indicator function may include safety range, energy consumption limit, etc. These are the weighting coefficients for the three items. The target reward function of the real-time feedback data comprehensively considers control accuracy, control action smoothness, and process constraint satisfaction, and achieves dynamic balance of the multi-objective control strategy through a weighted summation.

[0144] The real-time feedback data is compared with the current observation status. Controlling actions The observation status at the next moment Assemble into a complete set of empirical data Stored in the experience replay pool middle:

[0145]

[0146] From the experience replay pool Randomly sample a batch of empirical data As the first empirical tuple, it is input into the policy network for training. Specifically, the sampled observation states are used... Its corresponding control actions Input the value network and calculate the predicted state-action values. At the same time, data will be fed back in real time. Observation state at the next moment Input parameters are Construct a target value network to define target values. The target value network has the same network structure as the value network; the loss function is calculated by minimizing the temporal difference (TD) error between the predicted value and the target value. And used to update the parameters of the current value network. ;

[0147]

[0148] in , The parameters of the target value network represent the expected value of empirical data e sampled from batch B. Slowly track value network parameters using soft updates. get: ,in This is a soft update coefficient;

[0149] Minimize the loss function using stochastic gradient descent. For value network parameters and policy network parameters Update: ,in Let s be the learning rates of the value network and the policy network, respectively. Let the objective function be the policy network objective function; in the next iteration, the updated parameters... and Replace the old parameters and Online learning enables the cognitive decision-making module to continuously optimize control actions based on real-time feedback data. This improves the adaptive capability of the control system.

[0150] Preferably, the offline learning specifically includes:

[0151] Sample a batch of empirical data from the empirical knowledge base K. , as the second empirical tuple; the value network is trained using the Monte Carlo (MC) method, and the loss function is calculated by minimizing the mean squared error of the MC reward estimation as follows:

[0152]

[0153] Compared to the Time Difference Error (TD) learning approach, which relies on temporal difference errors, Motion-Motion (MC) learning uses the cumulative reward from empirical data as the supervision target, resulting in lower estimation bias. It is suitable for processing historical data with a large time span and wide policy distribution in long-term memory. Offline learning enhances the policy network's ability to model long-term rewards, contributing to the optimization of control action stability and generalization performance.

[0154] In summary, the adaptive learning mechanism continuously optimizes the dynamic decision-making capabilities of the control system by combining online and offline learning. Through information interaction and collaboration between different modules, it achieves dynamic cyclical optimization of the perception-memory-decision closed loop. This multi-level, two-way interactive learning paradigm enhances the adaptability and robustness of the control system on the one hand, and achieves extractive intelligent growth through dynamic knowledge updates and long-term accumulation on the other. The adaptive learning mechanism endows the control system with a continuous, biological-like capacity.

[0155] Step S106, Performance Evaluation and Optimization: Through the performance evaluation and optimization module, a multi-dimensional performance evaluation framework is established to quantitatively evaluate the control performance of the control system, establish a hierarchical optimization mechanism, and adaptively adjust the key components and parameters of the control system.

[0156] Preferably, since industrial process control typically involves multiple performance objectives, the evaluation metrics in the multidimensional performance evaluation framework include:

[0157] (1) Control accuracy, which characterizes the ability of the control system to track the set value, is quantified by the root mean square error of the deviation of the target variable: ,in To evaluate the number of time steps, For the first The target variables are The actual value at time, Set a value for it. The number of target variables. The smaller the value, the higher the control precision.

[0158] (2) Energy efficiency, which characterizes the energy utilization level of the control system in completing the control task, is expressed by the energy consumption per unit output. To quantify: ,in To assess the total energy consumption over the period, Total output; The smaller the value, the higher the energy efficiency.

[0159] (3) Safety indicators, which characterize the degree to which constraints are met in industrial processes, are measured by the total duration of the out-of-limit variables. To quantify: ,in The number of constraints. Let I(∙) be the sampling period, and I(∙) be the indicator function, when the constraints are... If violated, the value is 1; otherwise, it is 0. The smaller the value, the higher the safety of the industrial system.

[0160] Furthermore, the three evaluation indicators mentioned above are normalized and weighted to obtain a comprehensive performance index. :

[0161]

[0162] in, These are the weighting coefficients for the three evaluation indicators, satisfying... The normalization of the three evaluation metrics uses max-min normalization to map each evaluation metric to... Interval: ,in These are the maximum and minimum values ​​of the evaluation indicators, respectively.

[0163] Based on the aforementioned comprehensive performance indicators The control performance of the control system is quantitatively evaluated, and this evaluation is used as an optimization target to guide the adjustment and improvement of the control system.

[0164] It should be noted that, considering the complexity of industrial process control tasks, the self-optimization of the control system should be carried out at multiple levels.

[0165] Preferably, the hierarchical optimization mechanism includes:

[0166] (1) Decision-making level self-optimization:

[0167] by To achieve the learning objective, the policy network is continuously optimized using reinforcement learning algorithms such as policy gradient. parameters : ,in For state distribution, To accumulate returns, For state value functions, In the policy network Calculate the expectation of the state-action distribution generated by interaction with the environment;

[0168] According to the policy gradient theorem, the parameters are updated in the direction that increases the objective function: ,in This represents the policy learning rate. After multiple rounds of policy iteration, the policy network will converge to the optimal control action. .

[0169] (2) Memory layer self-optimization:

[0170] It should be noted that long-term memory units form a knowledge base through offline learning from historical data, but the effectiveness of this knowledge decreases with the dynamic changes in industrial processes. Therefore, it is necessary to periodically evaluate and optimize the experiential knowledge base K, as follows:

[0171] Sample a batch of empirical data from the empirical knowledge base K: ;

[0172] This batch of empirical data was re-evaluated using the latest strategy network and evaluation metrics to obtain updated reward values. :

[0173]

[0174] Based on reward value To achieve dynamic updating and optimization of the experience knowledge base K, i.e., the updated experience knowledge base is... Then its update rule can be expressed as:

[0175]

[0176] in These represent the updated reward value or the re-evaluated reward value of the lower subset of experience, the higher value subset of experience, and the newly acquired subset of experience, respectively.

[0177] (3) Adaptive optimization of parameters in the perception layer and the learning layer:

[0178] Automatic hyperparameter tuning: Using the network's key hyperparameters as optimization variables, the optimal hyperparameter vector is found online through an intelligent search method. : ,in For hyperparameter vectors, The hyperparameter search space is defined as follows: the key hyperparameters are the key hyperparameters of each neural network in the entire control system, including: the learning rate and number of network layers of the deep feature extraction network, the number of hidden units and forget gate parameters of the GRU network, the learning rate and batch size of the policy network, and the number of heads and dimensions of the attention mechanism.

[0179] Automatic Network Architecture Search: This method utilizes Neural Architecture Search (NAS) technology to search for the optimal network architecture within the architecture search space. : ,in For network structure, This refers to the network structure search space. The network structure refers to the key network structures in the control system, including: the architecture of the feature extraction network (a combination of CNN / LSTM / GCN), the depth and width of the decision network, and the structural design of the attention mechanism;

[0180] Through this multi-layered self-optimization mechanism, the performance of the control system is continuously improved at different levels, enabling the industrial process brain-inspired adaptive control system to achieve sustained performance enhancement and long-term stability. Comprehensive evaluation indicators provide clear direction and quantitative feedback for system optimization. Strategy optimization at the decision-making level, knowledge optimization at the memory level, and parameter optimization at the perception and learning level work together synergistically to ultimately achieve overall system optimization.

[0181] In this embodiment, a multi-layered intelligent control architecture is proposed to address the insufficient adaptability of traditional systems in complex environments. In implementation, the multimodal perception module integrates multiple sensors to collect and fuse process, equipment, and environmental data in real time. High-order features are extracted through deep learning to form a global state representation. The dynamic memory module employs a dual memory structure: short-term memory encodes recent states and action sequences, while long-term memory constructs an experience knowledge base and supports dynamic retrieval, ensuring efficient utilization of historical experience. The cognitive decision-making module integrates attention mechanisms and reinforcement learning to dynamically focus on key information and optimize control strategies, improving decision-making adaptability. The adaptive learning mechanism combines online real-time adjustments with offline deep analysis to continuously optimize system performance. The performance evaluation module quantifies control effectiveness through multi-dimensional indicators, driving self-optimization across the decision-making, memory, and perception layers.

[0182] Example 2

[0183] As attached Figure 2 The diagram shows a multimodal adaptive control system S200 for an industrial process provided in this embodiment, used to execute a multimodal adaptive control method for an industrial process described in Embodiment 1. It includes a multimodal sensing module S201, a dynamic memory module S202, a cognitive decision-making module S203, a control action execution and feedback acquisition module S204, an adaptive learning module S205, and a performance evaluation and optimization module S206, wherein:

[0184] The multimodal sensing module S201 includes a multimodal sensing unit and a feature extraction and fusion unit. The multimodal sensing unit uses multi-source heterogeneous sensors to collect real-time state information of the industrial process. The state information includes process parameters, equipment status, and environmental factors. The feature extraction and fusion unit extracts and fuses features from the state information to generate multimodal fusion features that characterize the global state of the industrial process.

[0185] The dynamic memory module S202 includes a short-term memory unit and a long-term memory unit, which are used to realize the temporal modeling of industrial process state and control behavior and the structured storage of historical experience data based on the multimodal fusion features through the short-term memory unit and the long-term memory unit, so as to obtain state memory sequence, action memory sequence and experience representation vector;

[0186] The cognitive decision-making module S203 is used to integrate the state memory sequence, action memory sequence and experience representation vector through an attention mechanism, and generate the optimal control action through a policy network.

[0187] The control action execution and feedback acquisition module S204 is used to apply the optimal control action to the actual industrial process and acquire the system response and performance feedback;

[0188] The adaptive learning module S205 is used for online learning based on real-time interactive data to optimize the dynamic decision-making ability of the cognitive decision-making module; and for offline learning based on historical experience data to optimize the knowledge management ability of the long-term memory unit.

[0189] The performance evaluation and optimization module S206 is used to establish a multi-dimensional performance evaluation framework, quantitatively evaluate the control performance of the control system, establish a hierarchical optimization mechanism, and adaptively adjust the key components and parameters of the control system.

[0190] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims. It should be understood that the invention is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A multimodal adaptive control method for industrial processes, characterized in that, include: The multimodal sensing module utilizes multi-source heterogeneous sensors to collect real-time status information of industrial processes, including process parameters, equipment status, and environmental factors. The state information is subjected to feature extraction and fusion to generate multimodal fusion features that characterize the global state of the industrial process; Based on the aforementioned multimodal fusion features, the short-term memory unit and long-term memory unit of the dynamic memory module are used to realize the temporal modeling of industrial process state and control behavior and the structured storage of historical experience data, resulting in state memory sequence, action memory sequence and experience representation vector. The cognitive decision-making module integrates the state memory sequence, action memory sequence, and experience representation vector using an attention mechanism, and generates the optimal control action through a policy network. The optimal control action is applied to the actual industrial process through the control action execution and feedback acquisition module, and the system response and performance feedback are obtained. Online learning based on real-time interactive data optimizes the dynamic decision-making capability of the cognitive decision-making module through an adaptive learning module, including: At any given time, the observed state based on the policy network The control actions that are actually executed are obtained. Industrial process response And migrate to the new observation state Generates real-time feedback data Offline learning based on historical experience data optimizes the knowledge management capabilities of the long-term memory unit. Through the performance evaluation and optimization module, a multi-dimensional performance evaluation framework is established to quantitatively evaluate the control performance of the control system, and a hierarchical optimization mechanism is established to adaptively adjust the key components and parameters of the control system.

2. The multimodal adaptive control method for industrial processes according to claim 1, characterized in that, The temporal modeling of industrial process state and control behavior specifically involves the short-term memory unit encoding state memory and action memory from multimodal fusion features and control action sequence through a gated recurrent network, respectively obtaining state memory sequence and action memory sequence.

3. The multimodal adaptive control method for industrial processes according to claim 1, characterized in that, The structured storage of the historical experience data includes: The long-term memory unit enables the storage, updating, and efficient utilization of historical experience data of industrial processes by constructing an experience knowledge base and a dynamic experience retrieval mechanism, in order to support long-term optimization control decisions.

4. The multimodal adaptive control method for industrial processes according to claim 2, characterized in that, The process of obtaining the state memory sequence includes: Based on the multimodal fusion features at the current moment and one's own state at the previous moment The multimodal fusion features are encoded using a gated recurrent network to generate a state memory sequence. ,in The mapping function representing the GRU network, Here are the parameters of the GRU network, and D is the dimension of the state memory vector and the action memory vector.

5. The multimodal adaptive control method for industrial processes according to claim 4, characterized in that, The process of acquiring the action memory sequence includes: remember ,express The control actions actually performed on the industrial process at all times, including To control the number of variables; A GRU network is used to encode the control action sequence into action memory, generating the action memory sequence: ,in For the parameters of the GRU network, action memory sequence It contains the pattern of control action adjustment over a period of time, guiding the system to produce a smooth and coordinated action sequence.

6. The multimodal adaptive control method for industrial processes according to claim 3, characterized in that, Experience knowledge base is ,in , indicating the first A number of empirical data, The quantity of existing experience data in the experience knowledge base. For historical multimodal fusion features, To control the action vector, To provide real-time feedback data for quantifying the control effect; The knowledge base content is dynamically maintained through an adaptive update mechanism. The update of the experience knowledge base is represented as follows: ,in The maximum capacity of the experience knowledge base is set; when the knowledge base is not full, new experience data is directly added to the knowledge base; when the knowledge base is full, it is necessary to evaluate the importance of each experience data and eliminate the data with the lowest value to make room for new data.

7. The multimodal adaptive control method for industrial processes according to claim 6, characterized in that, The dynamic experience retrieval mechanism is based on multimodal fusion features. Through nearest neighbor search, in the experience knowledge base We select a set of empirical data that is most relevant to the current state at time k as the relevant empirical subset. ; Based on the aforementioned relevant subset of experience Through attention mechanisms By assigning importance weights to different empirical data, the most valuable knowledge is extracted to obtain an empirical representation vector. : , ,in This represents a multilayer perceptron network. For the Kth in the experience knowledge base The attention weight of each piece of empirical data, i.e., the importance weight. , for the first Feature representation of empirical data.

8. The multimodal adaptive control method for industrial processes according to claim 1, characterized in that, The integration of the state memory sequence, action memory sequence, and experience representation vector using an attention mechanism includes: Multimodal fusion features and state memory sequence By concatenating the data, we obtain the state memory representation. ; The state memory representation As input, a query-key-value attention mechanism is used to compute the corresponding state-focused representation. Specifically, this involves fusing multimodal features. Mapped to query vector, state memory representation Mapping to key and value vectors, calculating the similarity between the query vector and the key vector, and then... The attention weights are obtained by function normalization, applied to the value vector, and mapped through a perceptron to obtain the state-focused representation. ; For empirical representation vectors The corresponding empirical focus representation is calculated using a query-key-value attention mechanism. .

9. The multimodal adaptive control method for industrial processes according to claim 8, characterized in that, The real-time feedback data is designed based on factors including process parameter deviations, energy consumption levels, and safety indicators. The multi-objective reward function is as follows: ,in The number of process parameters being adjusted. To control the number of actions, To constrain the number of indicators, they correspond to the dimensions of each sub-item in the multi-objective reward function; Let i be the actual value of the i-th process parameter at time k. For process parameters The set value, To control the action The change For the first One constraint index function; These are the weighting coefficients for the three items.

10. A multimodal adaptive control system for an industrial process, used to execute the multimodal adaptive control method for an industrial process according to any one of claims 1-9, characterized in that, include: Multimodal sensing module: Real-time acquisition of industrial process status information through multi-modal sensing units using multi-source heterogeneous sensors, including process parameters, equipment status, and environmental factors; The state information is extracted and fused by the feature extraction and fusion unit to generate multimodal fusion features that characterize the global state of the industrial process. Dynamic memory module: Based on the multimodal fusion features, through short-term memory units and long-term memory units, it realizes the temporal modeling of industrial process state and control behavior and the structured storage of historical experience data, and obtains state memory sequence, action memory sequence and experience representation vector; Cognitive decision-making module: Integrates the state memory sequence, action memory sequence and experience representation vector through an attention mechanism, and generates the optimal control action through a policy network; Control action execution and feedback acquisition module: applies the optimal control action to the actual industrial process and acquires the system response and performance feedback; Adaptive learning module: Based on online learning using real-time interactive data, it optimizes the dynamic decision-making capabilities of the cognitive decision-making module; Offline learning based on historical experience data optimizes the knowledge management capabilities of the long-term memory unit; Performance evaluation and optimization module: Establish a multi-dimensional performance evaluation framework to quantitatively evaluate the control performance of the control system, establish a hierarchical optimization mechanism, and adaptively adjust the key components and parameters of the control system.

Images