Industrial unattended intelligent control method and system based on AI model
By constructing a task chain representation model and hierarchical reinforcement learning, the main task is decomposed into sub-tasks and a policy model is trained, which solves the problems of task dependence and policy conflict in unattended control and realizes adaptive cooperative control and long-term stable operation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-04-03
AI Technical Summary
Existing unattended control technologies cannot effectively handle the dependencies between tasks, multi-level task triggering logic, and cross-process collaborative behavior. They lack unified modeling of hierarchical structures, resulting in fragmented and conflicting control strategies. Furthermore, they lack an adaptive update mechanism, making it difficult to guarantee long-term stable operation.
By constructing a task chain representation model, the main task is decomposed into sub-tasks and mapped to atomic-level control actions, a hierarchical control instruction space is established, and high-level, sub-level, and low-level policies are trained to form a hierarchical reinforcement learning policy model. The policy is updated in real time to achieve adaptive cooperative control.
It achieves a unified expression of complex industrial processes, avoids task fragmentation and strategy conflicts, improves control flexibility and responsiveness, and ensures autonomous control and long-term stable operation under unattended conditions.
Smart Images
Figure CN121785256A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial intelligent control technology, and in particular to an unattended intelligent control method and system for industrial applications based on an AI model. Background Technology
[0002] With the advancement of informatization and automation, various unattended control technologies have emerged, such as rule-based, threshold-based, and single-model-based control. However, these methods primarily rely on preset rules or single-level models for control decisions, failing to effectively handle task dependencies between devices, multi-level task triggering logic, and cross-process collaborative behavior. In scenarios with long task chains and strong task state correlations, traditional methods often exhibit fragmented control strategies, weak cross-task scheduling capabilities, and difficulty adapting to dynamic changes in task sequences or operational disturbances.
[0003] Furthermore, existing solutions generally lack unified modeling capabilities for hierarchical structures, making it difficult to support the collaborative relationships between task-level, sub-task-level, and underlying equipment actions. This leads to policy conflicts or behavioral inconsistencies in the control system during task switching, operating condition fluctuations, and multi-task parallelism. On the other hand, existing technologies generally lack detection and adaptive update mechanisms for policy decay during long-term operation, making it impossible to achieve continuous learning and self-evolution in unattended scenarios and ensuring long-term stable operation. Therefore, we propose an AI-based industrial unattended intelligent control method and system. Summary of the Invention
[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing an AI-based unattended intelligent control method and system for industrial applications, thereby resolving the technical problems mentioned in the background section.
[0005] To achieve the above objectives, the present invention provides the following technical solution: An AI-based unattended intelligent control method for industry includes the following steps: S1. By preprocessing the historical operation records, operation instruction sequences and equipment status sequences of the target industrial process, multi-level task features are extracted. Based on the dependencies and triggering conditions between tasks, a task chain representation model containing main tasks, sub-tasks and constraint tasks is established, so that subsequent control strategies can be analyzed and executed with the task chain as a unified target structure. S2. Based on the task chain representation model, each main task is decomposed into a set of executable subtasks, and each subtask is further mapped into a set of atomic-level control actions. A hierarchical control instruction space containing the main task level, subtask level and atomic action level is constructed so that the subsequent policy learning process can perform optimal decision search under a unified hierarchical structure. S3. In the hierarchical control instruction space, the completion rate of the main task of the task chain is used as the high-level reward indicator, the completion efficiency of the sub-task is used as the mid-level reward indicator, and the stability of atomic-level actions is used as the low-level reward indicator. The high-level policy, sub-level policy and low-level policy are trained respectively, and the hierarchical reinforcement learning policy model is formed by cross-layer coupling constraints, so that the improved policy has the ability to adaptively and collaboratively cooperate across task chains. S4. During the real-time operation of the industrial system, the task chain representation model is used as the task context input to map the real-time equipment status, process parameters and operation requirements to the hierarchical reinforcement learning strategy model. The high-level strategy selects the current main task, the sub-level strategy selects the target sub-task, and the low-level strategy generates atomic control actions and sends them to the execution unit, thereby completing multi-instruction level collaborative control driven by the task chain under unattended conditions. S5. After completing the control execution, collect the task chain execution results, task priority changes, subtask execution deviations and atomic action performance indicators, and write the feedback information back to the hierarchical reinforcement learning policy model. Update the value function and re-optimize the policy of each layer, so that the model has the control capabilities of self-evolution, self-adaptation and cross-process self-coordination under long-term unattended conditions.
[0006] Further, S1 specifically involves: collecting historical operation records, operation instruction sequences, and equipment status sequences of the industrial process; performing timestamp alignment, noise filtering, and format unification on the data to form a standardized sequence set that can be used for task feature extraction; based on the standardized sequence set, extracting multi-level task features including operation objectives, equipment responses, and state transition patterns to form a task feature set describing task behavior; based on the task feature set, identifying the preconditions, triggering conditions, and dependency structures between tasks, and establishing a relationship graph structure between main tasks, sub-tasks, and constraint tasks; based on the relationship graph structure, connecting the main tasks, sub-tasks, and constraint tasks in the execution order to form a task path, and constructing a hierarchical task chain framework based on task triggering conditions; and encoding the task chain framework into a task chain representation model that can be called upon by subsequent strategy learning, so that subsequent steps all use this task chain representation model as a unified task structure.
[0007] Furthermore, S2 specifically involves: based on the task chain representation model, identifying the target state and completion conditions of each main task, and determining the set of main tasks; for each main task, decomposing it into a set of executable subtasks according to its completion conditions, and establishing a correspondence between subtasks and main tasks; for each subtask, breaking it down into a set of low-level executable atomic control actions, and establishing a correspondence between atomic actions and subtasks; based on the set of main tasks, the set of subtasks, and the set of atomic actions, constructing a hierarchical control instruction space that includes the main task level, the subtask level, and the atomic action level; and associating the hierarchical control instruction space with the task chain representation model, enabling subsequent policy training to perform optimal decision search within a unified instruction structure.
[0008] Furthermore, S3 specifically involves: using the hierarchical control instruction space as the input to the training environment, constructing a hierarchical policy architecture that includes high-level policies, sub-level policies, and low-level policies; setting the main task completion rate as the high-level reward indicator, the sub-task completion efficiency as the mid-level reward indicator, and the atomic action stability as the low-level reward indicator; training the high-level policy model, sub-level policy model, and low-level policy model separately under the constraint of the reward indicator, so that each policy layer obtains the optimal behavior pattern matching its task level; based on cross-layer coupling constraints, jointly optimizing the high-level policy model, sub-level policy model, and low-level policy model, so that the policy can achieve adaptive collaborative behavior across task chains under the drive of task chains; integrating the jointly optimized high-level policy model, sub-level policy model, and low-level policy model into a hierarchical reinforcement learning policy model, which serves as the core decision model for unattended control.
[0009] Furthermore, S4 specifically involves: real-time acquisition of equipment status, process parameters, and operating conditions of the industrial system, inputting the real-time data into the task chain representation model to generate the current task chain context representation; based on the task chain context representation, the high-level policy of the hierarchical reinforcement learning policy model selects the current main task to form a main task-level objective; according to the main task-level objective, the sub-layer policy selects the target sub-tasks required for this execution to form sub-task-level objectives; based on the sub-task-level objectives, the low-level policy generates corresponding atomic-level control actions and sends the atomic-level control actions to the execution unit; under unattended conditions, the atomic-level control actions are used to achieve multi-instruction-level collaborative control driven by the task chain.
[0010] Furthermore, S5 specifically involves: collecting the main task completion rate, sub-task execution deviation, atomic action performance indicators, and task priority changes after the task chain is executed, forming a task execution feedback set; mapping the task execution feedback set into a reward update signal, enabling it to be recognized and utilized by the policies of each layer of the hierarchical reinforcement learning policy model; updating the value function of the high-level policy, sub-level policy, and low-level policy respectively based on the reward update signal, so that each layer of policy can obtain task-level adaptive correction; jointly updating the high-level policy, sub-level policy, and low-level policy based on cross-layer consistency constraints, so that the policy has self-evolution capability in multi-task chain scenarios; and writing the updated policy back to the hierarchical reinforcement learning policy model, so that the model can continuously maintain self-adaptation, self-coordination, and performance improvement under long-term unattended conditions.
[0011] On the other hand, this invention also discloses an AI-based unattended intelligent control system for industry, comprising: The task chain construction module is used to preprocess the historical operation records, operation instruction sequences and equipment status sequences of industrial processes, extract multi-level task features, identify the preconditions, triggering conditions and dependency structures between tasks, and generate a task chain representation model that includes main tasks, sub-tasks and constraint tasks. The hierarchical instruction space generation module is used to decompose the main task into a set of subtasks based on the task chain representation model, map the subtasks into a set of atomic-level control actions, and construct a hierarchical control instruction space containing the main task level, subtask level and atomic action level. The hierarchical reinforcement learning training module is used to train high-level policies, sub-level policies and low-level policies in the hierarchical control instruction space, and to jointly optimize the three-layer policies based on cross-layer coupling constraints to form a hierarchical reinforcement learning policy model. The unattended execution module is used to select the current main task by the high-level strategy, select the target sub-task by the sub-level strategy, and generate atomic-level control actions by the low-level strategy and send them to the execution unit during real-time system operation, so as to complete multi-instruction level collaborative control driven by task chain. The strategy self-evolution module is used to collect the main task completion rate, sub-task execution deviation, and atomic action performance indicators after execution. It maps the feedback information into reward update signals and performs value function updates and joint parameter optimization on high-level, sub-level, and low-level strategies based on cross-layer consistency constraints. The updated strategy is then written back to the hierarchical reinforcement learning strategy model, enabling the system to have adaptive and self-evolutionary capabilities under unattended conditions.
[0012] The beneficial effects of this invention are as follows: This invention extracts features from historical operation records, instruction sequences, and state sequences to construct a task chain representation model that includes a main task, sub-tasks, and constraint tasks, thus achieving a unified expression of complex industrial processes. This model not only ensures the sequentiality of control logic and the accuracy of triggering conditions but also provides a unified structural support for subsequent strategy training. This enables the control system to accurately understand the dependencies and connections between tasks in the industrial process, thereby avoiding problems such as task fragmentation and strategy conflicts found in traditional rule-based control.
[0013] This invention decomposes the main task into subtasks and maps the subtasks to atomic-level control actions, forming a three-layer instruction structure: main task level, subtask level, and atomic action level. This hierarchical instruction space allows control strategies to make decisions at different granularities. The higher layer is responsible for process organization, the middle layer for task selection, and the lower layer for equipment action execution, achieving coordinated and unified multi-level actions. This significantly improves the control flexibility and responsiveness of industrial systems in complex operating conditions and variable task scenarios.
[0014] This invention uses a task chain representation model as context input during actual operation. Based on the real-time state, a high-level strategy sequentially selects the main task, a sub-level strategy determines the sub-tasks, and a low-level strategy generates atomic-level control actions, ultimately issuing control commands to the execution unit. This decision-making mechanism enables the system to autonomously complete the entire control chain from task planning to action execution under unattended conditions, significantly reducing manual intervention and improving the system's processing capabilities under continuous operation and unforeseen circumstances. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of an AI-based unattended intelligent control method for industrial operations according to the present invention. Detailed Implementation
[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] Example 1: like Figure 1 As shown, this embodiment provides an AI-based unattended intelligent control method for industrial operations, including the following steps: S1. Steps for constructing a task chain representation model: By preprocessing the historical operation records, operation instruction sequences and equipment status sequences of the target industrial process, multi-level task features are extracted. Based on the dependencies and triggering conditions between tasks, a task chain representation model containing main tasks, sub-tasks and constraint tasks is established so that subsequent control strategies can be parsed and executed with the task chain as a unified target structure. S2. Steps for generating a hierarchical control instruction space: Based on the task chain representation model, each main task is decomposed into a set of executable subtasks, and each subtask is further mapped into a set of atomic-level control actions. A hierarchical control instruction space containing the main task level, subtask level, and atomic action level is constructed so that the subsequent policy learning process can perform optimal decision search under a unified hierarchical structure. S3. Steps for training a hierarchical reinforcement learning policy model: In the hierarchical control instruction space, the completion rate of the main task of the task chain is used as the high-level reward indicator, the completion efficiency of the sub-tasks is used as the mid-level reward indicator, and the stability of atomic-level actions is used as the low-level reward indicator. The high-level policy, sub-level policy and low-level policy are trained respectively, and the hierarchical reinforcement learning policy model is formed by cross-layer coupling constraints, so that the improved policy has the ability to adaptively and collaboratively perform actions across task chains. S4. Steps for unattended control execution based on task chain context: During the real-time operation of the industrial system, the task chain representation model is used as the task context input to map the real-time equipment status, process parameters and operation requirements to the hierarchical reinforcement learning strategy model. The current main task is selected by the high-level strategy, the target sub-task is selected by the sub-level strategy, and the atomic control action is generated by the low-level strategy and sent to the execution unit, thereby completing multi-instruction level collaborative control driven by task chain under unattended conditions. S5. Steps for policy self-evolution update based on task chain feedback output: After completing control execution, collect task chain execution results, changes in priority between tasks, sub-task execution deviations and atomic action performance indicators, write the feedback information back to the hierarchical reinforcement learning policy model, update the value function and re-optimize the policy at each layer, so that the model has the control capabilities of self-evolution, self-adaptation and cross-process self-coordination under long-term unattended conditions.
[0018] S1. Steps for constructing a task chain representation model: By preprocessing the historical operation records, operation command sequences, and equipment state sequences of the target industrial process, multi-level task features are extracted. Based on the dependencies and triggering conditions between tasks, a task chain representation model containing main tasks, sub-tasks, and constraint tasks is established, enabling subsequent control strategies to be parsed and executed using the task chain as a unified target structure. Specifically, this includes the following sub-steps: S110. Steps for collecting and standardizing operational sequence data: Collect historical operational records, operation instruction sequences, and equipment status sequences of industrial processes, and categorize data from different sources according to a unified timestamp. Time alignment is performed, and noise points are filtered out. Furthermore, the data format is normalized to convert all data into a standardized set of sequences.
[0019] in, Indicates time Operation instructions; Indicates time The device status; Indicates time The process response.
[0020] S120, Steps for extracting multi-level task features: Based on the standardized sequence set The sequence is subjected to temporal pattern recognition, statistical pattern mining, and deep network feature encoding. The operation purpose pattern, equipment response pattern, and state transition pattern are extracted at multiple levels to form a task feature set.
[0021] Each task feature This indicates the joint change pattern of the task in the time and state dimensions, providing a foundation for subsequent task structure recognition.
[0022] S130, Steps for identifying task relationships and establishing an association graph structure: Based on the task feature set The system identifies the prerequisites, triggering conditions, and dependencies between tasks, and uses nodes to represent main tasks, sub-tasks, and constraint tasks, and edges to represent triggering relationships, thus constructing an association graph structure.
[0023] in, Represents a set of task nodes; This represents the set of edges that the task depends on.
[0024] This graph structure enables the visualization and structured description of the hierarchical relationships between tasks.
[0025] S140, Steps for constructing a hierarchical task chain framework: The association graph structure... The task nodes in the sequence are topologically sorted according to time order, triggering conditions, and dependencies to obtain the main task chain sequence.
[0026] Where m is the total number of generated main tasks; further, for each main task... By associating the corresponding sets of subtasks and sets of constraint tasks, a hierarchical task chain framework is constructed, which includes the main task level, subtask level, and constraint task level.
[0027] S150, Steps for forming a task chain representation model that can be invoked by policy learning: Encoding the hierarchical task chain framework into a task chain representation model.
[0028] in, This represents the encoding function.
[0029] The task chain representation model serves as a unified task structure input for subsequent hierarchical control instruction space construction, hierarchical reinforcement learning training, and unattended collaborative control execution, ensuring that the entire control process operates under a consistent task chain structure.
[0030] S2. Steps for generating a hierarchical control instruction space: Based on the task chain representation model, each main task is decomposed into a set of executable subtasks, and each subtask is further mapped to a set of atomic-level control actions, constructing a hierarchical control instruction space containing main task level, subtask level, and atomic action level, so that the subsequent policy learning process can perform optimal decision search under a unified hierarchical structure; specifically including the following sub-steps: S210, Steps for identifying the main task set: Based on the task chain representation model
[0031] Parse the main task chain sequence
[0032] From each main task Extract the target state, completion conditions, and key triggering conditions to form the main task set.
[0033] Each main task As a high-level decision-making objective for the control process.
[0034] S220, Steps for parsing the main task and generating a set of subtasks: For the main task set For each main task Based on its completion conditions, state transition path, and execution logic, a set of subtasks is obtained.
[0035] And establish a mapping relationship:
[0036] The decomposition process can be executed based on rule splitting, sequence segmentation, or process template parsing, so that the logic of the subtasks is consistent with the goal of the main task, where k is the number of subtasks decomposed from the i-th main task.
[0037] S230, Steps for mapping subtasks to sets of atomic control actions: For each set of subtasks Based on the actuator capability constraints, control interface protocol, and motion discretization rules of the industrial system, each subtask is... Mapped to a set of atomic-level control actions
[0038] And construct the mapping relationship
[0039] Where n is the number of atomic actions mapped to this subtask; each atomic level controls an action. This corresponds to the underlying control instructions that can be directly sent to the execution unit.
[0040] S240. Steps for constructing a hierarchical control instruction space: Based on the main task set Subtask set
[0041] With atomic control action set
[0042] Constructing a hierarchical control instruction space
[0043] Among them, the main task level corresponds to the high-level decision-making action space; the sub-task level corresponds to the mid-level decision-making action space; and the atomic action level corresponds to the low-level execution action space, so that the control strategy can be optimized and searched in a unified three-layer structure.
[0044] S250, The step of establishing an association between the hierarchical control instruction space and the task chain representation model: The hierarchical control instruction space... With task chain representation model Structured binding is performed to form hierarchical control modeling inputs for subsequent reinforcement learning training.
[0045] in, The hierarchical control modeling function enables the subsequent training phase to perform optimal policy search within a unified task chain structure and instruction space structure.
[0046] S3. Steps for training a hierarchical reinforcement learning policy model: In the hierarchical control instruction space, the completion rate of the main task in the task chain is used as the high-level reward indicator, the efficiency of sub-task completion is used as the mid-level reward indicator, and the stability of atomic-level actions is used as the low-level reward indicator. High-level policies, sub-level policies, and low-level policies are trained separately, and a hierarchical reinforcement learning policy model is formed through cross-layer coupling constraints, enabling the improved policy to possess adaptive collaborative behavior capabilities across task chains. Specifically, this includes the following sub-steps: S310. Steps for constructing a hierarchical reinforcement learning training architecture: Modeling input based on hierarchical control. Construct a hierarchical reinforcement learning training architecture that includes high-level policies, sub-level policies, and low-level policies; Among them, the high-level strategy is based on the main task set. For the high-level action space, denoted as ; Sub-strategy with a set of sub-tasks The intermediate action space is denoted as ; The underlying strategy controls the set of actions at the atomic level. The underlying action space is denoted as .
[0047] This ultimately forms a three-layer strategy structure, which can execute hierarchical decisions in a unified task chain environment, where s is an environment state variable.
[0048] S320. Steps to construct a three-layer reward structure: Based on the task chain structure and hierarchical instruction structure, construct a reinforcement learning reward system that includes three layers of rewards: High-level rewards (primary mission level) ; Mid-level rewards (sub-task level) ; Bottom-level rewards (atomic action level) .
[0049] Further construct the hierarchical reward joint function:
[0050] in, , , To ensure that the reward system presents a consistent top-down hierarchy by assigning weights to rewards across different levels.
[0051] S330, the steps for training high-level, sub-level, and low-level policy models: In the hierarchical training architecture, each model is based on three layers of rewards. , , Optimize three types of strategies: High-level strategies are implemented through objective functions. Conduct training; Sub-layer strategies are implemented through objective functions. Conduct training; The underlying strategy uses the objective function Conduct training.
[0052] Based on the above objective function, optimize using either the policy gradient method or the value iteration method respectively. , , This ensures that each strategy layer achieves the optimal behavior pattern consistent with its task level.
[0053] S340. Steps for jointly optimizing a three-layer strategy based on cross-layer consistency constraints: To ensure that the high-level strategy, sub-layer strategy, and low-level strategy can cooperate in the task chain structure, cross-layer consistency constraints are introduced into the three-layer strategy:
[0054] Using a joint optimization objective function
[0055] in, , , The weight coefficients are used across layers. Joint loss constraints ensure that the three-layer strategy remains globally consistent across the main task chain, sub-task chain, and atomic action layer.
[0056] Ultimately, cross-layer coordination training of the three-layer strategy is completed, enabling the strategy group to generate cross-task chain adaptive collaborative behavior driven by the task chain.
[0057] S350, Steps for integrating three-layer policies to generate a hierarchical reinforcement learning policy model: Combine the jointly optimized... , , By combining elements in a hierarchical structure of high-level → sub-level → low-level, a hierarchical reinforcement learning strategy model is formed.
[0058] The strategy model, as the core decision-making model for unattended industrial control, can execute multi-instruction-level collaborative control operations in subsequent steps based on the task chain context.
[0059] S4. Steps for unattended control execution based on task chain context: During the real-time operation of the industrial system, the task chain representation model is used as the task context input. Real-time equipment status, process parameters, and operational requirements are mapped to a hierarchical reinforcement learning policy model. The higher-level policy selects the current main task, the sub-level policy selects the target sub-task, and the lower-level policy generates atomic control actions and sends them to the execution unit, thereby completing multi-instruction-level collaborative control driven by the task chain under unattended conditions. Specifically, this includes the following sub-steps: S410, Steps for generating task chain context representation: Real-time acquisition of equipment status, operating parameters, and external operational requirements of the industrial system, and the real-time status... Input to the task chain representation model
[0060] Constructing task chain context representations using representation mapping functions:
[0061] in, This indicates the semantic context of the task chain at the current moment, reflecting which stage of the main task chain the system is in and the expected task structure to be triggered.
[0062] S420. Steps for selecting the primary task based on high-level strategy: Represented by task chain context. As input to the high-level strategy, it is substituted into the high-level strategy model. Generate high-level actions .
[0063] in, This indicates the main task that should be performed at the moment.
[0064] Select function through high-level actions
[0065] Determine the primary task-level control objectives . S430, Steps for selecting subtasks based on sub-layer strategies: Select the main task-level control target... With task chain context representation The union is used as input to the sub-level policy, and then processed through the sub-level policy. Calculate sub-layer actions The formula for selecting subtasks is as follows:
[0066] in, This indicates the subtask that should be executed at the moment, and is used to provide fine-grained operational intentions to the underlying layer.
[0067] S440, Steps for generating atomic-level control actions based on underlying strategies: Transforming subtask-level control targets... With task chain context representation As input to the underlying policy, through the underlying policy Generate atomic actions And based on the action selection function
[0068] Obtain atomic-level control commands that can be directly issued to the actuators. This enables the specific execution behavior at the underlying level.
[0069] S450, Steps for executing multi-instruction-level cooperative control under unattended conditions: The atomic-level control instructions... The instruction is sent to the execution unit of the industrial system, enabling the execution unit to complete the specific execution action corresponding to the current subtask according to the instruction; simultaneously, according to the control chain... Maintain strict consistency between high-level task chain objectives, sub-task objectives and low-level execution actions, thereby achieving task chain-driven multi-instruction level collaborative control under unattended conditions.
[0070] S5. Steps for policy self-evolution update based on task chain feedback output: After completing control execution, collect task chain execution results, changes in task priorities, sub-task execution deviations, and atomic action performance indicators. Write the feedback information back to the hierarchical reinforcement learning policy model, update the value function and re-optimize the policy at each layer, enabling the model to possess self-evolution, self-adaptation, and cross-process self-coordination control capabilities under long-term unattended conditions. Specifically, this includes the following sub-steps: S510, Steps for collecting task chain execution feedback information: After completing the unattended control execution based on the three-layer strategy, collect the main task completion rate, sub-task execution deviation, atomic action stability index, and task priority changes to form an execution feedback set:
[0071] in, This indicates feedback on the main task completion rate; This indicates feedback on the execution efficiency of the subtask; This indicates the stability feedback of the atomic execution action; This indicates the amount of change in task priority.
[0072] Feedback set As input for policy self-evolution.
[0073] S520, Steps for mapping task chain feedback to reward update signals: Set the execution feedback set Input reward mapping function
[0074] Receive new level reward signal
[0075] in, This indicates the updated value of high-level rewards; This indicates the updated value of the mid-level reward; This indicates the update value of lower-level rewards.
[0076] The reward update signal is used to recalculate the value function of the three-layer strategy.
[0077] S530, Steps for updating the three-layer value function based on the reward update signal: According to the reward update signal... The value function is updated for each of the three strategies: High-level strategy value function:
[0078] Mid-level strategy value function:
[0079] Underlying strategy value function:
[0080] in, , , The learning rate is the value function of the three layers.
[0081] This step enables each strategy layer to obtain a value assessment consistent with the current execution reality.
[0082] S540. Steps for jointly updating the three-layer policy parameters based on cross-layer consistency constraints: To ensure the consistency of the three-layer policy under long-term unattended operation, cross-layer consistency constraints are introduced into the three-layer policy, and policy parameters are updated according to the joint loss function: Joint loss function:
[0083] Strategy update rules:
[0084]
[0085]
[0086] This joint optimization process ensures that the three-layer policy parameters maintain highly consistent cooperative behavior under the task chain semantics, guaranteeing the continuous self-evolution of the overall control system. , , These are the learning rates (or step sizes) for updating policy parameters at the high, sub-, and low levels, respectively.
[0087] S550. Steps to write the updated policy back to the hierarchical reinforcement learning policy model: Update the policy parameters under the cross-layer consistency constraint. Write back to the three-layer policy model respectively:
[0088] The hierarchical reinforcement learning strategy model is equipped with: adaptive correction capability, self-cooperative behavior optimization capability, and long-term self-evolution capability in long-term unattended scenarios, thereby continuously improving the performance of unattended intelligent control in industrial applications.
[0089] Example 2: This example provides an AI-based unattended intelligent control system for industrial applications, including: The task chain construction module is used to preprocess the historical operation records, operation instruction sequences and equipment status sequences of industrial processes, extract multi-level task features, identify the preconditions, triggering conditions and dependency structures between tasks, and generate a task chain representation model that includes main tasks, sub-tasks and constraint tasks. The hierarchical instruction space generation module is used to decompose the main task into a set of subtasks based on the task chain representation model, map the subtasks into a set of atomic-level control actions, and construct a hierarchical control instruction space containing the main task level, subtask level and atomic action level. The hierarchical reinforcement learning training module is used to train high-level policies, sub-level policies and low-level policies in the hierarchical control instruction space, and to jointly optimize the three-layer policies based on cross-layer coupling constraints to form a hierarchical reinforcement learning policy model. The unattended execution module is used to select the current main task by the high-level strategy, select the target sub-task by the sub-level strategy, and generate atomic-level control actions by the low-level strategy and send them to the execution unit during real-time system operation, so as to complete multi-instruction level collaborative control driven by task chain. The strategy self-evolution module is used to collect the main task completion rate, sub-task execution deviation, and atomic action performance indicators after execution. It maps the feedback information into reward update signals and performs value function updates and joint parameter optimization on high-level, sub-level, and low-level strategies based on cross-layer consistency constraints. The updated strategy is then written back to the hierarchical reinforcement learning strategy model, enabling the system to have adaptive and self-evolutionary capabilities under unattended conditions.
[0090] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.
[0091] In summary, this invention, through feature extraction from historical operation records, instruction sequences, and state sequences, constructs a task chain representation model comprising a main task, sub-tasks, and constraint tasks, thus achieving a unified expression of complex industrial processes. This model not only ensures the sequentiality of control logic and the accuracy of triggering conditions but also provides unified structural support for subsequent strategy training. This enables the control system to accurately understand the dependencies and connections between tasks in the industrial process, thereby avoiding problems such as task fragmentation and strategy conflicts in traditional rule-based control.
[0092] This invention decomposes the main task into subtasks and maps the subtasks to atomic-level control actions, forming a three-layer instruction structure: main task level, subtask level, and atomic action level. This hierarchical instruction space allows control strategies to make decisions at different granularities. The higher layer is responsible for process organization, the middle layer for task selection, and the lower layer for equipment action execution, achieving coordinated and unified multi-level actions. This significantly improves the control flexibility and responsiveness of industrial systems in complex operating conditions and variable task scenarios.
[0093] This invention uses a task chain representation model as context input during actual operation. Based on the real-time state, a high-level strategy sequentially selects the main task, a sub-level strategy determines the sub-tasks, and a low-level strategy generates atomic-level control actions, ultimately issuing control commands to the execution unit. This decision-making mechanism enables the system to autonomously complete the entire control chain from task planning to action execution under unattended conditions, significantly reducing manual intervention and improving the system's processing capabilities under continuous operation and unforeseen circumstances.
[0094] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.
[0095] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0096] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0097] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0098] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0099] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0100] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0102] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An industrial unattended intelligent control method based on an AI model, characterized in that, Includes the following steps: S1. By preprocessing the historical operation records, operation instruction sequences and equipment status sequences of the target industrial process, multi-level task features are extracted, and a task chain representation model containing main tasks, sub-tasks and constraint tasks is established, so that subsequent control strategies can be analyzed and executed with the task chain as a unified target structure. S2. Decompose each main task into a set of executable subtasks, and further map each subtask into a set of atomic-level control actions to construct a hierarchical control instruction space containing main task level, subtask level and atomic action level, so as to perform optimal decision search in the subsequent policy learning process. S3. In the hierarchical control instruction space, the completion rate of the main task of the task chain is used as the high-level reward indicator, the completion efficiency of the sub-task is used as the mid-level reward indicator, and the stability of atomic-level actions is used as the low-level reward indicator. The high-level policy, sub-level policy and low-level policy are trained respectively. The hierarchical reinforcement learning policy model is formed by cross-layer coupling constraints, so that the policy has the ability to adaptively and collaboratively perform cross-task chains. S4. Using the task chain representation model as the task context input, the real-time device status, process parameters and operation requirements are mapped to the hierarchical reinforcement learning policy model. The current main task is selected by the high-level policy, the target sub-task is selected by the sub-level policy, and the atomic control actions are generated by the low-level policy and sent to the execution unit, thereby completing the multi-instruction level collaborative control driven by the task chain under unattended conditions.
2. The industrial unattended intelligent control method based on an AI model according to claim 1, characterized in that, It also includes S5, collecting task chain execution results, task priority changes, subtask execution deviations and atomic action performance indicators, writing feedback information back to the hierarchical reinforcement learning policy model, updating the value function and re-optimizing the policy at each layer, so that the model has the ability to self-evolve, adapt, and self-coordinate across processes under long-term unattended conditions.
3. The industrial unattended intelligent control method based on an AI model according to claim 1, characterized in that, S1 specifically refers to: Collect historical operation records, operation instruction sequences, and equipment status sequences of industrial processes, and perform timestamp alignment, noise filtering, and format unification on the data to form a standardized sequence set that can be used for task feature extraction; Based on a standardized sequence set, multi-level task features including operation purpose, equipment response, and state transition mode are extracted to form a task feature set describing task behavior. Based on the task feature set, identify the prerequisite relationships, triggering conditions and dependency structures between tasks, and establish the relationship graph structure between the main task, sub-tasks and constraint tasks. Based on the association graph structure, the main task, sub-tasks and constraint tasks are linked together in the order of execution to form a task path, and a hierarchical task chain framework is constructed according to the task triggering conditions. The task chain framework is encoded into a task chain representation model that can be called upon by subsequent policy learning, so that all subsequent steps use the task chain representation model as a unified task structure.
4. The industrial unattended intelligent control method based on an AI model according to claim 1, characterized in that, S2 specifically refers to: Based on the task chain representation model, the target state and completion conditions of each main task are identified, and the set of main tasks is determined. For each main task, a set of executable subtasks is decomposed according to its completion conditions, and a correspondence between the subtasks and the main task is established. For each subtask, it is broken down into a set of low-level executable atomic control actions, and a correspondence between atomic actions and subtasks is established. Based on the main task set, sub-task set, and atomic action set, a hierarchical control instruction space is constructed, including the main task level, sub-task level, and atomic action level. By associating the hierarchical control instruction space with the task chain representation model, subsequent policy training can perform optimal decision search within a unified instruction structure.
5. The industrial unattended intelligent control method based on an AI model according to claim 1, characterized in that, S3 specifically refers to: Using the hierarchical control instruction space as the input to the training environment, a hierarchical policy architecture containing high-level policies, sub-level policies, and low-level policies is constructed. Set the main task completion rate as the high-level reward indicator, the sub-task completion efficiency as the mid-level reward indicator, and the atomic action stability as the low-level reward indicator. Under the constraint of reward indicators, high-level policy models, sub-level policy models and low-level policy models are trained respectively, so that each policy layer obtains the optimal behavior pattern that matches its task level. Based on cross-layer coupling constraints, the high-level policy model, sub-layer policy model and low-level policy model are jointly optimized to enable the policy to achieve adaptive collaborative behavior across task chains under the drive of task chains. The jointly optimized high-level strategy model, sub-level strategy model, and low-level strategy model are integrated into a hierarchical reinforcement learning strategy model, which serves as the core decision-making model for unattended control.
6. The industrial unattended intelligent control method based on an AI model according to claim 1, characterized in that, S4 specifically refers to: The system collects equipment status, process parameters, and operating conditions of the industrial system in real time, and inputs the real-time data into the task chain representation model to generate the current task chain context representation. Based on the task chain context representation, the high-level policy of the hierarchical reinforcement learning policy model selects the current main task to form the main task-level objective. Based on the main task-level objective, the sub-strategy selects the target sub-tasks required for this execution, forming the sub-task-level objective; Based on the sub-task level objectives, the underlying strategy generates corresponding atomic-level control actions, which are then sent to the execution units. Under unattended conditions, atomic-level control actions are used to achieve multi-instruction-level collaborative control driven by the task chain.
7. The industrial unattended intelligent control method based on an AI model according to claim 2, characterized in that, S5 specifically refers to: Collect the main task completion rate, sub-task execution deviation, atomic action performance indicators, and task priority changes after the task chain is executed to form a task execution feedback set; The task execution feedback set is mapped to a reward update signal, so that it can be recognized and utilized by the policies of each layer of the hierarchical reinforcement learning policy model. Based on the reward update signal, the value function is updated for the high-level strategy, sub-level strategy and low-level strategy respectively, so that each layer of strategy can be adaptively corrected at the task level.
8. The industrial unattended intelligent control method based on an AI model according to claim 7, characterized in that, S5 also includes: based on cross-layer consistency constraints, jointly updating high-level policies, sub-layer policies and low-level policies, so that the policies have self-evolution capabilities in multi-task chain scenarios; writing the updated policies back to the hierarchical reinforcement learning policy model, so that the model can continuously maintain self-adaptation, self-coordination and performance improvement under long-term unattended conditions.
9. An industrial unattended intelligent control system based on an AI model, comprising the industrial unattended intelligent control method based on an AI model as described in any one of claims 1-8, characterized in that, include: The task chain construction module is used to preprocess the historical operation records, operation instruction sequences and equipment status sequences of industrial processes to generate a task chain representation model that includes the main task, sub-tasks and constraint tasks. The hierarchical instruction space generation module is used to decompose the main task into a set of subtasks based on the task chain representation model, and construct a hierarchical control instruction space containing the main task level, subtask level and atomic action level. The hierarchical reinforcement learning training module is used to train high-level policies, sub-level policies and low-level policies in the hierarchical control instruction space, and to jointly optimize the three-layer policies based on cross-layer coupling constraints to form a hierarchical reinforcement learning policy model. The unattended execution module is used to select the current main task by the high-level strategy, the target sub-task by the sub-layer strategy, and the atomic-level control actions generated by the low-level strategy and sent to the execution unit during the real-time operation of the system. The strategy self-evolution module is used to collect the main task completion rate, sub-task execution deviation, and atomic action performance indicators after execution. It maps the feedback information into reward update signals and performs value function updates and joint parameter optimization on high-level, sub-level, and low-level strategies based on cross-layer consistency constraints.