Intelligent production scheduling system architecture integrating deep reinforcement learning and time sequence prediction

Abstract

The invention relates to an intelligent production scheduling system architecture integrated with deep reinforcement learning and time sequence prediction, which realizes production scheduling strategy autonomous learning, production risk pre-judgment and man-machine collaborative optimization through collaborative linkage of six layers of architectures. The architecture is composed of a data acquisition and fusion layer, a micro-service business core layer, a deep learning engine layer, a production scheduling execution layer, a visual decision-making layer and a system guarantee layer, and multi-source production data is gathered on the basis of the data acquisition and fusion layer; a deep learning engine layer is taken as a core, and production scheduling decision and risk prediction are carried out; the production scheduling execution layer is used as a bridge, and the AI decision is converted into a production instruction; man-machine cooperation is realized through a visual decision-making layer; and finally, a feedback data driving model in the deep learning engine layer is continuously optimized to form a self-evolved intelligent closed loop, the micro-service business core layer provides a necessary business data standardization function, and the system guarantee layer ensures stable operation of the AI service. The method is suitable for the technical field of manufacturing industry production scheduling.

Description

Technical Field

[0001] This invention relates to the field of production scheduling technology in the manufacturing industry, and in particular to an intelligent production scheduling system architecture that integrates deep reinforcement learning and time-series prediction. Background Technology

[0002] With the deepening of intelligent manufacturing, manufacturing enterprises are facing the challenge of transforming into a production model characterized by "multiple varieties, small batches, short lead times, and high disruption." Against the backdrop of frequent order fluctuations, diverse resource constraints, and a complex and ever-changing production environment, enterprises are placing higher demands on the intelligence, real-time performance, and flexibility of production scheduling systems. Currently, mainstream production scheduling systems in the manufacturing industry have the following inherent limitations in terms of technological implementation: In terms of core algorithms, most systems are based on classic operations research methods, such as heuristic rules (e.g., greedy algorithms), linear programming, or metaheuristic algorithms (e.g., genetic algorithms). These methods rely on predefined fixed rules and weights to handle multi-objective optimization problems such as delivery time and resource load, and their scheduling logic is static and deterministic.

[0003] In terms of system architecture, mainstream solutions generally adopt a monolithic or traditional layered architecture, tightly coupling functional modules such as order management, resource scheduling, and scheduling engines. This architecture leads to difficult system maintenance, poor scalability, and the need to refactor code when adding new scheduling strategies or connecting to new data sources, resulting in long development cycles, high costs, and difficulty in handling high-frequency real-time data. Furthermore, the overall system architecture is highly complex, with intricate dependencies between modules, making maintenance and upgrades exceptionally difficult. More importantly, these traditional systems generally lack deep learning capabilities, failing to automatically extract features and patterns from massive amounts of production data, and relying solely on manually set rules for decision-making, severely limiting the system's intelligence and adaptability.

[0004] In terms of risk management, existing solutions mainly rely on alarms based on static thresholds or manual inspections based on experience. This approach can only respond passively after a problem occurs and cannot proactively predict or intervene in potential risks such as equipment overload and material shortages, often leading to production interruptions and order delivery delays. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an intelligent scheduling system architecture that integrates deep reinforcement learning and time series prediction. Through the coordinated linkage of a six-layer architecture, it can realize the autonomous learning of scheduling strategies, the early prediction of production risks, and the human-machine collaborative optimization.

[0006] The technical solution adopted in this invention is as follows: The architecture of this invention consists of a data acquisition and fusion layer, a microservice-based business core layer, a deep learning engine layer, a production scheduling and execution layer, a visualization and decision-making layer, and a system assurance layer. The system is based on the data acquisition and fusion layer, aggregating multi-source production data; with the deep learning engine layer as its core, it performs production scheduling decisions and risk prediction; the production scheduling and execution layer acts as a bridge, transforming AI decisions into production instructions; the visualization and decision-making layer enables human-machine collaboration; and finally, the feedback data in the deep learning engine layer drives continuous model optimization, forming a self-evolving intelligent closed loop. The microservice-based business core layer provides necessary business data standardization functions, and the system assurance layer ensures the stable operation of the AI ​​service.

[0007] Furthermore, the deep learning engine layer includes a production scheduling strategy optimization submodule, a production risk prediction submodule, and an AI decision interpretation submodule. The production scheduling strategy optimization submodule performs multi-objective dynamic decision-making based on a DRL deep reinforcement learning model. The production risk prediction submodule performs time-series prediction and linkage based on an LSTM+Attention risk prediction model. The AI ​​decision interpretation submodule analyzes and interprets the logic of the AI ​​production scheduling scheme. Through the high collaboration of the three submodules, the integrated functions of dynamic production scheduling, risk prediction, and decision interpretation are realized.

[0008] Furthermore, the technical implementation of the production scheduling strategy optimization submodule is as follows: a. Algorithm selection and division of labor: A hybrid DRL architecture of "PPO (Proximal Policy Optimization) + DQN (Deep Q-Network)" is adopted, in which PPO (Proximal Policy Optimization) is responsible for continuous action space decision-making and DQN (Deep Q-Network) is responsible for discrete action space decision-making. b. Model Training Design: 1) Training Sample Construction: Historical production scheduling data from the past 3 years were extracted from HDFS. Each sample contains "input features (equipment load, order information, and material inventory at the time) + production scheduling plan (process allocation results at the time) + execution results (delivery achievement rate and equipment load balancing rate)", totaling 100,000+ valid samples; 2) State Space Definition: Covering equipment status, order attributes, material inventory, and process constraints; 3) Reward Function Design: Multi-objective reward function calculation is performed with "delivery achievement rate, equipment load balancing rate, and disturbance response speed" as the core; 4) Training Process: Based on the TensorFlow framework, an "offline pre-training + online fine-tuning" mode is adopted to ensure that the model adapts to the latest production scenarios. c. Model Deployment and Inference: The trained model is deployed as an inference service through TensorFlow Serving (which supports hot model updates without downtime). When real-time data is input into Redis, a production schedule is output and directly transmitted to the core layer of the microservice business.

[0009] Furthermore, in step 3) of the model training design, the specific weights and calculation methods for the reward function design are as follows: Reward value = Delivery date achievement rate × 40% + Equipment load balancing rate × 30% + Disturbance response speed × 30%. in, Delivery date achievement rate: 1 is calculated when the actual delivery time is ≤ the required delivery date, and 0.2 is deducted for each day of delay; Equipment load balancing rate: The smaller the standard deviation of the load rate of all equipment, the higher the score (maximum 1, minimum 0). Disturbance response speed: 1 is calculated when the time to adjust production schedule after order insertion / failure is ≤100ms, and 0.1 is deducted for every additional 50ms.

[0010] Furthermore, the technical implementation of the production risk prediction submodule is as follows: d. Rationale for Algorithm Selection: Production risks (equipment overload, material shortage) have a "temporal correlation" (if the equipment load rate exceeds 80% for 3 consecutive hours, the risk of overload will increase dramatically within 12 hours). LSTM (Long Short-Term Memory) is good at capturing long-term dependencies in temporal data, avoiding the gradient vanishing problem of traditional RNNs. The addition of Attention allows the model to focus on "high-risk features" (when material inventory is below the safety line, this feature is prioritized), improving prediction accuracy. e. Risk Types and Characteristics Input: Focus on three types of risks: equipment overload, material shortage, and process delay; f. Prediction and linkage logic: 1) Risk prediction: The model outputs "risk probability". A risk probability ≥ 70% is judged as "high risk", 50%-70% is "medium risk", and < 50% is "low risk"; 2) Production scheduling linkage: When judged as "high risk", the production scheduling strategy optimization sub-module is automatically triggered to recalculate; for medium risk, only a warning is sent to the visualization decision layer for manual reference.

[0011] Furthermore, the technical implementation of the AI ​​decision interpretation submodule is as follows: g. Explanation Dimensions and Methods: 1) Feature Contribution Analysis: SHAP algorithm is used to calculate the contribution ratio of each feature; 2) Natural Language Conversion: Feature contribution is converted into a plain description; 3) Comparative Explanation: Explaining the reasons for not selecting alternative solutions; h. Output method: The interpretation results and production scheduling plan are synchronously transmitted to the visualization decision layer. Click on any process in the production scheduling Gantt chart to view the corresponding AI decision interpretation.

[0012] Furthermore, the data acquisition and fusion layer collects data via the MQTT / OPC UA protocol, transmits it through Kafka, cleans it, and stores it in layers in Redis (real-time data for inference) and HDFS (historical data for training) to ensure that the data is "complete, accurate, and fast." The microservice-based core business layer is built on Spring Cloud Alibaba to create a microservice underlying architecture, developing system management services, basic data services, low-code services, etc., to achieve elastic deployment and efficient collaboration of business modules. The production scheduling and execution layer converts AI decisions into executable instructions for equipment through standardized APIs, sends them to various data exchange modules, and transmits execution progress and abnormal data back in real time.

[0013] Furthermore, the technical implementation of the visualization decision-making layer is as follows: i. Visualization: Based on Echarts, a production scheduling Gantt chart and a risk warning heatmap are built to intuitively display the production scheduling plan and risk warnings; j. Human-machine collaborative intervention: Supports managers to adjust parameters and modify plans, enabling manual intervention.

[0014] Finally, the system assurance layer uses Kubernetes to achieve containerized deployment and elastic scaling of core AI services; it utilizes the Prometheus+Grafana monitoring platform to track the performance metrics (inference latency, resource consumption) of model services in real time; and it leverages the ELK technology stack for full-link log analysis to ensure long-term stable operation of the system and its ability to quickly recover from failures.

[0015] Compared with existing technologies, the beneficial effects of this invention are: This invention uses a deep learning engine layer as the absolute core and a microservice architecture as the supporting technology to construct a highly cohesive, loosely coupled, and autonomously evolving intelligent system. The entire architecture is divided into six layers, which work together to achieve intelligent scheduling, and has the following advantages: 1. AI-driven dynamic optimization capabilities: Through the DRL hybrid algorithm, the system can autonomously learn from historical experience and output adjustment plans without manual intervention when faced with disturbances such as order insertions and failures. It is estimated that the on-time delivery rate will increase by about 15-20 percentage points, the equipment load balancing rate will increase by about 25 percentage points, and the disturbance response speed will increase by about 10 times.

[0016] 2. Risk control capabilities based on time-series prediction: The risk prediction model based on LSTM+Attention can predict risks approximately 1-2 days in advance. Combined with a coordinated mechanism, it can proactively adjust production schedules, reducing production interruption time by about 30-40 percentage points and order delivery delay rate by about 25 percentage points. This will realize a shift from "passive response" to "proactive prevention."

[0017] 3. Continuous evolutionary capability through closed-loop iteration: Through a closed loop of "data collection → model training → decision output → execution feedback", the system fine-tunes the model daily with new execution data, and the scheduling accuracy gradually improves with usage time. After about 3 months of use, the delivery date prediction deviation decreased from about 8% to about 3%, and the risk prediction accuracy increased from about 75% to about 90%, avoiding the limitation of traditional systems that are "fixed as soon as they go live".

[0018] 4. Explainable human-machine collaboration capabilities: The integration of the AI ​​decision interpretation module with the visual interface allows managers to clearly understand the production scheduling logic, supports manual intervention and rapid AI recalculation, solves the problem of "pure AI decision-making being difficult to implement", and shortens the learning cycle for enterprise employees by about 50%. Attached Figure Description

[0019] Figure 1 This is the overall system architecture diagram of the present invention; Figure 2 This is a hierarchical division of labor diagram of the present invention; Figure 3 This is the architecture diagram of the deep learning engine layer. Detailed Implementation

[0020] To address the aforementioned shortcomings of existing technologies, this invention aims to provide an intelligent scheduling system framework based on "deep learning-driven + microservice decoupling." The core objectives are divided into four points, each corresponding to a solution to the core deficiencies of existing technologies: Objective 1: To construct an intelligent scheduling core capable of learning from historical data and dynamically optimizing in real time under multi-objective constraints by introducing autonomous learning algorithms such as deep reinforcement learning, so as to solve the problems of rigidity and slow response of traditional algorithms; Objective 2: To decouple the system and containerize the deployment solution using a microservice architecture, thereby addressing the issues of insufficient architectural flexibility and scalability, and overcoming the pain points of traditional monolithic architectures such as difficulty in functional iteration, the need for downtime for expansion, and weak high-frequency data processing capabilities. Objective 3: To integrate a deep learning-based time series prediction model to achieve proactive prediction and intelligent early warning of potential risks in the production process, and to establish a control mechanism that links risk and scheduling, transforming passive response into proactive intervention, improving the stability and reliability of production, and solving the problem of passive and lagging risk prediction. Objective 4: To construct a closed-loop optimization process of "data input - model calculation - decision output - feedback iteration" to ensure that the system has the ability to continuously optimize and evolve, and ultimately promote the upgrading of the production scheduling system from "passively executing tasks" to "actively optimizing production".

[0021] like Figures 1 to 3As shown, the architecture of this invention consists of a data acquisition and fusion layer, a microservice-based business core layer, a deep learning engine layer, a production scheduling and execution layer, a visualization and decision-making layer, and a system assurance layer. The system is based on the data acquisition and fusion layer, aggregating multi-source production data; with the deep learning engine layer as its core, it performs production scheduling decisions and risk prediction; the production scheduling and execution layer acts as a bridge, transforming AI decisions into production instructions; the visualization and decision-making layer enables human-machine collaboration; and finally, the feedback data in the deep learning engine layer drives continuous model optimization, forming a self-evolving intelligent closed loop. The microservice-based business core layer provides necessary business data standardization functions, and the system assurance layer ensures the stable operation of the AI ​​service.

[0022] This invention addresses the simultaneous existence of continuous (e.g., load allocation ratio) and discrete (e.g., equipment selection) problems in production scheduling decisions. It innovatively constructs a hybrid deep reinforcement learning framework that collaboratively integrates PPO (Proximal Policy Optimization) and DQN (Deep Q-Network), achieving dynamic real-time optimization under multi-objective constraints. In this hybrid architecture, PPO handles continuous action space decisions, while DQN handles discrete action space decisions, creating a collaborative mechanism. It employs a state space construction method incorporating four dimensions: equipment status, order attributes, material inventory, and process constraints. A multi-objective reward function design method is used, centered on "delivery achievement rate, equipment load balancing rate, and disturbance response speed." This invention abandons traditional static threshold alarms, utilizing LSTM (Long Short-Term Memory) combined with an attention mechanism to deeply mine long-term dependencies in production time-series data. This enables early quantitative prediction of potential risks such as equipment overload and material shortages, and establishes an automatic linkage control loop between "high-risk events" and "production scheduling strategy recalculation." The LSTM+Attention model is used for time-series prediction of production risks, clearly defining its input features and prediction cycle. When the model outputs risk... When the probability reaches a preset threshold, the scheduling strategy optimization module is automatically triggered to recalculate the logic. This invention constructs a closed-loop data process of "real-time data-driven inference, historical data-driven training, and execution feedback-driven iteration," enabling the intelligent scheduling system to continuously learn and evolve. It solves the bottleneck of traditional systems becoming fixed upon deployment. It adopts a dual-track data storage mechanism: real-time data is cached through Redis to ensure low-latency inference of the AI ​​model, and historical data is stored through HDFS to support offline training of the model. The interaction and complementary logic of the two types of data are defined, and the model self-evolution mechanism is used: based on production execution feedback data, a closed-loop iterative process of "data filtering - model fine-tuning - effect verification" is constructed to achieve incremental updates and continuous optimization of model parameters. To solve the "black box" problem of AI decision-making, this invention introduces the SHAP (SHapley Additive exPlanations) algorithm to analyze the contribution of each input feature in the scheduling decision and convert it into natural language descriptions. Combined with a visual interface and a manual intervention interface, a new human-machine collaborative scheduling model that is understandable and interventionable is constructed.

[0023] In this invention, the deep learning engine layer includes a production scheduling strategy optimization submodule, a production risk prediction submodule, and an AI decision interpretation submodule. The production scheduling strategy optimization submodule performs multi-objective dynamic decision-making based on a DRL deep reinforcement learning model. The production risk prediction submodule performs time-series prediction and linkage based on an LSTM+Attention risk prediction model. The AI ​​decision interpretation submodule analyzes and interprets the logic of the AI ​​production scheduling scheme. Through the high collaboration of the three submodules, the integrated functions of dynamic production scheduling, risk prediction, and decision interpretation are realized.

[0024] The technical implementation of the production scheduling strategy optimization submodule is as follows. a. Algorithm selection and division of labor: A hybrid DRL architecture of "PPO (Proximal Policy Optimization) + DQN (Deep Q-Network)" is adopted because the scheduling scenario includes both "continuous decision-making" and "discrete decision-making". PPO is responsible for decisions in the continuous action space, such as "the load allocation ratio of equipment A (50% / 60% / 70%)" and "the processing time of a certain process (2 hours / 2.5 hours)". PPO has a fast convergence speed in the continuous action space and can avoid parameter oscillation. DQN is responsible for the division of labor and cooperation mechanism for decisions in the discrete action space, such as "whether order X is assigned to equipment A or equipment B" and "whether process Y takes precedence over process Z or vice versa". DQN has high decision-making accuracy in the discrete action space and is suitable for handling "two-choice / multiple-choice" problems. b. Model Training Design: 1) Training Sample Construction: Historical production scheduling data from the past 3 years were extracted from HDFS. Each sample contains "input features (equipment load, order information, and material inventory at the time) + production scheduling plan (process allocation results at the time) + execution results (delivery achievement rate and equipment load balancing rate)", totaling 100,000+ valid samples; 2) State Space Definition: Covering equipment status, order attributes, material inventory, and process constraints; 3) Reward Function Design: Multi-objective reward function calculation is performed with "delivery achievement rate, equipment load balancing rate, and disturbance response speed" as the core; 4) Training Process: Based on the TensorFlow framework, an "offline pre-training + online fine-tuning" mode is adopted to ensure that the model adapts to the latest production scenarios. c. Model Deployment and Inference: The trained model is deployed as an inference service through TensorFlow Serving (which supports hot model updates without downtime). When real-time data is input into Redis, a production schedule is output and directly transmitted to the core layer of the microservice business.

[0025] In step 3) of the model training design, the specific weights and calculation methods of the reward function design are as follows: Reward value = Delivery date achievement rate × 40% + Equipment load balancing rate × 30% + Disturbance response speed × 30%. in, Delivery date achievement rate: 1 is calculated when the actual delivery time is ≤ the required delivery date, and 0.2 is deducted for each day of delay; Equipment load balancing rate: The smaller the standard deviation of the load rate of all equipment, the higher the score (maximum 1, minimum 0). Disturbance response speed: 1 is calculated when the time to adjust production schedule after order insertion / failure is ≤100ms, and 0.1 is deducted for every additional 50ms.

[0026] The technical implementation of the production risk prediction submodule is as follows. d. Rationale for Algorithm Selection: Production risks (such as equipment overload and material shortage) have "temporal correlation" (e.g., if the equipment load rate exceeds 80% for 3 consecutive hours, the risk of overload will increase dramatically within 12 hours). LSTM (Long Short-Term Memory) is good at capturing long-term dependencies in temporal data, avoiding the gradient vanishing problem of traditional RNNs. The addition of Attention allows the model to focus on "high-risk features" (e.g., when material inventory is below the safety line by 50%, this feature is prioritized), improving prediction accuracy. e. Risk Types and Characteristics Input: Focus on three types of risks: equipment overload, material shortage, and process delay; f. Prediction and linkage logic: 1) Risk prediction: The model outputs "risk probability". A risk probability ≥ 70% is judged as "high risk", 50%-70% is "medium risk", and < 50% is "low risk"; 2) Production scheduling linkage: When judged as "high risk", the production scheduling strategy optimization sub-module is automatically triggered to recalculate; for medium risk, only a warning is sent to the visualization decision layer for manual reference.

[0027] The technical implementation of the AI ​​decision interpretation submodule is as follows. g. Explanation Dimensions and Methods: 1) Feature Contribution Analysis: SHAP algorithm is used to calculate the contribution ratio of each feature; 2) Natural Language Conversion: Feature contribution is converted into a plain description; 3) Comparative Explanation: Explaining the reasons for not selecting alternative solutions; h. Output method: The interpretation results and production scheduling plan are synchronously transmitted to the visualization decision layer. Click on any process in the production scheduling Gantt chart to view the corresponding AI decision interpretation.

[0028] The data acquisition and fusion layer collects data via MQTT / OPC UA protocol, transmits it through Kafka, cleans it, and stores it in layers in Redis (real-time data for inference) and HDFS (historical data for training), ensuring the data is "complete, accurate, and fast." The microservice-based core business layer is built on a Spring Cloud Alibaba microservice architecture, developing system management services, basic data services, low-code services, etc., to achieve elastic deployment and efficient collaboration of business modules. The production scheduling execution layer converts AI decisions into executable instructions for equipment through standardized APIs, distributes them to various data exchange modules, and transmits execution progress and anomaly data back in real time. The technical implementation of the visualization decision-making layer is as follows. i. Visualization: Based on Echarts, a production scheduling Gantt chart and a risk warning heatmap are built to intuitively display the production scheduling plan and risk warnings; j. Human-machine collaborative intervention: Supports managers to adjust parameters and modify plans, enabling manual intervention.

[0029] The system assurance layer uses Kubernetes to achieve containerized deployment and elastic scaling of core AI services; it utilizes the Prometheus+Grafana monitoring platform to track the performance metrics of model services in real time (such as inference latency and resource consumption); and it leverages the ELK technology stack for end-to-end log analysis to ensure long-term stable operation of the system and its ability to quickly recover from failures.

[0030] Compared to the traditional scheduling system's reliance on static operations research algorithms and a fixed monolithic architecture, this invention achieves three core breakthroughs through deep learning technology, driving a paradigm shift in scheduling systems from "passive execution" to "proactive intelligent optimization": 1. Decision-making evolution from "static rules" to "dynamic learning": Traditional systems rely on preset rules, resulting in slow responses and requiring manual intervention when faced with disturbances such as order insertions or failures. This invention employs a hybrid deep reinforcement learning framework, endowing the system with autonomous learning and dynamic optimization capabilities. This enables it to extract optimal strategies from massive amounts of historical data, achieving millisecond-level responses under complex disturbances, and improving multi-objective optimization effects (such as on-time delivery rate and equipment utilization rate) by over 20%. 2. Risk control from "passive alarm" to "proactive prediction": Existing technologies can only issue threshold alarms after a risk occurs, leading to frequent production interruptions. This invention uses an LSTM time-series prediction model to advance the risk identification window by 24-48 hours and combines it with a "prediction-linkage" mechanism to proactively adjust production scheduling, transforming passive response into proactive intervention. This can reduce production interruption time by more than 30% and significantly improve production continuity. 3. System Capabilities from "One-Time Deployment" to "Continuous Evolution": Traditional systems have fixed performance after deployment and cannot adapt to environmental changes. This invention constructs a closed-loop process of "data-model-decision-feedback," enabling the system to fine-tune and optimize the model daily based on the latest production execution data. This allows scheduling accuracy to continuously improve with accumulated usage, truly possessing the self-evolution capability of "becoming smarter with use." Simultaneously, the integrated AI decision-making explainability technology effectively addresses the pain points of difficulty in implementing AI technology in manufacturing and low trust levels.

[0031] Finally, it should be emphasized that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction, characterized in that: The architecture consists of a data acquisition and fusion layer, a microservice-based business core layer, a deep learning engine layer, a production scheduling and execution layer, a visualization and decision-making layer, and a system assurance layer. The system is based on the data acquisition and fusion layer and aggregates production data from multiple sources. Using the deep learning engine layer as the core, production scheduling decisions and risk predictions are made; Using the production scheduling execution layer as a bridge, AI decisions are transformed into production instructions; human-machine collaboration is achieved through the visualization decision layer; and finally, the feedback data in the deep learning engine layer drives the model to continuously optimize, forming a self-evolving intelligent closed loop. The microservice-based business core layer provides the necessary business data standardization functions, and the system assurance layer ensures the stable operation of the AI ​​service.

2. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 1, characterized in that: The deep learning engine layer includes a production scheduling strategy optimization submodule, a production risk prediction submodule, and an AI decision interpretation submodule. The production scheduling strategy optimization submodule performs multi-objective dynamic decision-making based on a DRL deep reinforcement learning model. The production risk prediction submodule performs time-series prediction and linkage based on an LSTM+Attention risk prediction model. The AI ​​decision interpretation submodule analyzes and interprets the logic of the AI ​​production scheduling scheme. Through the high collaboration of the three submodules, the integrated functions of dynamic production scheduling, risk prediction, and decision interpretation are realized.

3. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 2, characterized in that: The technical implementation of the production scheduling strategy optimization submodule is as follows. a. Algorithm selection and division of labor: A hybrid DRL architecture of "PPO (Proximal Policy Optimization) + DQN (Deep Q-Network)" is adopted, in which PPO (Proximal Policy Optimization) is responsible for continuous action space decision-making and DQN (Deep Q-Network) is responsible for discrete action space decision-making. b. Model Training Design: 1) Training Sample Construction: Historical production scheduling data from the past 3 years were extracted from HDFS. Each sample contains "input features (equipment load, order information, and material inventory at the time) + production scheduling plan (process allocation results at the time) + execution results (delivery achievement rate and equipment load balancing rate)", totaling 100,000+ valid samples; 2) State Space Definition: Covering equipment status, order attributes, material inventory, and process constraints; 3) Reward Function Design: Multi-objective reward function calculation is performed with "delivery achievement rate, equipment load balancing rate, and disturbance response speed" as the core; 4) Training Process: Based on the TensorFlow framework, an "offline pre-training + online fine-tuning" mode is adopted to ensure that the model adapts to the latest production scenarios. c. Model Deployment and Inference: The trained model is deployed as an inference service through TensorFlow Serving (which supports hot model updates without downtime). When real-time data is input into Redis, a production schedule is output and directly transmitted to the core layer of the microservice business.

4. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 3, characterized in that: In step 3) of the model training design, the specific weights and calculation methods of the reward function design are as follows: Reward value = Delivery date achievement rate × 40% + Equipment load balancing rate × 30% + Disturbance response speed × 30%. in, Delivery date achievement rate: 1 is calculated when the actual delivery time is ≤ the required delivery date, and 0.2 is deducted for each day of delay; Equipment load balancing rate: The smaller the standard deviation of the load rate of all equipment, the higher the score (maximum 1, minimum 0). Disturbance response speed: 1 is calculated when the time to adjust production schedule after order insertion / failure is ≤100ms, and 0.1 is deducted for every additional 50ms.

5. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 2, characterized in that: The technical implementation of the production risk prediction submodule is as follows. d. Rationale for Algorithm Selection: Production risks (equipment overload, material shortage) have "temporal correlation" (if the equipment load rate exceeds 80% for 3 consecutive hours, the risk of overload will increase dramatically within 12 hours). LSTM (Long Short-Term Memory) is good at capturing long-term dependencies in temporal data, which can avoid the gradient vanishing problem of traditional RNNs. With the addition of Attention, the model can focus on "high-risk features" (when material inventory is below the safety line, this feature is prioritized), improving prediction accuracy. e. Risk Types and Characteristics Input: Focus on three types of risks: equipment overload, material shortage, and process delay; f. Prediction and linkage logic: 1) Risk prediction: The model outputs "risk probability". A risk probability ≥ 70% is judged as "high risk", 50%-70% is "medium risk", and < 50% is "low risk"; 2) Production scheduling linkage: When judged as "high risk", the production scheduling strategy optimization sub-module is automatically triggered to recalculate; for medium risk, only a warning is sent to the visualization decision layer for manual reference.

6. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 2, characterized in that: The technical implementation of the AI ​​decision interpretation submodule is as follows. g. Explanation Dimensions and Methods: 1) Feature Contribution Analysis: SHAP algorithm is used to calculate the contribution ratio of each feature; 2) Natural Language Conversion: Feature contribution is converted into a plain description; 3) Comparative Explanation: Explaining the reasons for not selecting alternative solutions; h. Output method: The interpretation results and production scheduling plan are synchronously transmitted to the visualization decision-making layer. Click on any process in the production scheduling Gantt chart to view the corresponding AI decision interpretation.

7. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 1, characterized in that: The data acquisition and fusion layer collects data via MQTT / OPC UA protocol, transmits it through Kafka, cleans it, and stores it in layers in Redis (real-time data for inference) and HDFS (historical data for training) to ensure that the data is "complete, accurate, and fast". The microservice business core layer is built on Spring Cloud Alibaba to build a microservice underlying architecture, develop system management services, basic data services, low-code services, etc., and realize the elastic deployment and efficient collaboration of business modules. The production scheduling execution layer uses standardized APIs to transform AI decisions into executable instructions for equipment, which are then sent to various data exchange modules and the execution progress and abnormal data are transmitted back in real time.

8. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 1, characterized in that: The technical implementation of the visualization decision-making layer is as follows. i. Visualization: Based on Echarts, a production scheduling Gantt chart and a risk warning heatmap are built to intuitively display the production scheduling plan and risk warnings; j. Human-machine collaborative intervention: Supports managers to adjust parameters and modify plans, enabling manual intervention.

9. The intelligent scheduling system architecture integrating deep reinforcement learning and temporal prediction according to claim 1, characterized in that: The system security layer uses Kubernetes to achieve containerized deployment and elastic scaling of core AI services; it uses the Prometheus+Grafana monitoring platform to track the performance metrics (inference latency, resource consumption) of model services in real time; and it uses the ELK technology stack to perform full-link log analysis to ensure long-term stable operation of the system and the ability to quickly recover from failures.

Images