Real-time scheduling scheme dynamic correction method and system based on reinforcement learning and storage medium

Abstract

The invention relates to the technical field of production scheduling planning, in particular to a real-time scheduling scheme dynamic correction method and system based on reinforcement learning and a storage medium, and the method simulates an actual production system by establishing a production system space containing a plurality of production element objects, and the production system space comprises a plurality of production element objects. Each production element object comprises a plurality of different production attributes, a multi-objective optimization algorithm is adopted to obtain a state-time sequence of each production attribute of each production element object, an initial production plan is constructed, and then a real-time value of each production attribute of each production element object is obtained in real time; when the real-time value does not accord with the predicted value of the corresponding time node, the initial production plan is adjusted according to the real-time values of all production attributes of all production element objects, so that disturbance existing in the production process is digitized, the disturbance can be responded through a reinforcement learning algorithm, and the production efficiency is improved. Therefore, real-time adjustment of the production plan is realized.

Description

Technical Field

[0001] This invention relates to the field of production scheduling and planning technology, specifically to a method, system, and storage medium for dynamic correction of real-time scheduling schemes based on reinforcement learning. Background Technology

[0002] With the rapid development of intelligent manufacturing, production scheduling systems face the challenge of real-time decision-making in dynamic environments. Traditional scheduling methods often fail to meet the requirements of real-time performance and adaptability when dealing with complex disturbances such as random workpiece arrivals and equipment failures, due to their reliance on static rules or offline optimization models.

[0003] Traditional production scheduling methods are mainly divided into two categories: exact algorithms and heuristic algorithms. Although exact algorithms can guarantee the optimal solution, their computational cost increases exponentially with the complexity and scale of the problem, making it difficult to meet the real-time requirements of large-scale dynamic scenarios. Heuristic algorithms, while having a faster solution speed, rely on fixed rules designed based on human experience, resulting in insufficient generalization ability in complex dynamic environments.

[0004] In recent years, reinforcement learning technology has provided a new approach to dynamic scheduling problems through the interactive learning mechanism between agents and the environment. Existing research attempts to introduce machine learning techniques into the scheduling field, but most solutions only train static scheduling strategies through supervised learning, lacking adaptability to dynamic environmental changes. While some reinforcement learning-based methods can achieve dynamic decision-making, their state space design is limited to single-dimensional features, failing to effectively integrate multi-source heterogeneous information such as equipment load, job attributes, and task priorities. Furthermore, existing reward functions often employ a single-objective-oriented design, making it difficult for agents to balance constraints such as efficiency, cost, and stability in multi-objective optimization scenarios.

[0005] For example, the prior art CN 119250490 B discloses a garment hanging production scheduling method that considers transportation distance. It introduces a worker proficiency coefficient matrix and a workstation distance matrix to establish a mathematical model for garment hanging production scheduling, thereby realizing garment hanging production scheduling. In essence, it is still a prediction-based mathematical model for garment hanging production scheduling, and it does not provide solutions for how to respond to unexpected situations that occur during the production process (such as hanging equipment failure, worker errors, etc.).

[0006] For example, the prior art CN 120013216 B discloses a production scheduling method that considers multi-cavity imbalance and gas collection failure, which is used to schedule the production of injection molding equipment. First, it sets the multi-cavity imbalance probability and machine failure rate for each machine, and sets the corresponding upper and lower limits of processing thresholds. Then, it calculates the processing time based on the multi-cavity imbalance probability, machine failure rate, upper and lower limits of processing thresholds. Then, it sets a dual objective function based on the processing time, which is the maximum completion time of all workpieces and the combined production energy consumption of all machines. Finally, it uses an optimization algorithm to solve the problem and obtain a real-time scheduling scheme.

[0007] While the existing technology takes into account the impact of equipment failure rate on production scheduling, it covers a limited range of disturbance types. In actual production, especially in the machining industry, there may be other disturbances such as order changes and design changes. The existing technology does not provide solutions for how to deal with these different types of disturbances. Summary of the Invention

[0008] To address the technical problem that existing technologies cannot handle various complex disturbances in the production process, this application provides a method, system, and storage medium for dynamic correction of real-time scheduling schemes based on reinforcement learning. The method includes the following steps: Establish a production system space, which includes multiple production factor objects, each of which contains multiple different production attributes; With the optimization objectives of minimizing total processing time, maximizing equipment utilization and minimizing energy consumption, a multi-objective optimization algorithm is used to obtain the state-time series of each production attribute of each production factor object based on the initial value of each production attribute of each production factor object, and to construct an initial production plan. The state-time series contains the predicted values ​​of each production attribute of each production factor object at multiple time points; The system acquires the real-time value of each production attribute of each production factor object. When the real-time value does not match the predicted value at the corresponding time node, the initial production plan is adjusted based on the real-time values ​​of all production attributes of all production factor objects.

[0009] The present invention also provides a real-time scheduling scheme dynamic correction system based on reinforcement learning, comprising: a memory, a processor, and a real-time scheduling scheme dynamic correction program based on reinforcement learning stored in the memory and executable on the processor. When the program is executed by the processor, it implements the steps of the above-described real-time scheduling scheme dynamic correction method based on reinforcement learning.

[0010] The present invention also provides a computer-readable storage medium storing a dynamic correction program for a real-time scheduling scheme based on reinforcement learning, which, when executed by a processor, implements the steps of the above-described dynamic correction method for a real-time scheduling scheme based on reinforcement learning.

[0011] The technical effects and advantages of this invention are as follows: The method provided in this application can effectively handle complex and ever-changing production environments. By establishing a production system space containing multiple production factor objects to simulate the actual production system, a mapping from reality to virtuality is achieved. This allows for the simulation of the production process based on the production system space. Simultaneously, each production factor object has multiple production attributes. Changes in these attributes reflect changes in the state of the production factor, thereby digitizing disturbances in the production process. This enables the disturbances to be responded to through reinforcement learning algorithms, thereby achieving real-time adjustments to the production plan and real-time scheduling of different production factors to cope with disturbances that occur during the production process. Attached Figure Description

[0012] Figure 1 This is a schematic flowchart of the method of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] refer to Figure 1 This invention provides a method for dynamically correcting a real-time scheduling scheme based on reinforcement learning, comprising the following steps: S1. Establish a production system space, which includes multiple production factor objects, each of which contains multiple different production attributes. S2. With the optimization objectives of minimizing total processing time, maximizing equipment utilization and minimizing energy consumption, based on the initial values ​​of each production attribute of each production factor, a multi-objective optimization algorithm is used to obtain the state-time series of each production attribute of each production factor object, and an initial production plan is constructed. The state-time series includes the predicted values ​​of each production attribute of each production factor object at multiple time nodes. S3. Obtain the real-time value of each production attribute of each production factor object. When the real-time value does not match the predicted value at the corresponding time node, adjust the initial production plan based on the real-time values ​​of all production attributes of all production factor objects.

[0015] Specifically, for a typical machinery or semiconductor manufacturing enterprise, the production system space can include the following production element objects: production equipment objects, workpiece objects, storage area objects, and handling equipment objects.

[0016] A production system space can contain multiple production equipment objects, such as multiple CNC lathes, machining centers, wire cutting machines, etc. Production equipment objects are not limited to a single type of processing machinery; they can also represent a technological step, such as an assembly process or a quality inspection process. Each production equipment object has multiple production attributes, such as load rate, fault status, yield rate, processing energy consumption, and spatial coordinates.

[0017] Some production equipment can process multiple workpieces simultaneously, such as wire cutting machines, machining centers, and injection molding machines. Therefore, the load rate indicates the number of workpieces it can process concurrently. Fault status reflects whether the production equipment can perform processing tasks, while yield rate indicates the pass rate of the workpieces processed by this equipment. For some automated processing equipment, such as machining centers or lithography machines used in semiconductor chip manufacturing, there is a certain probability that the produced workpieces will not meet quality requirements. Defective products either require rework or are scrapped. When the yield rate of the production equipment drops to a certain level, the equipment can no longer handle production tasks. Processing energy consumption indicates the energy consumption of the production equipment during the production process. It is generally calculated using the rated power of the equipment, but can also be adjusted according to the load rate. Spatial coordinates refer to the position of the production equipment within the production system space, used to calculate the time required for workpiece transfer between different production devices.

[0018] In mass production, multiple workpieces are often processed simultaneously on the production line. Therefore, the production system space should also contain multiple workpiece objects. Each workpiece object mainly includes two production attributes: production process and processing time. The production process refers to the production steps that the workpiece object must undergo throughout the entire production cycle. The processing time is the time required for each production step. The production process includes the steps performed by the production equipment, as well as the handling process and temporary storage, to facilitate the calculation of the entire workpiece's processing time. Throughout the entire production cycle, a workpiece object needs to undergo at least one processing step. Therefore, the value of the production process attribute changes over time. The value of the production process represents the number of the object performing that step. For example, during processing, it corresponds to the number of the production equipment object; during handling, it corresponds to the number of the handling equipment object; and during temporary storage, it corresponds to the number of the storage area object.

[0019] A production system space can contain multiple storage area objects to record the transfer and temporary storage processes of workpieces. During production, after a workpiece completes a certain process, it is often temporarily stored in a storage area, awaiting the next process. Some storage areas may be located near production equipment, while others may be designated areas within the workshop. Therefore, the process of moving workpieces from production equipment to storage areas also takes time. The main production attributes of storage area objects are spatial coordinates and storage capacity. Storage capacity reflects the number of workpieces the storage area can hold, used to plan the storage schedule for each workpiece, thereby optimizing the workpiece handling process. Furthermore, storage area objects can also be configured with two production attributes: workpiece quantity and workpiece number. These record the specific workpiece objects stored within the storage area, enabling refined management of the production process. This allows managers to promptly view the status of workpieces or storage areas and supervise and inspect the production process.

[0020] Material handling equipment is an essential production element in a production system, used to transfer workpieces from one process to the next. A production system space can contain multiple material handling equipment objects, and each object can include operational attributes such as handling capacity, movement speed, real-time coordinates, handling energy consumption, and operating status. Handling capacity clarifies the upper limit of the number of workpieces the equipment can handle at one time, which is crucial for the rational allocation of handling tasks in production scheduling. Real-time coordinates reflect the location of the material handling equipment within the production system space, facilitating the rational planning of its handling path, while movement speed is used to calculate the time required for the handling process. Material handling energy consumption reflects the energy consumption of material handling equipment during the handling process. A production system typically uses various types of material handling equipment to move different workpieces or equipment. The energy consumption of these different equipment varies significantly. For example, heavy industry production workshops are equipped not only with large-tonnage cranes for moving heavy workpieces or equipment (several tons or even hundreds of tons), but also with small-tonnage simple slide rails or transport flatcars for transferring lighter workpieces or equipment (tens to hundreds of kilograms). Therefore, the material handling process requires reasonable planning of the material handling equipment to reduce the overall energy consumption of the production plan. Operating status can be categorized as idle, running, or faulty, controlling whether the material handling equipment is included in the production plan.

[0021] The aforementioned production system space can basically reflect the state of each element in the production system, thereby simulating the entire production process of the workpiece.

[0022] After establishing the production system space, the value of each working attribute of each production factor object needs to be set according to the specific situation of the production unit. The initial value of the working attribute of each production factor object can be estimated according to the start date expected by the production planning department.

[0023] In the subsequent optimization process, it is necessary to optimize the production attributes that can be optimized in each production factor object to obtain the state-time series of each production attribute of each production factor object in the entire production plan.

[0024] Specifically, for a workpiece object to be produced, a workpiece constraint space is established, reflecting the process sequence required to produce the workpiece, as well as the time required for each process and the available equipment. The process sequence of a workpiece is determined by the design, while the equipment specifically performing each process usually has multiple options. For example, a manufacturing company can use multiple CNC lathes to perform turning operations. When a workpiece requires a turning operation, one CNC lathe can be selected to perform this operation. At the same time, each process also has a corresponding processing time. The table below shows the constraint space for a specific workpiece object.

[0025]

[0026] When a production process corresponds to a storage or handling process, the processing time of that production process is set to be adjustable. For example, the handling time needs to be calculated based on the spatial coordinates of the production equipment object, the spatial coordinates of the storage area object, and the moving speed of the handling equipment. The storage time needs to be determined based on the production plan of the production equipment or handling equipment used in the next process. After the production equipment of the next process is ready for processing, or when the handling equipment is idle, the workpiece object is transferred to the next process.

[0027] For production equipment objects, storage area objects, and handling equipment objects, production attributes are used to reflect the tasks they can undertake and energy consumption during the production process. Therefore, production condition constraint space is established based on the production attributes of production equipment objects, storage area objects, and handling equipment objects, which together with the workpiece constraint space constitute the constraint conditions of the optimization process.

[0028] Generally speaking, the optimization goals of production planning mainly include shortening the processing cycle, reducing energy consumption, and improving equipment utilization. Therefore, three optimization goals can be established: minimizing total processing time, maximizing equipment utilization, and minimizing total energy consumption.

[0029] The minimum total processing time is defined as the maximum completion time, i.e. ,in The processing time for process i includes not only the production and processing steps, but also the time spent on transportation and temporary storage.

[0030] Maximizing equipment utilization is defined as the ratio of the total busy time of all equipment to the duration of the initial production plan, i.e. ,in, Let be the processing time of the j-th workpiece in the i-th operation. This represents the total duration of the entire initial production plan. It includes not only the utilization rate of production equipment but also the utilization rate of material handling equipment.

[0031] Minimize total energy consumption as defined by the total energy consumption of all equipment throughout the initial production plan, i.e. , This represents the energy consumption of process i (production equipment or handling equipment).

[0032] It should be noted that when process i corresponds to a handling or production process, the energy consumption is the energy consumption of the corresponding handling or production equipment. When process i only corresponds to temporary storage, the energy consumption is zero.

[0033] Based on the aforementioned workpiece constraint space, production condition constraint space, and optimization objective, a multi-objective constraint optimization algorithm is used to obtain the optimal production plan, namely, the initial state-time series of each workpiece object. The initial state-time series contains the predicted values ​​of the production process of the workpiece object at each time node. Based on the initial state-time series of each workpiece object, the initial state-time series of each production equipment object, storage area object, and handling equipment object can be derived. The initial state-time series of all production element objects constitute the initial production plan. After the production management personnel confirm that there are no errors, the plan is executed accordingly.

[0034] The table below shows a portion of the initial state-time series data for each production factor object: When formulating the initial production plan (optimal production plan), the system is at a relatively "static" starting point. Traditional optimization algorithms can utilize the complete production system space to generate a high-quality, globally optimized plan. Specific optimization algorithms can employ traditional evolutionary-based multi-objective optimization algorithms, such as NSGA-II (Non-Dominated Sorting Genetic Algorithm) or MOEA / D (Decomposition-Based Multi-Objective Evolutionary Algorithm). These algorithms are suitable for handling nonlinear, high-dimensional problems and can generate Pareto optimal solution sets. These algorithms are existing technologies and will not be elaborated upon further here.

[0035] During production according to the initial production plan, each workpiece, production equipment, storage area, and handling equipment is tracked and monitored. Real-time values ​​of each production attribute of each production element are acquired and compared with the predicted values ​​at each time point to determine consistency. Significant deviations indicate an anomaly in a certain stage of production. For example, a production equipment malfunction may cause the equipment's fault type to change to "failure," preventing it from completing its processing task. This will cause a discrepancy between the real-time and predicted values ​​of the workpiece participating in this process at the corresponding time point, affecting subsequent time point updates and the entry of other workpieces into this process. Therefore, adjustments to the original production plan are necessary.

[0036] Furthermore, external disturbances can actively change the value of the production attribute of a production factor object. For example, if an order change causes a workpiece object to be removed from the production process, the workpiece objects participating in the production plan will change, and the production plan will need to be recalculated. Or, if a design change causes a workpiece object to be returned to the previous process for correction, the attribute value of the production process of the workpiece object at this time point needs to be changed, which will lead to the need to adjust the original production plan.

[0037] Traditional optimization algorithms (such as the NSGA-II or MOEA / D algorithms mentioned above) can provide a detailed production plan containing global information, but they are computationally intensive, time-consuming, and slow to respond to the disturbances mentioned above. This makes them unsuitable for scenarios like the automotive or semiconductor manufacturing industries, which have very strict requirements for the cycle time of the entire production line.

[0038] This application adjusts the production plan through the following steps: S4. Establish a real-time state-time series based on the real-time value of each production attribute of each production factor object. Establish an action space based on the production system space, the initial production plan, and the real-time state-time series. The action space records the values ​​that each production attribute of each production factor object can select during the adjustment process. S5. Select at least one objective from minimizing total processing time, maximizing equipment utilization, and minimizing energy consumption as the adjustment rule. The RL real-time decision model searches for the best decision that satisfies the adjustment rule in the action space, namely the real-time state-time series of the workpiece object, and adjusts the real-time state-time series of the other production factor objects to generate a revised production plan.

[0039] Specifically, the action space can take the form of a database that stores the selectable values ​​of each production attribute of each production element object. For example, in the initial production plan, the predicted values ​​of the working status of transport equipment object 1 and transport equipment object 2 at each time node of the production plan are determined. When it is necessary to transport a workpiece object, if both transport equipment object 1 and transport equipment object 2 are in the space state, then both can perform the transport task. That is, the value of the production process of the workpiece object at this time node can be either transport equipment object 1 or transport equipment object 2.

[0040] Specifically, the real-time RL decision-making model is obtained through the following steps: S51. Collect historical production data, including historical status-time series information of different production factor objects under various production scenarios, such as the load rate, fault status, yield rate, processing energy consumption of production equipment objects, and processing steps and processing time of workpiece objects. Extract characteristic patterns under different production scenarios through analysis of this historical data; S52. Map the historical state-time series of multiple production factor objects to the production system space, establish the interaction relationship between production factor objects, and determine the action space; S53. Continuously change the adjustment rules. Set the reward function according to the selected adjustment rules (such as the priority or weight of minimizing total processing time, maximizing equipment utilization and minimizing energy consumption), select a suitable reinforcement learning algorithm, such as the Deep Q-Network (DQN) algorithm, pre-train the model, and continuously adjust the production attributes of each production factor object in the action space. When the decision made by the model satisfies the adjustment rules, a positive reward is given; otherwise, a negative reward is given. The model continuously learns the mapping relationship between state-action-reward, gradually optimizes its own decision-making strategy, and finally obtains the RL real-time decision model.

[0041] Furthermore, to improve model convergence efficiency, this application employs a distributed training framework to train the real-time RL decision-making model, specifically including the following steps: S54. Establish a distributed computing environment by connecting and configuring multiple computing nodes. These nodes possess different computing resources and capabilities, enabling them to collaboratively complete the model training task. Through efficient communication protocols, ensure that the nodes can exchange data and information quickly and accurately.

[0042] S55. Parameter decomposition is performed on the real-time RL decision-making model, distributing the model's parameters across different computing nodes. Each computing node is responsible for updating and calculating a portion of the parameters, thus fully utilizing the computing resources of each node to achieve parallel computing.

[0043] S56. During distributed training, each computing node independently trains the model and updates its parameters based on the received data and allocated parameters. Simultaneously, parameter synchronization between nodes is performed periodically, summarizing and integrating the updated parameters from each node to ensure the consistency of the entire model. Through continuous iterative training, the model's performance on the validation set is monitored. When the model's performance reaches the preset convergence condition, training stops, resulting in the optimized real-time RL decision model obtained from the distributed training framework.

[0044] Furthermore, when users' production managers not only want to achieve the optimization of a single goal, but also to achieve a balance among two or three optimization goals, such as choosing a balanced adjustment scheme between minimizing total processing time and minimizing energy consumption, the above-mentioned single-goal optimization method is difficult to meet actual needs.

[0045] Therefore, the real-time RL decision-making model is trained in the following way: S57. Different weight coefficients are assigned to each optimization objective, thus establishing different strategy groups. Different optimization objectives within each strategy group have different weight coefficients, resulting in different optimization tendencies in the real-time RL decision-making models formed by each strategy group. For example, the first strategy group prioritizes minimizing energy consumption, so the weight coefficient assigned to minimizing energy consumption will be relatively high, while the weight coefficient assigned to minimizing total processing time will be relatively low. Correspondingly, if the second strategy group prioritizes minimizing total processing time, then the weight coefficient assigned to minimizing total processing time will be higher, while the weight coefficient assigned to minimizing energy consumption will be lower. Through this setup, each strategy group will adjust parameters and optimize the model according to its specific optimization tendency during training.

[0046] S58. Establish a reward function for each strategy group, and train the RL real-time decision model using historical state-time series based on the reward function.

[0047] During training, for the first strategy group, the reward function focuses on the positive incentive of reduced energy consumption. A higher reward value is given when the model's decision effectively reduces energy consumption; however, the reward function provides relatively weaker feedback on changes in total processing time. For the second strategy group, the reward function primarily focuses on shortening total processing time. A larger reward is given when the model's decision leads to a reduction in total processing time, while feedback on changes in energy consumption is less sensitive. Using historical state-time series data, various state data and corresponding time sequence information from past production processes are input into the model. Based on the specific reward function of each strategy group, the parameters of the real-time RL decision-making model are continuously adjusted, allowing it to gradually optimize under different optimization tendencies, thus obtaining real-time RL decision-making models with different optimization tendencies. In subsequent production plan adjustments, the real-time RL decision-making model can create multiple revised production plans with different optimization tendencies, providing more basis for production management personnel's decisions and better adapting to the needs of multi-objective balance in actual production.

[0048] Furthermore, to improve the generalization ability of the real-time RL decision-making model and reduce the difficulty of system deployment, this application also introduces a meta-learning mechanism to pre-train the real-time RL decision-making model to adapt to different production scenarios, specifically including the following steps: S6. Establish an independent task for the historical production data of each production scenario, along with the corresponding state transition probability and reward function; S7. For each independent task, the MAML meta-learning algorithm is used to train the RL real-time decision model in multiple rounds. In each round of training, the model first performs adaptive learning on a small number of historical production data samples of the task, quickly adjusting its own parameters to better handle the task. Then, the performance of the adjusted model is evaluated on another set of samples. Based on the evaluation results, the parameters are further fine-tuned. Through multiple rounds of such adaptation and evaluation processes, the parameters of the RL real-time decision model are pre-optimized, thereby improving the adaptability of the RL real-time decision model to different production scenarios.

[0049] As can be seen from the above analysis, the method provided in this application can effectively handle complex and ever-changing production environments. By establishing a production system space containing multiple production factor objects to simulate the actual production system, a mapping from reality to virtuality is achieved. This allows for the simulation of the production process based on the production system space. Furthermore, each production factor object has multiple production attributes, and changes in these attributes reflect changes in the state of the production factor. This digitizes disturbances in the production process, enabling them to respond to disturbances through reinforcement learning algorithms. Consequently, real-time adjustments to the production plan and real-time scheduling of different production factors are achieved to address disturbances that occur during the production process.

[0050] The present invention also provides a real-time scheduling scheme dynamic correction system based on reinforcement learning, comprising: a memory, a processor, and a real-time scheduling scheme dynamic correction program based on reinforcement learning stored in the memory and executable on the processor. When the program is executed by the processor, it implements the steps of the above-described real-time scheduling scheme dynamic correction method based on reinforcement learning.

[0051] The present invention also provides a computer-readable storage medium storing a dynamic correction program for a real-time scheduling scheme based on reinforcement learning, which, when executed by a processor, implements the steps of the above-described dynamic correction method for a real-time scheduling scheme based on reinforcement learning.

[0052] The specific implementation methods of the above-mentioned system and storage medium are all existing technologies and will not be described in detail here.

[0053] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic correction of real-time scheduling schemes based on reinforcement learning, characterized in that, Includes the following steps: Establish a production system space, which includes multiple production factor objects, each of which contains multiple different production attributes; With the optimization objectives of minimizing total processing time, maximizing equipment utilization and minimizing energy consumption, a multi-objective optimization algorithm is used to obtain the state-time series of each production attribute of each production factor object based on the initial value of each production attribute of each production factor object, and to construct an initial production plan. The state-time series contains the predicted values ​​of each production attribute of each production factor object at multiple time points; The system acquires the real-time value of each production attribute of each production factor object. When the real-time value does not match the predicted value at the corresponding time node, the initial production plan is adjusted based on the real-time values ​​of all production attributes of all production factor objects.

2. The method according to claim 1, characterized in that, The production system space includes the following production element objects: production equipment objects, workpiece objects, storage area objects, and material handling equipment objects; The production attributes of the production equipment object include: load rate, fault status, yield rate, processing energy consumption, and spatial coordinates; The production attributes of the workpiece object include: processing steps and processing time; The production attributes of the storage area object include: spatial coordinates and storage capacity; The production attributes of the material handling equipment include: handling capacity, moving speed, real-time coordinates, handling energy consumption, and working status.

3. The method according to claim 1, characterized in that, The initial production plan was adjusted using the following steps: An action space is established based on the production system space and the initial production plan. The action space records the values ​​that each production attribute of each production element object can select during the adjustment process. The RL real-time decision model selects an objective as the adjustment rule from minimizing total processing time, maximizing equipment utilization, and minimizing energy consumption. It then searches for the best decision that satisfies the adjustment rule within the action space and adjusts the state-time series of each production factor object.

4. The method according to claim 3, characterized in that, The real-time RL decision-making model is obtained through the following steps: Obtain the historical state-time series of multiple production factor objects, establish the production system space based on the historical state-time series, and determine the action space; By continuously changing the adjustment rules, setting the reward function according to the adjustment rules, and using reinforcement learning algorithms to train the model, a real-time RL decision model is obtained.

5. The method according to claim 4, characterized in that, The real-time RL decision-making model is trained using the following steps: Build a distributed computing environment by connecting and configuring multiple computing nodes; The parameters of the real-time RL decision model are split and distributed to different computing nodes. Distributed training is performed, and training stops when the model's performance reaches the preset convergence condition, resulting in a real-time RL decision model.

6. The method according to claim 4, characterized in that, The real-time RL decision-making model is trained using the following steps: Establish multiple strategy groups, each strategy group contains different optimization objectives, and each optimization objective corresponds to a different weight coefficient; A reward function is established for each policy group, and the real-time RL decision model is trained using historical state-time series based on the reward function to obtain multiple real-time RL decision models with different optimization tendencies.

7. The method according to claim 4, characterized in that, The RL real-time decision model is pre-trained using the following steps: Acquire historical production data from different production scenarios; For each production scenario, a separate task is created based on historical production data, along with the corresponding state transition probability and reward function; For each independent task, the MAML meta-learning algorithm is used to train the RL real-time decision model in multiple rounds, and the parameters of the RL real-time decision model are adjusted.

8. A real-time scheduling scheme dynamic correction system based on reinforcement learning, characterized in that, include: The system includes a memory, a processor, and a reinforcement learning-based real-time scheduling scheme dynamic correction program stored in the memory and executable on the processor. When executed by the processor, the program implements the steps of the reinforcement learning-based real-time scheduling scheme dynamic correction method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, A readable storage medium stores a dynamic correction program for a real-time scheduling scheme based on reinforcement learning, which, when executed by a processor, implements the steps of the dynamic correction method for a real-time scheduling scheme based on reinforcement learning as described in any one of claims 1-7.

Images