Multi-target full-process scheduling method, device and equipment for casting workshop and storage medium

Abstract

The invention belongs to the technical field of casting production workshop scheduling, and particularly discloses a casting workshop multi-target full-process scheduling method and device, equipment and a storage medium. According to the invention, the deep reinforcement learning environment is established according to the production data and the workshop scheduling task; performing agent training based on the target neural network framework to obtain a process scheduling agent model of the casting enterprise; wherein the target neural network framework comprises an action network and a plurality of evaluation networks; and carrying out multi-target full-process scheduling on the production tasks of the workshops of the casting enterprises according to the scheduling targets of the workshop scheduling tasks based on the process scheduling agent model. Through the above mode, the feature vector generalization deep reinforcement learning algorithm capable of describing the global state of the casting workshop and the casting task is utilized, that is, agent training is performed in a deep reinforcement learning environment, and multi-target full-process scheduling is performed based on the process scheduling agent model, so that the efficiency and accuracy of scheduling the casting workshop can be effectively improved.

Description

Technical Field

[0001] This application belongs to the field of casting production workshop scheduling technology, and more specifically, relates to a multi-objective full-process scheduling method, device, equipment and storage medium for casting workshops. Background Technology

[0002] Sand casting is one of the main production methods in modern manufacturing. It is characterized by a long production process and the intersection of batch processing and single-piece processing. For products with similar materials and heat treatment processes, batch processing is generally carried out in stages such as smelting and heat treatment, while single-piece processing is carried out in stages such as post-processing and machining. At the same time, most sand casting companies currently have multiple candidate machines to choose from for all processes, which is a typical parallel machine scheduling problem. The complexity of scheduling the entire sand casting process is relatively large, which is a very difficult task because many limiting factors play a role and require reasonable planning and scheduling.

[0003] The foundry scheduling problem is often viewed as a hybrid flow shop scheduling problem. This simplification can describe the sand casting production process to some extent, but it cannot accurately capture the single-batch coupled processing characteristics of sand casting. Furthermore, due to the increasing pursuit of economy and efficiency by enterprises, the traditional single-objective shop scheduling problem, which aims to minimize the maximum completion time, is no longer sufficient. Since the hybrid flow shop scheduling problem has been proven to be NP-hard, its solution currently relies on metaheuristic algorithms. However, traditional metaheuristic algorithms, seeking Pareto solutions through multi-objective optimization, often only approximate the Pareto front, failing to reach it. Moreover, the ever-changing production environment in foundry production leads to a continuously increasing demand for dynamic scheduling, while metaheuristic algorithms often need to be recalculated from scratch in each dynamic scheduling iteration, significantly reducing response time. Therefore, the above methods result in low efficiency and accuracy in scheduling foundries. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application aims to provide a multi-objective full-process scheduling method, apparatus, equipment, and storage medium for foundry workshops. The goal is to solve the problems of low efficiency and accuracy in scheduling foundry workshops due to the fact that existing technologies, such as the inability of heuristic algorithms to obtain Pareto solutions for multi-objective optimization to truly reach the Pareto front, the ever-increasing demand for dynamic scheduling, and the need to recalculate from scratch in each dynamic scheduling process.

[0005] To achieve the above objectives, in a first aspect, this application provides a multi-objective full-process scheduling method for a foundry workshop, comprising: Production data and order data of the foundry are obtained separately, and workshop scheduling tasks are generated based on the order data and historical production data. The workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A deep reinforcement learning environment is established based on the production data and the workshop scheduling tasks. In the deep reinforcement learning environment, an agent is trained based on a target neural network framework to obtain a process scheduling agent model for the foundry enterprise; wherein, the target neural network framework includes an action network and multiple value estimation networks; Based on the process scheduling intelligent agent model, the production tasks of the foundry enterprise's workshop are scheduled in a multi-objective, full-process manner according to the scheduling objectives of the workshop scheduling tasks.

[0006] In one embodiment, the deep reinforcement learning environment includes: an agent state space, an action space, and a reward design; the step of establishing the deep reinforcement learning environment based on the production data and the shop floor scheduling task includes: Establish an agent state space based on the workshop scheduling tasks; Based on the production data and the workshop scheduling tasks, various process scheduling rules and various machine scheduling rules are determined respectively. The various process scheduling rules and the various machine scheduling rules are fully permuted and combined to establish an action space; Dense reward signals and sparse reward signals are determined based on the production data and the workshop scheduling tasks, respectively, and a reward design is established based on the dense reward signals and the sparse reward signals.

[0007] In one embodiment, the step of establishing the agent state space based on the workshop scheduling task includes: The number of processes that have already been scheduled for the workpiece and the total number of processes that need to be scheduled for the workpiece are determined based on the workshop scheduling task. The ratio of the number of processes that have been scheduled for the workpiece to the total number of processes that need to be scheduled for the workpiece is calculated, and the scheduling task status is determined based on the results of each ratio calculation. The batch processing machine status and the non-batch processing machine status are determined based on the workshop scheduling task, and the machine status is determined based on the batch processing machine status and the non-batch processing machine status. Based on the workshop scheduling task, determine the end time of the previous process of the workpiece with the fewest completed processes among all workpieces that have not yet completed all processes, the total number of processes that have entered the batch processing stage but have not yet been batched, and the total number of processes that have been scheduled. The current time is determined based on the end time of the previous process of the workpiece with the fewest completed processes; The proportion of workpieces in the batch processing waiting pool is calculated based on the total number of processes that have entered the batch processing stage but have not yet been batched and the total number of processes that the workpieces currently need to be scheduled. The proportion of allocated processes is calculated based on the total number of processes that have been scheduled and the total number of processes that the workpieces currently need to be scheduled. The risk of delay is determined by the sum of the differences between the actual delivery date and the specified delivery date after prioritizing the scheduling of all remaining processes for each workpiece; A global state is established based on the current time, the proportion of workpieces in the batch processing waiting pool, the proportion of assigned processes, the delay risk, and the current maximum completion time.

[0008] In one embodiment, the step of determining dense reward signals and sparse reward signals based on the production data and the shop floor scheduling task, and establishing a reward design based on the dense reward signals and the sparse reward signals, includes: Based on the production data and the workshop scheduling task, determine the additional idle time for all parallel machines based on the currently scheduled machines; A dense reward signal is determined based on the newly added idle time; The continuous scheduling reward signal is determined based on the completion time of the machine selected in the current scheduling after the process of the target workpiece is assigned, the completion time of the parallel machine of the current scheduling process, and the processing time of the current scheduling workpiece on the parallel machine. The cost penalty signal is determined based on the maximum completion time of the workpiece that has completed all process scheduling before the current workpiece process scheduling is performed, and the maximum completion time of the workpiece that has completed all process scheduling after the current workpiece process scheduling is performed. The delivery penalty signal is determined based on the specified delivery date of the current workpiece and the delivery date after the current workpiece is scheduled to be completed. The utilization penalty signal is determined based on the average utilization of all batch processors that have been allocated to all batch processors before the current workpiece process scheduling is performed, and the average utilization of all batch processors that have been allocated to all batch processors after the current workpiece process scheduling is performed. A sparse reward signal is obtained based on the continuous scheduling reward signal, the cost penalty signal, the delivery date penalty signal, and the utilization rate penalty signal, and a reward design is established based on the dense reward signal and the sparse reward signal.

[0009] In one embodiment, the step of training the agent based on the target neural network framework in the deep reinforcement learning environment to obtain the process scheduling agent model of the foundry enterprise includes: During the sampling phase, the recommended actions are decoded according to the scheduling rules to obtain the current process scheduling rules and the current machine scheduling rules. After scheduling the workload according to the current process scheduling rules and the current machine scheduling rules, the melting window time is determined according to the minimum completion time of all parallel machines in the previous process of melting, and the heat treatment window time is determined according to the minimum completion time of all parallel machines in the previous process of heat treatment. When the melting window time is not within the time range of the current melting window and the heat treatment window time is not within the time range of the current heat treatment window, the workpieces in the current waiting queue are batched according to capacity and material constraints, and the end time of the current melting window is set as the start time of the next melting window, the end time of the current heat treatment window is set as the start time of the next heat treatment window, and the next melting window and the next heat treatment window are opened according to the window size respectively. After updating the environmental state based on the current scheduling results, the target sparse reward backtracking value is calculated based on the backtracking mechanism of global time series, and the target trajectory sample is generated based on the target sparse reward backtracking value, the logarithmic policy probability, and the value estimates of multiple value estimation networks in the target neural network framework. Write the target trajectory sample into the sampling buffer of the current round; If the amount of trajectory samples written to the sampling buffer reaches a preset scale, then in the deep reinforcement learning environment, agent training is performed based on the target neural network framework to obtain the process scheduling agent model of the casting enterprise.

[0010] In one embodiment, the step of training an agent based on a target neural network framework to obtain a process scheduling agent model for the foundry enterprise in the deep reinforcement learning environment if the amount of trajectory samples written to the sampling buffer reaches a preset scale includes: If the amount of trajectory samples written to the sampling buffer is detected to reach a preset scale, then the value targets of multiple value estimation networks are obtained in the deep reinforcement learning environment. During the update phase, the generalized advantage estimation is recursively calculated for each of the value objectives, and the calculated multi-way advantages are standardized with zero mean and unit variance. The processed standardized advantages are linearly fused into mixed advantages according to the preset weight vector; A training dataset is generated based on the aforementioned hybrid advantage, four-way rewards, and current round sample data. The training dataset is split into mini-batches, and multiple complete rounds of training are performed based on multiple mini-batches. Calculate the probability ratio of the new and old policies in each mini-batch dataset, calculate the pruning replacement loss based on the probability ratio of the new and old policies, and limit the policy update range based on the pruning replacement loss; Based on the value function loss, the mean square error of the value estimates and the corresponding returns of the value estimates are fitted separately. The value function is then used to suppress excessive bias. The value losses of each clipped value are weighted and summed to obtain the total value loss value. The total value loss, strategy loss, and entropy regularization term are weighted and summed to obtain the total loss scalar value, and the parameters are updated based on the total loss scalar value.

[0011] Secondly, this application provides a multi-objective full-process scheduling device for a foundry workshop, comprising: The generation module is used to acquire production data and order data of the foundry, and generate workshop scheduling tasks based on the order data and historical production data. The workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A module is established to create a deep reinforcement learning environment based on the production data and the workshop scheduling tasks. The training module is used to train the agent based on the target neural network framework in the deep reinforcement learning environment to obtain the process scheduling agent model of the foundry enterprise; wherein, the target neural network framework includes an action network and multiple value estimation networks; The scheduling module is used to perform multi-objective full-process scheduling of the production tasks of the foundry's workshop based on the process scheduling intelligent agent model and the scheduling objectives of the workshop scheduling tasks.

[0012] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0013] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0014] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0015] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0016] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) After obtaining the production data and order data of the foundry, this application generates workshop scheduling tasks based on the order data and historical production data, and establishes a deep reinforcement learning environment based on the production data and workshop scheduling tasks. It uses feature vectors that can describe the global state of the foundry workshop and the foundry task to generalize the deep reinforcement learning algorithm, that is, to train the agent in the deep reinforcement learning environment. In view of the problems of batch splitting and batching in the foundry process, difficulty in full-process scheduling and slow dynamic scheduling response speed, the application uses a batching mechanism based on rolling time window + multi-value estimation network deep reinforcement learning algorithm to realize the full-process scheduling of production tasks in the foundry workshop with batch processing characteristics, thereby effectively improving the efficiency and accuracy of scheduling the foundry workshop.

[0017] (2) The target neural network framework trained in this application follows the PPO (Proximal Policy Optimization) algorithm architecture, which is divided into a sampling phase and a training phase. Unlike the traditional PPO algorithm, the actor network in the target neural network framework is used for action selection, and the four value estimation networks are used for value function evaluation of dense rewards and three sparse rewards based on three target values, respectively. In addition, this application also uses pruning to replace the target as the policy loss, which limits the policy update magnitude and improves training stability. At the same time, an entropy regularization term is added to encourage the policy to keep exploring. To prevent gradient explosion, gradient norm pruning is applied after each backpropagation before completing the parameter update.

[0018] In summary, this application acquires production data and order data from foundry enterprises, and generates workshop scheduling tasks based on the order data and historical production data. These workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A deep reinforcement learning environment is established based on the production data and the workshop scheduling tasks. Within this deep reinforcement learning environment, an agent is trained using a target neural network framework to obtain a process scheduling agent model for the foundry enterprise. The target neural network framework includes an action network and multiple evaluation networks. Based on the process scheduling agent model, multi-objective full-process scheduling of the foundry enterprise's workshop production tasks is performed according to the scheduling objectives of the workshop scheduling tasks. Through this method, after acquiring the production data and order data of the foundry enterprise, a feature vector generalization deep reinforcement learning algorithm is used to describe the global state of the foundry workshop and casting tasks. This involves agent training within a deep reinforcement learning environment and multi-objective full-process scheduling based on the process scheduling agent model, thereby effectively improving the efficiency and accuracy of foundry workshop scheduling. Attached Figure Description

[0019] Figure 1 This is one of the flowcharts of the multi-objective full-process scheduling method for a foundry workshop provided in the embodiments of this application; Figure 2 This is a schematic diagram of the agent training process provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the scheduling effect provided in an embodiment of this application; Figure 4 This is the second flowchart of the multi-objective full-process scheduling method for the foundry workshop provided in the embodiments of this application; Figure 5 This is a schematic diagram of the module structure of the multi-objective full-process scheduling device for the foundry workshop provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0022] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0023] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0024] Based on this, the embodiments of this application provide a multi-objective full-process scheduling method for a foundry workshop, referring to... Figure 1 , Figure 1 This is one of the flowcharts illustrating the multi-objective full-process scheduling method for a foundry workshop provided in this application. In this embodiment, the multi-objective full-process scheduling method for a foundry workshop includes steps S10 to S40: Step S10: Obtain the production data and order data of the casting enterprise respectively, and generate workshop scheduling tasks based on the order data and historical production data; wherein, the workshop scheduling tasks include batch processing process scheduling tasks and non-batch processing process scheduling tasks, the processes of the batch processing process scheduling tasks include molding and core making, cooling, shot blasting, grinding and machining; the processes of the non-batch processing scheduling tasks include smelting and heat treatment.

[0025] It should be noted that the casting enterprise refers to an enterprise that conducts mass production using sand casting. When implementing the multi-objective full-process scheduling method for the casting workshop in this embodiment, it is necessary to acquire the casting enterprise's production data and order data separately. The production data includes, but is not limited to, casting name, casting material, casting process, available parallel machines for each process, and the time spent on each available parallel machine for each process. This data can be collected through Enterprise Resource Planning (ERP) and Manufacturing Execution System (MES) and stored in a large relational database, SQL Server. Similarly, the casting enterprise's order data can also be collected through the ERP and MES systems.

[0026] It should be understood that after obtaining the order data, the casting data that needs to be produced can be extracted from the order data and associated with historical production data to generate a shop floor scheduling task. The shop floor scheduling task has three objectives: minimizing the maximum completion time, minimizing the delay penalty, and maximizing the batch processor utilization rate. The batch processor utilization rate can be determined based on the average ratio of the batch weight of all batches on the batch processor to the maximum capacity of the batch processor.

[0027] Step S20: Establish a deep reinforcement learning environment based on the production data and the workshop scheduling task.

[0028] Understandably, this embodiment utilizes feature vectors that describe the global state of the foundry workshop and foundry tasks to generalize a deep reinforcement learning algorithm, i.e., agent training is performed in a deep reinforcement learning environment. This deep reinforcement learning environment consists of three parts: agent state space, action space, and reward design. The agent state space is used to describe the feature vectors of the overall deep reinforcement learning environment. The action space represents process scheduling and machine scheduling. The reward design is divided into two parts: a dense reward signal and a sparse reward signal. The dense reward signal is the reward that the agent receives after each action, while the sparse reward signal is the reward given only after the agent completes the scheduling of all processes for a workpiece at the end of the action.

[0029] Step S30: In the deep reinforcement learning environment, the agent is trained based on the target neural network framework to obtain the process scheduling agent model of the foundry enterprise; wherein, the target neural network framework includes an action network and multiple value estimation networks.

[0030] It should be understood that, in this embodiment, after the deep reinforcement learning environment is established, the agent can be trained based on the target neural network framework. The target neural network framework for training follows the PPO algorithm architecture and is divided into a sampling phase and a training phase. Unlike the traditional PPO algorithm, the target neural network framework used in this embodiment can be an actor-4-critic architecture, in which the actor network is used for action selection, and the four value estimation networks (critic networks) are used for dense reward and value function evaluation based on three sparse rewards based on three target values, respectively.

[0031] It should be noted that the reference Figure 2 , Figure 2 The diagram illustrates the agent training process, which is as follows: First, the agent selects a scheduling order X from all orders pending production in the environment based on the action, and schedules its current non-batch processing step O. Then, it determines whether the next step O+1 of the order is a batch processing step. If so, it batches step O+1 according to the rolling time window mechanism. If not, the evaluation network calculates the corresponding reward. After the calculation is completed, the action, the new state of the environment, and the reward are all recorded in the buffer pool. After all the steps of all orders are scheduled, the next round of training is carried out. If not, the scheduling of the next task continues until all scheduling tasks are completed.

[0032] Further, step S30 includes: during the sampling phase, decoding the scheduling rules for the recommended actions to obtain the current process scheduling rules and the current machine scheduling rules; after scheduling the workload according to the current process scheduling rules and the current machine scheduling rules, determining the melting window time based on the minimum completion time of all parallel machines in the previous melting process and determining the heat treatment window time based on the minimum completion time of all parallel machines in the previous heat treatment process; when the melting window time is not within the time range of the current melting window and the heat treatment window time is not within the time range of the current heat treatment window, batching the workpieces in the current waiting queue according to capacity and material constraints, and setting the end time of the current melting window as the start time of the next melting window. The starting time is set as the end time of the current heat treatment window, and the starting time of the next heat treatment window is set as the end time of the current heat treatment window. The next smelting window and the next heat treatment window are opened according to the window size. After updating the environmental state according to the current scheduling result, the target sparse reward backtracking value is calculated based on the backtracking mechanism of global time series. The target trajectory sample is generated according to the target sparse reward backtracking value, the logarithmic policy probability, and the value estimation of multiple value estimation networks in the target neural network framework. The target trajectory sample is written to the sampling buffer of the current round. If the amount of trajectory sample written to the sampling buffer reaches the preset scale, the agent is trained based on the target neural network framework in the deep reinforcement learning environment to obtain the process scheduling agent model of the casting enterprise.

[0033] It should be noted that during agent training, in the sampling phase, the current policy... For each time step, an action recommendation is made based on the current state. Specifically, in terms of scheduling, the recommended actions are first decoded using scheduling rules to obtain the current process scheduling rules and the current machine scheduling rules. Then, a complete process scheduling and resource allocation is executed according to the predetermined sequence of "workpiece selection → machine selection → batch processing judgment → batch selection → batch processing machine selection" to obtain the density reward for the current step. The batch processing judgment determines whether the batching conditions are met after workpiece scheduling based on the recommended actions. The batching conditions are determined based on a rolling time window mechanism. Specifically, the melting window time is determined based on the minimum completion time of all parallel machines in the previous melting process, and the heat treatment window time is determined based on the minimum completion time of all parallel machines in the previous heat treatment process. Then, the melting window time is determined (…). T_window_melting Is it located in the current smelting window? T_melting The time range and heat treatment window time () T_window_heating Is it within the current heat treatment window? T_heatingIf the time range is within the specified range, no batching operation is performed; otherwise, if it is not, the workpieces in the current waiting queue are batched according to capacity and material constraints. The end time of the current melting window is set as the start time of the next melting window, and the end time of the current heat treatment window is set as the start time of the next heat treatment window. The next melting window and the next heat treatment window are then opened according to their respective window sizes. Furthermore, the window value can be a specific time period, such as 03:00-03:30, and the window size is a fixed value; in this embodiment, the size can be one of 5, 10, or 15.

[0034] Understandably, after obtaining the current scheduling result, the environmental state is updated, and the target sparse reward backtracking value is calculated based on the global time-series backtracking mechanism. This target sparse reward backtracking value can be the sparse reward backtracking value of the three objectives. Finally, together with the logarithmic policy probability and the value estimates of multiple value estimation networks in the target neural network framework, a target trajectory sample is generated. This target trajectory sample can be an on-policy trajectory sample, represented as ( ),in, Indicates time t The environmental conditions, Indicates the environmental state The actions taken by the intelligent agent, Indicates the environmental state The policy network (with parameter θ) selects an action. Log-strategy probability, Indicates an immediate reward. Indicates the environmental state at the next moment. This indicates whether the current round has ended. Furthermore, since the PPO algorithm architecture is a policy algorithm, target trajectory samples are not placed in the experience revisit pool, but are directly written to the sampling buffer of the current round. The algorithm also continuously monitors whether the amount of trajectory samples written to the sampling buffer has reached a preset size. If not, writing continues; if so, agent training is performed.

[0035] Further, the step of training an agent based on a target neural network framework to obtain the process scheduling agent model of the casting enterprise in the deep reinforcement learning environment if the trajectory sample writing volume of the sampling buffer reaches a preset scale includes: if the trajectory sample writing volume of the sampling buffer reaches a preset scale, obtaining the value targets of multiple value estimation networks in the deep reinforcement learning environment; in the update phase, performing back-recursive calculation of generalized advantage estimation for each value target, and standardizing the calculated multi-way advantages with zero mean and unit variance; linearly fusing the processed standardized advantages into a mixed advantage according to a preset weight vector; and based on the mixed advantage, four-way rewards, and... The training dataset is generated from the sample data of each round; the training dataset is divided into mini-batches, and multiple complete rounds of training are performed based on multiple mini-batches; the probability ratio of the new and old policies is calculated in each mini-batchset, and the pruning replacement loss is calculated based on the probability ratio of the new and old policies, and the policy update magnitude is limited based on the pruning replacement loss; the mean squared error is fitted to the value estimates of multiple paths and the corresponding returns based on the value estimates based on the value function loss, and the value function pruning is used to suppress excessive bias, and the value losses of each pruned path are weighted and summed to obtain the total value loss value; the total value loss value, the policy loss value, and the entropy regularization term are weighted and summed to obtain the total loss scalar value, and the parameters are updated based on the total loss scalar value.

[0036] It should be understood that, in addition to the sampling stage described above, this embodiment also includes an update stage. First, it is necessary to perform a reverse recursive calculation of the generalized advantage estimation for each value objective. After obtaining the advantage, in order to suppress the training bias caused by scale inconsistency, it is necessary to perform zero-mean unit variance standardization on the calculated multi-way advantages. In order to achieve gradient update of the driving policy, it is also necessary to linearly fuse the four advantages into a mixed advantage according to the preset weight vector. The four rewards are used as the supervision signals of their respective value heads.

[0037] It is important to emphasize that after generating the training dataset, mini-batch processing can be performed on the training dataset. mini- batch Segmentation and multiple complete rounds ( epochThe model repeatedly learns from the previous model and calculates the probability ratio of the new and old policies in each mini-batch dataset. It then calculates a pruning replacement loss based on this ratio, limiting the policy update magnitude and improving training stability. Furthermore, it performs mean squared error fitting on the multi-path value estimates and their corresponding rewards based on the value function loss, suppressing excessive bias by employing value function pruning. The pruned value losses are then weighted and summed. An entropy regularization term is added to encourage continued exploration, and the total value loss and policy loss are combined and weighted. Finally, to prevent gradient explosion, this embodiment applies gradient norm pruning after each backpropagation before parameter updates. The relevant hyperparameters set at this time can be found in Table 1. Table 1:

[0038] Step S40: Based on the process scheduling intelligent agent model, perform multi-objective full-process scheduling of the production tasks of the foundry enterprise's workshop according to the scheduling objectives of the workshop scheduling tasks.

[0039] Understandably, after training the process scheduling agent model, multi-objective full-process scheduling of production tasks in the foundry workshop is performed based on the scheduling objectives of the workshop scheduling tasks. This involves a batching mechanism based on rolling time windows combined with a multi-criteria deep reinforcement learning algorithm to achieve full-process scheduling of production tasks in the foundry workshop, which exhibits batch processing characteristics. This effectively improves the efficiency and accuracy of scheduling the foundry workshop. It should be noted that the reference... Figure 3 , Figure 3 The diagram illustrates the scheduling effect, taking workpieces 0-19 as an example. Specifically, the horizontal axis represents time, and the vertical axis represents the machine number, including the cooling process, melting batch, and hot batch. After multi-objective full-process scheduling according to the technical solution of this embodiment, the completion time is 60.33, the delay penalty is 49.15, and the batch processing machine utilization rate is 0.738.

[0040] This embodiment acquires production data and order data from a foundry, and generates workshop scheduling tasks based on the order data and historical production data. The workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A deep reinforcement learning environment is established based on the production data and the workshop scheduling tasks. In this deep reinforcement learning environment, an agent is trained based on a target neural network framework to obtain a process scheduling agent model for the foundry. The target neural network framework includes an action network and multiple evaluation networks. Based on the process scheduling agent model, multi-objective full-process scheduling of the foundry's workshop production tasks is performed according to the scheduling objectives of the workshop scheduling tasks. Through this method, after acquiring the foundry's production data and order data, a feature vector generalization deep reinforcement learning algorithm is used to describe the global state of the foundry workshop and casting tasks. That is, agent training is performed in a deep reinforcement learning environment, and multi-objective full-process scheduling is performed based on the process scheduling agent model, thereby effectively improving the efficiency and accuracy of scheduling the foundry workshop.

[0041] In one specific implementation, this application provides steps for establishing a deep reinforcement learning environment. Please refer to... Figure 4 , Figure 4 This is the second flowchart illustrating the multi-objective full-process scheduling method for a foundry workshop provided in this application embodiment. The deep reinforcement learning environment includes: agent state space, action space, and reward design; step S20 includes steps S201~S204: Step S201: Establish the intelligent agent state space based on the workshop scheduling task.

[0042] It should be noted that the agent state space is used to describe the feature vector of the overall deep reinforcement learning environment. This feature vector can be composed of three parts: the scheduling task state, the machine state, and the global state. This agent state space can be established based on the workshop scheduling task.

[0043] Further, step S201 includes: determining the number of processes currently scheduled for a workpiece and the total number of processes currently requiring scheduling for the workpiece based on the workshop scheduling task; calculating the ratio between the number of processes currently scheduled for the workpiece and the total number of processes currently requiring scheduling for the workpiece, and determining the scheduling task status based on the results of each ratio calculation; determining the batch processing machine status and the non-batch processing machine status based on the workshop scheduling task, and determining the machine status based on the batch processing machine status and the non-batch processing machine status; determining, based on the workshop scheduling task, the end time of the previous process of the workpiece with the fewest completed processes among all workpieces that have not yet completed all processes, the total number of processes that have entered the batch processing stage but have not yet been batched and completed, and... The current time is determined based on the total number of processes that have been scheduled; the current time is determined based on the end time of the previous process of the workpiece with the fewest completed processes; the proportion of workpieces in the batch processing waiting pool is calculated based on the total number of processes that have entered the batch processing stage but have not yet been completed and the total number of processes that the workpiece currently needs to be scheduled; the proportion of allocated processes is calculated based on the total number of processes that have been scheduled and the total number of processes that the workpiece currently needs to be scheduled; the delay risk is determined based on the sum of the differences between the actual delivery date and the specified delivery date after prioritizing the scheduling of all remaining processes for each workpiece; a global state is established based on the current time, the proportion of workpieces in the batch processing waiting pool, the proportion of allocated processes, the delay risk, and the current maximum completion time.

[0044] It should be understood that, based on the workshop scheduling tasks, after determining the number of processes that have already been scheduled for a workpiece and the total number of processes that need to be scheduled for the workpiece, the ratio of the number of processes that have already been scheduled for the workpiece to the total number of processes that need to be scheduled for the workpiece can be calculated, specifically as follows: .

[0045] in, Indicates workpiece n The proportion of allocated processes, Indicates workpiece n The number of processes that have been scheduled so far. Indicates workpiece n The total number of processes that need to be scheduled.

[0046] It should be noted that in calculating the above workpieces n After determining the allocated process ratio, the current scheduling task status can be represented as follows: Because the casting process includes both batch and non-batch processing steps, and the machines used are in different states, the machine state can be divided into two parts: batch processing machine state and non-batch processing machine state, specifically:

[0047]

[0048] .

[0049] in, Indicates machine status. This indicates a non-batch processing machine state. Indicates the status of the batch processing machine. T end Indicates the estimated completion time.

[0050] It should be understood that, for the global state, it can be established by the current time, the proportion of workpieces in the batch waiting pool, the proportion of assigned processes, the delay risk, and the current maximum completion time. The formulas used to calculate the current time, the proportion of workpieces in the batch waiting pool, and the proportion of assigned processes can be:

[0051]

[0052] .

[0053] in, Indicates the current time. This indicates the end time of the preceding process for the workpiece with the fewest completed processes among all workpieces for which scheduling has not yet been completed. This indicates the proportion of workpieces in the batch processing waiting pool. This indicates the total number of processes that have entered the batch processing stage but have not yet been batched. Indicates workpiece n Total number of processes that need to be scheduled Indicates the proportion of processes that have been allocated. This indicates the total number of processes that have been scheduled so far.

[0054] Step S202: Determine multiple process scheduling rules and multiple machine scheduling rules based on the production data and the workshop scheduling task.

[0055] Step S203: Perform full permutation and combination of the various process scheduling rules and the various machine scheduling rules to establish the action space.

[0056] It should be understood that the motion space represents process scheduling and machine scheduling, and can be obtained by a full permutation and combination of various process scheduling rules and various machine scheduling rules. When there are 6 types of process scheduling rules and 6 types of machine scheduling rules, the motion space has a total of 36 types of jobs. Firstly, the 6 process scheduling rules can be referred to in Table 2: Table 2:

[0057] On the other hand, the six machine scheduling rules can be found in Table 3: Table 3:

[0058] Step S204: Determine dense reward signals and sparse reward signals based on the production data and the workshop scheduling task, respectively, and establish a reward design based on the dense reward signals and the sparse reward signals.

[0059] Understandably, the reward design is divided into two parts: a dense reward signal and a sparse reward signal. The dense reward signal is the reward that the agent receives after each action, while the sparse reward signal is the reward given only after the agent has completed the action and the scheduling of all processes of a workpiece has been completed.

[0060] Further, step S204 includes: determining the new idle time of all parallel machines based on the production data and the shop floor scheduling task; determining a dense reward signal based on the new idle time; determining a continuous scheduling reward signal based on the completion time of the machine selected in the current scheduling after the process of assigning the target workpiece, the completion time of the parallel machine of the current scheduling process, and the processing time of the process of the current scheduling workpiece on the parallel machine; and determining a continuous scheduling reward signal based on the maximum completion time of workpieces that have completed all process scheduling before the current workpiece process scheduling and the maximum completion time of workpieces that have completed all process scheduling after the current workpiece process scheduling. The time-based cost penalty signal is determined; the delivery date penalty signal is determined based on the specified delivery date of the current workpiece and the delivery date after the current workpiece is scheduled; the utilization rate penalty signal is determined based on the average utilization rate of all batches already allocated on all batches before the current workpiece process scheduling and the average utilization rate of all batches already allocated on all batches after the current workpiece process scheduling; a sparse reward signal is obtained based on the continuous scheduling reward signal, the cost penalty signal, the delivery date penalty signal, and the utilization rate penalty signal, and a reward design is established based on the dense reward signal and the sparse reward signal.

[0061] It should be noted that for dense reward signals, after determining the additional idle time of all parallel machines based on the production data and shop floor scheduling tasks, the dense reward signal can be determined based on the additional idle time. For sparse reward signals, the signal can be determined based on the following objective functions:

[0062]

[0063]

[0064] .

[0065] in, This indicates a continuous scheduling reward signal. This indicates that the machine currently selected for scheduling is allocating the currently scheduled artifacts. n process o The completion time after that, The parallel machine representing the current scheduling operation. m The completion time, Indicates the currently scheduled workpiece n process o In parallel machines m On the processing time, This indicates a cost penalty signal. This indicates the maximum completion time of a workpiece that has completed all process scheduling before the current workpiece process scheduling is performed. This indicates the maximum completion time for a workpiece that has completed all process scheduling after the current workpiece process scheduling is performed. This indicates a penalty signal for late delivery. Indicates the specified delivery date for the current workpiece. This indicates the delivery date after the current workpiece scheduling is completed. This indicates a utilization penalty signal. This represents the average utilization rate of all batch processors that have been allocated to all batches before the current workpiece process scheduling is performed. This represents the average utilization rate of all batch processors that have been allocated to all batches after the current workpiece process scheduling is performed.

[0066] It should be noted that for sparse rewards, a backtracking mechanism based on global time sequence can be used to determine them. Specifically, assume that the currently scheduled workpiece is... n If the current execution step step_n The workpiece was not completed after the event. n The scheduling of all processes will then be in step_buffer Record the workpiece n scheduling trajectory step_n However, if the workpiece is completed n If all processes are scheduled, then based on the above formula, a sparse reward value can be calculated once, and then the sparse reward value can be evenly distributed to... step_buffer Recorded workpiece n All scheduling trajectories step_n The corresponding trajectory samples.

[0067] This embodiment establishes an agent state space based on the workshop scheduling task; determines multiple process scheduling rules and multiple machine scheduling rules based on the production data and the workshop scheduling task; performs full permutations and combinations of the multiple process scheduling rules and the multiple machine scheduling rules to establish an action space; determines dense reward signals and sparse reward signals based on the production data and the workshop scheduling task, and establishes a reward design based on the dense reward signals and the sparse reward signals. Through the above method, after establishing the agent state space based on the workshop scheduling task, and establishing the action space by performing full permutations and combinations of multiple process scheduling rules and multiple machine scheduling rules, and then establishing the reward design based on the dense reward signals and the sparse reward signals after determining the dense reward signals and the sparse reward signals, the accuracy of establishing the agent state space, action space, and reward design can be effectively improved, thereby effectively improving the accuracy of establishing a deep reinforcement learning environment.

[0068] The following describes the multi-objective full-process scheduling device for the foundry workshop provided in this application. The multi-objective full-process scheduling device described below corresponds to the multi-objective full-process scheduling method for the foundry workshop described above. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the module structure of the multi-objective full-process scheduling device for a foundry workshop provided in this application embodiment, including: The generation module T10 is used to acquire the production data and order data of the casting enterprise respectively, and generate workshop scheduling tasks based on the order data and historical production data; wherein, the workshop scheduling tasks include batch processing process scheduling tasks and non-batch processing process scheduling tasks, the processes of the batch processing process scheduling tasks include molding and core making, cooling, shot blasting, grinding and machining; the processes of the non-batch processing scheduling tasks include smelting and heat treatment.

[0069] Module T20 is established to create a deep reinforcement learning environment based on the production data and the workshop scheduling tasks.

[0070] The training module T30 is used to train the agent based on the target neural network framework in the deep reinforcement learning environment to obtain the process scheduling agent model of the foundry enterprise; wherein, the target neural network framework includes an action network and multiple value estimation networks.

[0071] The scheduling module T40 is used to perform multi-objective full-process scheduling of the production tasks of the foundry's workshop based on the process scheduling intelligent agent model and the scheduling objectives of the workshop scheduling tasks.

[0072] This embodiment acquires production data and order data from a foundry, and generates workshop scheduling tasks based on the order data and historical production data. The workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A deep reinforcement learning environment is established based on the production data and the workshop scheduling tasks. In this deep reinforcement learning environment, an agent is trained based on a target neural network framework to obtain a process scheduling agent model for the foundry. The target neural network framework includes an action network and multiple evaluation networks. Based on the process scheduling agent model, multi-objective full-process scheduling of the foundry's workshop production tasks is performed according to the scheduling objectives of the workshop scheduling tasks. Through this method, after acquiring the foundry's production data and order data, a feature vector generalization deep reinforcement learning algorithm is used to describe the global state of the foundry workshop and casting tasks. That is, agent training is performed in a deep reinforcement learning environment, and multi-objective full-process scheduling is performed based on the process scheduling agent model, thereby effectively improving the efficiency and accuracy of scheduling the foundry workshop.

[0073] It is understood that the detailed functional implementation of each of the above modules can be found in the description of the aforementioned method embodiments, and will not be repeated here.

[0074] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0075] Based on the methods in the above embodiments, this application provides an electronic device, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0076] It should be noted that the system may include: a processor 10, a communications interface 20, a memory 30, and a communication bus 40. The processor 10, communications interface 20, and memory 30 communicate with each other via the communication bus 40. The processor 10 can invoke logical instructions stored in the memory 30 to execute the methods described in the above embodiments.

[0077] Furthermore, the logical instructions in the aforementioned memory 30 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0078] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0079] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0080] It is understood that the processor in the embodiments of this application can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0081] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory, flash memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, registers, hard disks, portable hard disks, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor.

[0082] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. Those skilled in the art will readily understand that the above descriptions are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multi-objective full-process scheduling method for a foundry workshop, characterized in that, include: Production data and order data of the foundry are obtained separately, and workshop scheduling tasks are generated based on the order data and historical production data. The workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A deep reinforcement learning environment is established based on the production data and the workshop scheduling tasks. In the deep reinforcement learning environment, an agent is trained based on a target neural network framework to obtain a process scheduling agent model for the foundry enterprise; wherein, the target neural network framework includes an action network and multiple value estimation networks; Based on the process scheduling intelligent agent model, the production tasks of the foundry enterprise's workshop are scheduled in a multi-objective, full-process manner according to the scheduling objectives of the workshop scheduling tasks.

2. The method as described in claim 1, characterized in that, The deep reinforcement learning environment includes: an agent state space, an action space, and a reward design; the step of establishing the deep reinforcement learning environment based on the production data and the shop floor scheduling task includes: Establish an agent state space based on the workshop scheduling tasks; Based on the production data and the workshop scheduling tasks, various process scheduling rules and various machine scheduling rules are determined respectively. The various process scheduling rules and the various machine scheduling rules are fully permuted and combined to establish an action space; Dense reward signals and sparse reward signals are determined based on the production data and the workshop scheduling tasks, respectively, and a reward design is established based on the dense reward signals and the sparse reward signals.

3. The method as described in claim 2, characterized in that, The step of establishing the agent state space based on the workshop scheduling task includes: The number of processes that have already been scheduled for the workpiece and the total number of processes that need to be scheduled for the workpiece are determined based on the workshop scheduling task. The ratio of the number of processes that have been scheduled for the workpiece to the total number of processes that need to be scheduled for the workpiece is calculated, and the scheduling task status is determined based on the results of each ratio calculation. The batch processing machine status and the non-batch processing machine status are determined based on the workshop scheduling task, and the machine status is determined based on the batch processing machine status and the non-batch processing machine status. Based on the workshop scheduling task, determine the end time of the previous process of the workpiece with the fewest completed processes among all workpieces that have not yet completed all processes, the total number of processes that have entered the batch processing stage but have not yet been batched, and the total number of processes that have been scheduled. The current time is determined based on the end time of the previous process of the workpiece with the fewest completed processes; The proportion of workpieces in the batch processing waiting pool is calculated based on the total number of processes that have entered the batch processing stage but have not yet been batched and the total number of processes that the workpieces currently need to be scheduled. The proportion of allocated processes is calculated based on the total number of processes that have been scheduled and the total number of processes that the workpieces currently need to be scheduled. The risk of delay is determined by the sum of the differences between the actual delivery date and the specified delivery date after prioritizing the scheduling of all remaining processes for each workpiece; A global state is established based on the current time, the proportion of workpieces in the batch processing waiting pool, the proportion of assigned processes, the delay risk, and the current maximum completion time.

4. The method as described in claim 2, characterized in that, The step of determining dense reward signals and sparse reward signals based on the production data and the workshop scheduling tasks, and establishing a reward design based on the dense reward signals and the sparse reward signals, includes: Based on the production data and the workshop scheduling task, determine the additional idle time for all parallel machines based on the currently scheduled machines; A dense reward signal is determined based on the newly added idle time; The continuous scheduling reward signal is determined based on the completion time of the machine selected in the current scheduling after the process of the target workpiece is assigned, the completion time of the parallel machine of the current scheduling process, and the processing time of the current scheduling workpiece on the parallel machine. The cost penalty signal is determined based on the maximum completion time of the workpiece that has completed all process scheduling before the current workpiece process scheduling is performed, and the maximum completion time of the workpiece that has completed all process scheduling after the current workpiece process scheduling is performed. The delivery penalty signal is determined based on the specified delivery date of the current workpiece and the delivery date after the current workpiece is scheduled to be completed. The utilization penalty signal is determined based on the average utilization of all batch processors that have been allocated to all batch processors before the current workpiece process scheduling is performed, and the average utilization of all batch processors that have been allocated to all batch processors after the current workpiece process scheduling is performed. A sparse reward signal is obtained based on the continuous scheduling reward signal, the cost penalty signal, the delivery date penalty signal, and the utilization rate penalty signal, and a reward design is established based on the dense reward signal and the sparse reward signal.

5. The method as described in claim 1, characterized in that, The step of training the agent based on the target neural network framework in the deep reinforcement learning environment to obtain the process scheduling agent model of the foundry enterprise includes: During the sampling phase, the recommended actions are decoded according to the scheduling rules to obtain the current process scheduling rules and the current machine scheduling rules. After scheduling the workload according to the current process scheduling rules and the current machine scheduling rules, the melting window time is determined according to the minimum completion time of all parallel machines in the previous process of melting, and the heat treatment window time is determined according to the minimum completion time of all parallel machines in the previous process of heat treatment. When the melting window time is not within the time range of the current melting window and the heat treatment window time is not within the time range of the current heat treatment window, the workpieces in the current waiting queue are batched according to capacity and material constraints, and the end time of the current melting window is set as the start time of the next melting window, the end time of the current heat treatment window is set as the start time of the next heat treatment window, and the next melting window and the next heat treatment window are opened according to the window size respectively. After updating the environmental state based on the current scheduling results, the target sparse reward backtracking value is calculated based on the backtracking mechanism of global time series, and the target trajectory sample is generated based on the target sparse reward backtracking value, the logarithmic policy probability, and the value estimates of multiple value estimation networks in the target neural network framework. Write the target trajectory sample into the sampling buffer of the current round; If the amount of trajectory samples written to the sampling buffer reaches a preset scale, then in the deep reinforcement learning environment, agent training is performed based on the target neural network framework to obtain the process scheduling agent model of the casting enterprise.

6. The method as described in claim 5, characterized in that, The step of training an agent based on a target neural network framework to obtain a process scheduling agent model for the foundry enterprise in the deep reinforcement learning environment if the amount of trajectory samples written to the sampling buffer reaches a preset scale includes: If the amount of trajectory samples written to the sampling buffer is detected to reach a preset scale, then the value targets of multiple value estimation networks are obtained in the deep reinforcement learning environment. During the update phase, the generalized advantage estimation is recursively calculated for each of the value objectives, and the calculated multi-way advantages are standardized with zero mean and unit variance. The processed standardized advantages are linearly fused into mixed advantages according to the preset weight vector; A training dataset is generated based on the aforementioned hybrid advantage, four-way rewards, and current round sample data. The training dataset is split into mini-batches, and multiple complete rounds of training are performed based on multiple mini-batches. Calculate the probability ratio of the new and old policies in each mini-batch dataset, calculate the pruning replacement loss based on the probability ratio of the new and old policies, and limit the policy update range based on the pruning replacement loss; Based on the value function loss, the mean square error of the value estimates and the corresponding returns of the value estimates are fitted separately. The value function is then used to suppress excessive bias. The value losses of each clipped value are weighted and summed to obtain the total value loss value. The total value loss, strategy loss, and entropy regularization term are weighted and summed to obtain the total loss scalar value, and the parameters are updated based on the total loss scalar value.

7. A multi-objective full-process scheduling device for a foundry workshop, characterized in that, include: The generation module is used to acquire production data and order data of the foundry, and generate workshop scheduling tasks based on the order data and historical production data. The workshop scheduling tasks include batch processing scheduling tasks and non-batch processing scheduling tasks. The batch processing scheduling tasks include molding and core making, cooling, shot blasting, grinding, and machining. The non-batch processing scheduling tasks include smelting and heat treatment. A module is established to create a deep reinforcement learning environment based on the production data and the workshop scheduling tasks. The training module is used to train the agent based on the target neural network framework in the deep reinforcement learning environment to obtain the process scheduling agent model of the foundry enterprise; wherein, the target neural network framework includes an action network and multiple value estimation networks; The scheduling module is used to perform multi-objective full-process scheduling of the production tasks of the foundry's workshop based on the process scheduling intelligent agent model and the scheduling objectives of the workshop scheduling tasks.

8. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, it causes the processor to perform the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a processor, the processor causes the processor to perform the method as described in any one of claims 1-6.

Images