A scheduling method for additive manufacturing workshops based on safety reinforcement learning

Abstract

This invention provides a scheduling method for additive manufacturing workshops based on safety reinforcement learning, relating to the field of deep learning technology. The method includes: determining a workshop state vector S based on workshop simulation environment parameters; where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled; using the DQN algorithm, combined with the workshop state vector S and a reward function, determining the priority of tasks to be scheduled, and determining the batch of tasks to be scheduled based on the priority; using an improved A3C algorithm for machine selection, allocating the tasks to be scheduled in different batches to target machines; the improved A3C algorithm is an A3C algorithm incorporating a safety shield mechanism. By comprehensively considering task urgency, processing time, and machine load, the method dynamically optimizes production scheduling, improving scheduling efficiency and resource utilization, and achieving multi-objective balance.

Description

Technical Field

[0001] This invention relates to the field of additive manufacturing workshop scheduling technology, and in particular to an additive manufacturing workshop scheduling method based on safety reinforcement learning. Background Technology

[0002] In the field of intelligent manufacturing and production scheduling, especially in online production scheduling in additive manufacturing workshops, how to effectively allocate production tasks has become an important research topic. Additive manufacturing (AM) constructs products by adding materials layer by layer and is widely used in aerospace, medical device, and mold manufacturing. However, traditional scheduling methods have significant limitations when dealing with the complex dynamic environment of additive manufacturing workshops.

[0003] Currently, task scheduling in additive manufacturing workshops primarily employs heuristic algorithms, such as shortest processing time priority and earliest deadline priority. These methods are simple to implement, consume relatively low computational resources, and are suitable for static or low-complexity production environments. However, as additive manufacturing processes become more complex, these methods cannot effectively handle conflicts between multiple tasks and competition for equipment resources, leading to reduced production efficiency and potentially even equipment overload. On the other hand, reinforcement learning-based scheduling methods are gaining attention. Reinforcement learning learns strategies that maximize cumulative rewards through interaction with the environment. This approach can adaptively adjust scheduling strategies to adapt to dynamically changing production environments. However, pure reinforcement learning methods require extensive exploration in the early stages of training, potentially leading to unsafe scheduling decisions, such as equipment overload or frequent material switching, impacting production safety and efficiency. Furthermore, due to the high dimensionality of the state space in additive manufacturing scheduling problems, model training and convergence require significant computational resources, making them unsuitable for deployment in actual production. Summary of the Invention

[0004] The purpose of this invention is to address the significant limitations of current scheduling methods in dealing with the complex and dynamic environment of additive manufacturing workshops. This invention provides a scheduling method for additive manufacturing workshops based on safety reinforcement learning.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] The first aspect of this application provides an additive manufacturing workshop scheduling method based on security reinforcement learning, including:

[0007] The workshop state vector S is determined based on the workshop simulation environment parameters. Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled.

[0008] The priority of the tasks to be scheduled is determined by using the DQN algorithm, combining the shop state vector S and the reward function, and the batch of tasks to be scheduled is determined based on the priority.

[0009] The improved A3C algorithm is used for machine selection, and the scheduled tasks in different batches are assigned to the target machines; the improved A3C algorithm is the A3C algorithm with added security shield mechanism.

[0010] Optionally, before determining the workshop state vector S based on the workshop simulation environment parameters, the method further includes:

[0011] Obtain workshop simulation environment parameters; the workshop simulation environment parameters include task number, material type, task urgency, area occupied by the task, task due time, available machines for each operation and corresponding running time.

[0012] Optionally, the step of using the DQN algorithm, combining the shop floor state vector S and the reward function to determine the priority of the tasks to be scheduled, and determining the batch of tasks to be scheduled based on the priority, includes:

[0013] The Q-value of each scheduling action is predicted using a deep Q-network algorithm. For each workshop state vector S, the deep Q-network outputs an action A, and the action with the largest Q-value is selected according to the following formula:

[0014] ;

[0015] Where A represents the set of all possible actions, Representing the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, which are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, real-time priority rule with multi-objective trade-offs, and adaptive scheduling rule based on weighted average of remaining time and urgency.

[0016] Optionally, the formula for the reward function is:

[0017] ;

[0018] ;

[0019] Where, Tard represents the total delay time of all tasks, Load represents the machine's workload, changeCost represents the material changeover cost between different tasks, and isSafe indicates whether the current action is safe. Indicates the currently selected action. This represents a set of unsafe actions. It is an adjustment factor used to control the impact of delay time, equipment load, material changeover, and operational safety on the total reward.

[0020] Optionally, the step of using the improved A3C algorithm to select machines and assigning different batches of the tasks to be scheduled to the target machines includes:

[0021] Create a policy network and a value network; the policy network is used to select actions using a policy gradient method, and the value network is used to evaluate the value of the current state.

[0022] The policy network calculates the priority of each available machine and selects the available machine with the highest priority. The value network guides the policy network to select the target machine in descending order of priority.

[0023] Optionally, before using the improved A3C algorithm to perform machine selection and assign different batches of the scheduled tasks to the target machine, the method further includes:

[0024] The A3C algorithm is optimized based on a security shield mechanism to ensure that the machine load does not exceed the maximum allowable load and that the number of material switching times does not exceed the maximum allowable number of switching times.

[0025] A second aspect of this application provides an additive manufacturing workshop scheduling device based on security reinforcement learning, comprising: a determination module, a partitioning module, and an allocation module, wherein...

[0026] The determining module is configured to determine the workshop state vector S based on the workshop simulation environment parameters. Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled.

[0027] The partitioning module is configured to use the DQN algorithm, combined with the workshop state vector S and the reward function, to determine the priority of the tasks to be scheduled, and to determine the batch of the tasks to be scheduled based on the priority.

[0028] The allocation module is configured to use the improved A3C algorithm to select machines and allocate different batches of the scheduled tasks to the target machines; the improved A3C algorithm is an A3C algorithm with added security shield mechanism.

[0029] A third aspect of this application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the additive manufacturing workshop scheduling method based on security reinforcement learning as described in the first aspect.

[0030] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0031] Compared with the prior art, the beneficial effects of the technical solution provided in this application are:

[0032] This invention provides a method, apparatus, and equipment for additive manufacturing workshop scheduling based on safety reinforcement learning. Based on workshop simulation environment sample data, a deep Q-network algorithm divides tasks into different batches. The batching method is selected according to the workshop state vector S and the reward function to dynamically adjust batch information. The A3C algorithm is used for machine selection, assigning higher-priority tasks within each batch to machines. Machine allocation is dynamically adjusted according to the workshop state vector S and the reward function. By comprehensively considering task urgency, processing time, and machine load, production scheduling is dynamically optimized, improving scheduling efficiency and resource utilization, achieving multi-objective balance, reducing the additional time consumption and production costs caused by frequent material changes, and ensuring material continuity between batches. A safety shield mechanism is added to the model to prevent it from entering an unsafe state, ensuring that equipment load operates within a safe threshold, thus improving the safety and stability of the production process. Attached Figure Description

[0033] Figure 1 A flowchart illustrating an additive manufacturing workshop scheduling method based on security reinforcement learning, provided for an embodiment of this application;

[0034] Figure 2 A schematic diagram illustrating the principle of the additive manufacturing workshop scheduling method based on security reinforcement learning provided in this application embodiment;

[0035] Figure 3 A schematic diagram illustrating the application effect of the additive manufacturing workshop scheduling method based on security reinforcement learning provided in this application embodiment;

[0036] Figure 4 This is a schematic diagram of the structure of an additive manufacturing workshop scheduling device based on security reinforcement learning, provided as an embodiment of this application. Detailed Implementation

[0037] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0038] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0039] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0040] The accompanying drawings show some block diagrams and / or flowcharts. It should be understood that some blocks or combinations thereof in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create means for implementing the functions / operations described in these block diagrams and / or flowcharts.

[0041] In some embodiments, please refer to Figure 1 , Figure 1 A flowchart illustrating the additive manufacturing workshop scheduling method based on security reinforcement learning provided in this application embodiment; the additive manufacturing workshop scheduling method based on security reinforcement learning provided in this application embodiment includes:

[0042] S110, Determine the workshop state vector S based on workshop simulation environment parameters. Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled.

[0043] In this embodiment, a simulation environment is first constructed, which can accurately simulate the state changes of the workshop in the real production environment, ensuring that the scheduling strategy can adapt to the actual situation of the workshop.

[0044] In some embodiments, before determining the workshop state vector S based on the workshop simulation environment parameters in S110, the method further includes:

[0045] Obtain workshop simulation environment parameters; workshop simulation environment parameters include task number, material type, task urgency, area occupied by the task, task due time, available machines for each operation and corresponding running time.

[0046] S120 utilizes a deep Q-network algorithm, combining the workshop state vector S and the reward function, to divide the tasks to be scheduled into sub-tasks in different batches.

[0047] The Deep Q-Network algorithm processes the input state using a deep neural network and outputs the Q-value for each action. By training, the Q-function is approximated, enabling the estimation of the Q-value for each state-action pair in a high-dimensional state space. The reward function defines the immediate feedback received by the agent at each time step and is the agent's learning objective. Essentially, it conveys the objective to the agent; all of the agent's "goals" or "purposes" can be reduced to maximizing the expected probability of the cumulative sum of the scalar reward signals received by the agent.

[0048] In some embodiments, S120, the priority of the tasks to be scheduled is determined using the DQN algorithm, combined with the shop floor state vector S and the reward function, and the batch of tasks to be scheduled is determined based on the priority, including:

[0049] The deep Q-network algorithm is used to predict the Q-value of each scheduling action. For each shop state vector S, the deep Q-network outputs an action A, and the action with the largest Q-value is selected according to the following formula:

[0050] ;

[0051] Where A represents the set of all possible actions, Representing the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, which are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, real-time priority rule with multi-objective trade-offs, and adaptive scheduling rule based on weighted average of remaining time and urgency.

[0052] In this embodiment, the Deep Q-Network (DQN) selects the most suitable scheduling rule based on the current environment. Through continuous learning, the Q-value network can effectively measure the long-term returns of different actions. Among these rules are: Urgency-priority batch processing rule: This rule is selected when the workshop receives urgent tasks or priority orders from specific customers, batching tasks according to urgency value; Shortest processing time batch processing rule: This rule is selected when there is a large workload and limited production resources, shortening the overall production cycle; Job load balancing batch processing rule: This rule is selected when the workshop's task load is uneven, with some machines overloaded, balancing workshop resource utilization; Real-time priority rule with multi-objective trade-offs: This rule is selected in complex workshop environments (such as multiple orders with different delivery dates, different material requirements, and conflicting processing conditions), dynamically adjusting priorities by weighing different objectives to balance processing time, material compatibility, urgency, and other needs; Adaptive scheduling rule based on remaining time and urgency weighting: This rule is selected when the workshop faces uneven remaining time and different urgency levels for tasks, and the adaptive scheduling rule determines priorities based on the remaining time and urgency of the tasks.

[0053] It's important to note that DQN also involves an experience replay mechanism and a target network mechanism. The experience replay mechanism stores each scheduling decision in a memory pool and randomly samples a small batch of decision records for training, mitigating the unstable training problem caused by sample correlation. The target network mechanism updates the Q-value through a target network with the same parameters as the main network but not shared with it, thereby stabilizing the training process. The target network parameters are synchronized with the main network at regular training steps.

[0054] In some embodiments, the formula for the reward function is:

[0055] ;

[0056] ;

[0057] Where, Tard represents the total delay time of all tasks, Load represents the machine's workload, changeCost represents the material changeover cost between different tasks, and isSafe indicates whether the current action is safe. Indicates the currently selected action. This represents a set of unsafe actions. It is an adjustment factor used to control the impact of delay time, equipment load, material changeover, and operational safety on the total reward.

[0058] In this embodiment, Tard represents the task's delay time; failure to complete the task on time will result in a negative reward. Load represents the machine's workload; overloading the machine will also incur penalties. changeCost represents the material switching cost between different tasks; the more frequent the material switching, the higher the cost and the greater the negative reward. isSafe indicates whether the current action is safe, i.e., whether the machine will exceed its load. Indicates the currently selected action. This represents a set of unsafe actions. This is an adjustment factor used to control the impact of delay time, equipment load, material changeover, and operational safety on the total reward. The reward function quantifies the benefit of each action, and the reinforcement learning model gradually learns to avoid high-penalty actions during training.

[0059] In this embodiment, the optimization process of the reinforcement learning strategy is completed through continuous iteration and multiple rounds of training. After each training cycle, the model updates its strategy to gradually approach the optimal scheduling strategy. Specifically, this includes: 1) Model update: Through training feedback from DQN and A3C, the scheduling strategy is continuously optimized to make the model more adaptable to scheduling different shop states. 2) Loss function: The loss function measures the accuracy of the model output, including the loss of the original Q-value and the policy value. The loss function is calculated as follows: For the DQN model, the loss function calculates the difference in Q-values ​​and updates the network weights. For the A3C model, the loss function calculates the value difference of the critic network and the action advantage of the policy network, and updates the policy parameters. 3) Model parameter saving and replay: After each iteration, the model saves the current parameters for use in practical applications. In addition, new data is continuously input into the experience replay pool for secondary learning.

[0060] S130 uses an improved A3C algorithm for machine selection, allocating different batches of tasks to be scheduled to target machines; the improved A3C algorithm is an A3C algorithm with added security shield mechanism.

[0061] The A3C (Asynchronous Advantage Actor-Critic) algorithm combines policy gradient methods and value function learning to approximate Markov decision process problems. It employs an Actor-Critic structure, consisting of two networks: an Actor network and a Critic network, which learn the policy and value function respectively, enabling fast and stable reinforcement learning training.

[0062] In some embodiments, S130, the improved A3C algorithm is used to determine the priority of different sub-tasks for machine selection, and sub-tasks with a priority greater than a preset value are assigned to the target machine, including:

[0063] Create a policy network and a value network; the policy network is used to select actions using the policy gradient method, and the value network is used to evaluate the value of the current state.

[0064] The policy network calculates the priority of each available machine and selects the available machine with the highest priority. The value network guides the policy network to select the target machine in descending order of priority.

[0065] In this embodiment, the A3C model can use the output of the DQN model as input. In the current state, the policy network calculates the selection probability of each available machine. The selection probability represents the priority of assigning a task to a particular machine. Based on the selected machine, the task is assigned to that machine, and the environment enters a new state. The selected machine is then continuously adjusted according to the value network to achieve efficient additive manufacturing shop floor scheduling.

[0066] In some embodiments, before S130, which uses the improved A3C algorithm to select machines and assigns different batches of tasks to be scheduled to the target machines, the method further includes:

[0067] The A3C algorithm is optimized based on a safety shield mechanism to ensure that the machine load does not exceed the maximum allowable load and that the number of material changes does not exceed the maximum allowable number of changes.

[0068] In the actual scheduling of additive manufacturing workshops, ensuring the safety and stability of scheduling strategies is paramount. Here, during each scheduling session, the system performs a safety assessment of the selected actions to ensure that unsafe situations such as equipment overload or excessive material changeover costs are not triggered. Specific assessments include: equipment load safety checks, ensuring that the machine load does not exceed the maximum permissible load; and material changeover frequency checks, limiting the number of material changes to ensure that it does not exceed the maximum permissible number. If an action is deemed unsafe during scheduling, the system records the specific information of that action and updates the model to ensure safer and more efficient future scheduling.

[0069] In an alternative embodiment, please refer to Figure 2 , Figure 2This is a schematic diagram illustrating the principle of the additive manufacturing workshop scheduling method based on safety reinforcement learning provided in this application embodiment. After training the scheduling model, the reinforcement learning scheduling strategy can be applied to actual workshop production. The specific execution steps are as follows: 1) Real-time status monitoring: The system monitors the status of tasks and equipment in the workshop in real time and inputs the current state vector S into the reinforcement learning model. 2) Optimal action selection: Based on the scheduling action output by the reinforcement learning model and combined with the shield mechanism, an optimal and safe scheduling scheme is selected. 3) Task execution: The selected additive manufacturing workshop is scheduled to be executed on suitable equipment, ensuring that the load of each piece of equipment is within a safe range and minimizing material switching between tasks. 4) Feedback and learning: The execution effect of each additive manufacturing workshop scheduling is recorded, including information such as task completion time, equipment load, and material switching. This data will be used to further optimize the reinforcement learning model. A schematic diagram of its application effect is shown below. Figure 3 As shown.

[0070] This invention, based on workshop simulation environment sample data, uses a deep Q-network algorithm to divide tasks into different batches. The batching method is selected according to the workshop state vector S and the reward function to dynamically adjust batch information. The A3C algorithm is used for machine selection, assigning higher-priority tasks within each batch to specific machines. Machine allocation is dynamically adjusted based on the workshop state vector S and the reward function. By comprehensively considering task urgency, processing time, and machine load, production scheduling is dynamically optimized, improving scheduling efficiency and resource utilization. This achieves multi-objective balance, reduces the additional time and production costs caused by frequent material changes, and ensures material continuity between batches. A safety shield mechanism is added to the model to prevent it from entering an unsafe state, ensuring that equipment load operates within safe thresholds and improving the safety and stability of the production process.

[0071] In some embodiments, please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an additive manufacturing workshop scheduling device based on security reinforcement learning; this application provides an additive manufacturing workshop scheduling device 400 based on security reinforcement learning, including: a determination module 410, a division module 420, and an allocation module 430, wherein,

[0072] Module 410 is configured to determine the workshop state vector S based on workshop simulation environment parameters. Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled.

[0073] The partitioning module 420 is configured to use the DQN algorithm, combined with the workshop state vector S and the reward function, to determine the priority of the tasks to be scheduled, and to determine the batch of the tasks to be scheduled based on the priority.

[0074] The allocation module 430 is configured to use the improved A3C algorithm to select machines and allocate the scheduled tasks in different batches to the target machines; the improved A3C algorithm is the A3C algorithm with added security shield mechanism.

[0075] In some embodiments, the additive manufacturing workshop scheduling device 400 based on security reinforcement learning further includes an acquisition module; the acquisition module is configured as follows:

[0076] Obtain workshop simulation environment parameters; workshop simulation environment parameters include task number, material type, task urgency, area occupied by the task, task due time, available machines for each operation and corresponding running time.

[0077] In some embodiments, the partitioning module 420 is specifically configured as follows:

[0078] The deep Q-network algorithm is used to predict the Q-value of each scheduling action. For each shop state vector S, the deep Q-network outputs an action A, and the action with the largest Q-value is selected according to the following formula:

[0079] ;

[0080] Where A represents the set of all possible actions, Representing the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, which are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, real-time priority rule with multi-objective trade-offs, and adaptive scheduling rule based on weighted average of remaining time and urgency.

[0081] In some embodiments, the formula for the reward function is:

[0082] ;

[0083] ;

[0084] Where, Tard represents the total delay time of all tasks, Load represents the machine's workload, changeCost represents the material changeover cost between different tasks, and isSafe indicates whether the current action is safe. Indicates the currently selected action. This represents a set of unsafe actions. It is an adjustment factor used to control the impact of delay time, equipment load, material changeover, and operational safety on the total reward.

[0085] In some embodiments, the allocation module 430 is specifically configured as follows:

[0086] Create a policy network and a value network; the policy network is used to select actions using the policy gradient method, and the value network is used to evaluate the value of the current state.

[0087] The policy network calculates the priority of each available machine and selects the available machine with the highest priority. The value network guides the policy network to select the target machine in descending order of priority.

[0088] In some embodiments, the allocation module 430 is specifically configured as follows:

[0089] The A3C algorithm is optimized based on a safety shield mechanism to ensure that the machine load does not exceed the maximum allowable load and that the number of material changes does not exceed the maximum allowable number of changes.

[0090] The additive manufacturing workshop scheduling device based on security reinforcement learning provided in this application embodiment can realize the various processes in the embodiments corresponding to the additive manufacturing workshop scheduling method based on security reinforcement learning described above. To avoid repetition, it will not be described again here.

[0091] It should be noted that the additive manufacturing workshop scheduling device based on security reinforcement learning provided in this application embodiment and the additive manufacturing workshop scheduling method based on security reinforcement learning provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned additive manufacturing workshop scheduling method based on security reinforcement learning, and the repeated parts will not be described again.

[0092] This application also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the aforementioned additive manufacturing workshop scheduling method based on security reinforcement learning. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0093] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0094] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A scheduling method for additive manufacturing workshops based on safety reinforcement learning, characterized in that, include: The workshop state vector S is determined based on the workshop simulation environment parameters, where S = {T, L, Tard, C, Index}; where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled. The priority of the tasks to be scheduled is determined by using the DQN algorithm, combining the shop state vector S and the reward function, and the batch of tasks to be scheduled is determined based on the priority. Using a deep Q-network algorithm, combined with the workshop state vector S and the reward function, the task to be scheduled is divided into subtasks in different batches, including: The Q-value of each scheduling action is predicted using a deep Q-network algorithm. For each workshop state vector S, the deep Q-network outputs an action A, and the action with the largest Q-value is selected according to the following formula: TO * =arg maxQ(S,A); Where A represents the set of all possible actions, A * The optimal action is represented by A, which corresponds to five different scheduling rules, A = {0, 1, 2, 3, 4}. The corresponding scheduling rules are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, real-time priority rule with multi-objective trade-offs, and adaptive scheduling rule based on weighted average of remaining time and urgency. The formula for the reward function is: R=-((α·Tard+β·Load+γ·changeCostt+η·isSafe); Where, Tard represents the total delay time of all tasks, Load represents the machine's workload, changeCost represents the material switching cost between different tasks, isSafe indicates whether the current action is safe, 'a' represents the currently selected action, and A Safe This represents the set of unsafe actions, where α, β, γ, and η are adjustment coefficients used to control the impact of delay time, equipment load, material changeover, and action safety on the total reward. The improved A3C algorithm is used for machine selection, and the scheduled tasks in different batches are assigned to target machines; the improved A3C algorithm is the A3C algorithm with added security shield mechanism; Before using the improved A3C algorithm to select machines and assigning different batches of the scheduled tasks to the target machines, the process also includes: The A3C algorithm is optimized based on a security shield mechanism to ensure that the machine load does not exceed the maximum allowable load and that the number of material switching times does not exceed the maximum allowable number of switching times.

2. The additive manufacturing workshop scheduling method based on security reinforcement learning according to claim 1, characterized in that, Before determining the workshop state vector S based on the workshop simulation environment parameters, the method further includes: Obtain workshop simulation environment parameters; the workshop simulation environment parameters include task number, material type, task urgency, area occupied by the task, task due time, number of operations, available machines for each operation and corresponding running time.

3. The additive manufacturing workshop scheduling method based on security reinforcement learning according to claim 1, characterized in that, The step of using the improved A3C algorithm for machine selection, allocating different batches of tasks to be scheduled to target machines, includes: Create a policy network and a value network; the policy network is used to select actions using a policy gradient method, and the value network is used to evaluate the value of the current state. The policy network calculates the priority of each available machine and selects the available machine with the highest priority. The value network guides the policy network to select the target machine in descending order of priority.

4. An additive manufacturing workshop scheduling device based on safety reinforcement learning, used to implement the additive manufacturing workshop scheduling method based on safety reinforcement learning according to any one of claims 1 to 3, characterized in that, include: The process involves defining modules, dividing modules, and allocating modules. The determining module is configured to determine the workshop state vector S based on the workshop simulation environment parameters, S = {T, L, Tard, C, Index}; where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled. The partitioning module is configured to use the DQN algorithm, combined with the workshop state vector S and the reward function, to determine the priority of the tasks to be scheduled, and to determine the batch of the tasks to be scheduled based on the priority. The partitioning module, using a deep Q-network algorithm, combines the workshop state vector S and the reward function to divide the task to be scheduled into subtasks in different batches, including: The Q-value of each scheduling action is predicted using a deep Q-network algorithm. For each workshop state vector S, the deep Q-network outputs an action A, and the action with the largest Q-value is selected according to the following formula: TO * =arg maxQ(S,A); Where A represents the set of all possible actions, A * The optimal action is represented by A, which corresponds to five different scheduling rules, A = {0, 1, 2, 3, 4}. The corresponding scheduling rules are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, real-time priority rule with multi-objective trade-offs, and adaptive scheduling rule based on weighted average of remaining time and urgency. The formula for the reward function is: R=-(α·Tard+β·Load+)γ·changeCost+η·isSafe); Where, Tard represents the total delay time of all tasks, Load represents the machine's workload, changeCost represents the material switching cost between different tasks, isSafe indicates whether the current action is safe, 'a' represents the currently selected action, and A Safe This represents the set of unsafe actions, where α, β, γ, and η are adjustment coefficients used to control the impact of delay time, equipment load, material changeover, and action safety on the total reward. The allocation module is configured to use an improved A3C algorithm to select machines and allocate different batches of the tasks to be scheduled to target machines; the improved A3C algorithm is an A3C algorithm with added security shield mechanism. The allocation module, before using the improved A3C algorithm to select machines and allocate different batches of the tasks to be scheduled to the target machines, further includes: The A3C algorithm is optimized based on a security shield mechanism to ensure that the machine load does not exceed the maximum allowable load and that the number of material switching times does not exceed the maximum allowable number of switching times.

5. An electronic device, comprising a processor and a memory; said memory having a storage for computer programs, wherein, When the computer program is executed by the processor, it implements the additive manufacturing workshop scheduling method based on security reinforcement learning as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method of any one of claims 1 to 3.

Images