Additive manufacturing workshop scheduling method based on safety reinforcement learning

Abstract

The invention provides an additive manufacturing workshop scheduling method based on safety reinforcement learning, and relates to the technical field of deep learning, and the method comprises the steps: determining a workshop state vector S, # imgabs0 # based on workshop simulation environment parameters; wherein T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total tardiness time of all the tasks, C represents the total completion time of completed tasks, and Index represents the tasks being scheduled at present; using a DQN algorithm, combining the workshop state vector S and a reward function to determine the priority of the to-be-scheduled task, and determining the batch of the to-be-scheduled task based on the priority; carrying out machine selection by utilizing an improved A3C algorithm, and distributing the to-be-scheduled tasks of different batches to a target machine; the improved A3C algorithm is an A3C algorithm added with a safe shield mechanism. By comprehensively considering the task emergency degree, the processing time and the machine load, the production scheduling is dynamically optimized, the scheduling efficiency and the resource utilization rate are improved, and multi-target balance is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of additive manufacturing workshop scheduling, and in particular to an additive manufacturing workshop scheduling method based on secure reinforcement learning. Background Art

[0002] In the field of intelligent manufacturing and production scheduling, especially in the online production scheduling of additive manufacturing workshops, how to effectively allocate production tasks has become an important research topic. Additive Manufacturing (AM) builds products by adding materials layer by layer and is widely used in aerospace, medical equipment, and mold manufacturing. However, traditional scheduling methods have obvious limitations when dealing with the complex and dynamic environment of additive manufacturing workshops.

[0003] At present, the task scheduling of additive manufacturing workshops mainly adopts heuristic algorithms, such as shortest processing time priority and earliest deadline priority. These methods are simple to implement, consume low computing resources, and are suitable for static or low-complexity production environments. However, with the complexity of additive manufacturing processes, these methods cannot effectively handle conflicts between multiple tasks and competition for equipment resources, resulting in reduced production efficiency and even possible equipment overload. On the other hand, scheduling methods based on reinforcement learning have gradually attracted attention. Reinforcement learning learns strategies that can maximize cumulative rewards through interaction with the environment. This method can adaptively adjust scheduling strategies to adapt to dynamically changing production environments. However, pure reinforcement learning methods require a lot of exploration in the early stages of training, which may produce unsafe scheduling decisions, such as equipment overload or frequent material switching, affecting the safety and efficiency of production. In addition, due to the high dimensionality of the state space of the additive manufacturing scheduling problem, the training and convergence of the model require a lot of computing resources, which is not conducive to deployment in actual production. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the current scheduling method has obvious limitations when dealing with the complex dynamic environment of an additive manufacturing workshop. The present invention provides an additive manufacturing workshop scheduling method based on secure reinforcement learning.

[0005] The technical solution of the embodiment of the present application is implemented as follows: A first aspect of an embodiment of the present application provides an additive manufacturing workshop scheduling method based on secure reinforcement learning, comprising: Determine the workshop state vector S based on the workshop simulation environment parameters, ; Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled; Using the DQN algorithm, combined with the workshop state vector S and the reward function, the priority of the tasks to be scheduled is determined, and the batch of the tasks to be scheduled is determined based on the priority; The improved A3C algorithm is used to select machines, and the tasks to be scheduled in different batches are allocated to the target machine; the improved A3C algorithm is an A3C algorithm with a safety shield mechanism added.

[0006] Optionally, before determining the workshop state vector S based on the workshop simulation environment parameters, the method further includes: Acquire workshop simulation environment parameters; the workshop simulation environment parameters include task number, material type, task urgency, task area, task expiration time, available machines for each operation and corresponding running time.

[0007] Optionally, the using of the DQN algorithm, in combination with the workshop state vector S and the reward function, to determine the priority of the task to be scheduled, and determining the batch of the task to be scheduled based on the priority, includes: The deep Q network algorithm is used to predict the Q value of each scheduling action. For each workshop state vector S, the deep Q network outputs an action A and selects the action with the largest Q value according to the following formula: ; Where A represents the set of all possible actions. represents the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, and the corresponding scheduling rules are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, multi-objective trade-off real-time priority rule, and adaptive scheduling rule based on remaining time and urgency weighting.

[0008] Optionally, the formula of the reward function is: ; ; Tard represents the total delay time of all tasks, Load represents the load of the machine, changeCost represents the material switching cost between different tasks, and isSafe represents whether the current action is safe. Indicates the currently selected action. represents a set of unsafe actions, is an adjustment factor used to control the impact of delay time, equipment load, material switching and operation safety on the total reward.

[0009] Optionally, the improved A3C algorithm is used to select a machine to allocate different batches of the to-be-scheduled tasks to a target machine, including: Create a policy network and a value network; the policy network is used to select actions through a policy gradient method, and the value network is used to evaluate the value of the current state; The priority of each available machine is calculated through the policy network and the available machine with the highest priority is selected, and the value network is used to guide the policy network to select the target machine in order from high to low priority.

[0010] Optionally, before using the improved A3C algorithm to select a machine and assigning different batches of the to-be-scheduled tasks to the target machine, the method further includes: The A3C algorithm is optimized based on a safety shield mechanism to ensure that the load of the machine does not exceed the maximum allowable load and that the number of material switching times does not exceed the maximum allowable number of switching times.

[0011] A second aspect of the embodiment of the present application provides an additive manufacturing workshop scheduling device based on secure reinforcement learning, including: a determination module, a division module and an allocation module, wherein: The determination module is configured to determine the workshop state vector S based on the workshop simulation environment parameters, ; Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled; The partitioning module is configured to use a DQN algorithm, in combination with a workshop state vector S and a reward function, to determine the priority of the tasks to be scheduled, and to determine the batches of the tasks to be scheduled based on the priority; The allocation module is configured to use an improved A3C algorithm to select machines and allocate different batches of the tasks to be scheduled to target machines; the improved A3C algorithm is an A3C algorithm with a safety shield mechanism added.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the additive manufacturing workshop scheduling method based on secure reinforcement learning described in the first aspect.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Compared with the prior art, the technical solution provided by this application has the following beneficial effects: The present invention provides an additive manufacturing workshop scheduling method, device and equipment based on secure reinforcement learning. Based on the sample data of the workshop simulation environment, the deep Q network algorithm divides the tasks into different batches, and selects the batching method according to the workshop state vector S and the reward function to dynamically adjust the batch information. The A3C algorithm is used to select machines, and the machines for the tasks with higher priority in the batch are selected in batch processing. The machine allocation is dynamically adjusted according to the workshop state vector S and the reward function. By comprehensively considering the urgency of the task, the processing time and the machine load, the production scheduling is dynamically optimized, the scheduling efficiency and resource utilization are improved, the multi-objective balance is achieved, and the extra time consumption and production costs caused by frequent material switching are reduced, and the continuity of materials between batches is guaranteed. A safety shield mechanism is added to the model to prevent the model from entering an unsafe state, ensure that the equipment load operates within the safety threshold, and improve the safety and stability of the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of an additive manufacturing workshop scheduling method based on secure reinforcement learning provided in an embodiment of the present application; Figure 2 A schematic diagram of the principle of an additive manufacturing workshop scheduling method based on secure reinforcement learning provided in an embodiment of the present application; Figure 3 A schematic diagram of the application effect of the additive manufacturing workshop scheduling method based on secure reinforcement learning provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an additive manufacturing workshop scheduling device based on secure reinforcement learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] Below, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present application.

[0017] The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components. All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0018] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0019] In some embodiments, see Figure 1 , Figure 1 A flow chart of an additive manufacturing workshop scheduling method based on secure reinforcement learning provided in an embodiment of the present application; an additive manufacturing workshop scheduling method based on secure reinforcement learning provided in an embodiment of the present application includes: S110, determining a workshop state vector S based on workshop simulation environment parameters, ; Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled.

[0020] In this embodiment, a simulation environment is first constructed, which can accurately simulate the state changes of the workshop in the real production environment to ensure that the scheduling strategy can adapt to the actual situation of the workshop.

[0021] In some embodiments, before determining the workshop state vector S based on the workshop simulation environment parameters at S110, the method further includes: Obtain workshop simulation environment parameters; workshop simulation environment parameters include task number, material type, task urgency, task area, task expiration time, available machines for each operation and corresponding running time.

[0022] S120, using a deep Q network algorithm, combined with the workshop state vector S and a reward function, the tasks to be scheduled are divided into subtasks of different batches.

[0023] The deep Q network algorithm uses a deep neural network to process the input state and output the corresponding Q value for each action. The Q function is approximated through training, so that the Q value of each state-action pair can be estimated in the high-dimensional state space. The reward function defines the immediate feedback obtained by the agent at each time step and is the goal of the agent's learning. Its essence is to convey the goal to the agent, that is, all the "goals" or "purposes" of the agent can be attributed to maximizing the probability expectation value of the cumulative sum of the scalar reward signals received by the agent.

[0024] In some embodiments, S120, using the DQN algorithm, combined with the workshop state vector S and the reward function to determine the priority of the task to be scheduled, and determining the batch of the task to be scheduled based on the priority, includes: The deep Q network algorithm is used to predict the Q value of each scheduling action. For each workshop state vector S, the deep Q network outputs an action A and selects the action with the largest Q value according to the following formula: ; Where A represents the set of all possible actions. represents the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, and the corresponding scheduling rules are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, multi-objective trade-off real-time priority rule, and adaptive scheduling rule based on remaining time and urgency weighting.

[0025] In this embodiment, the deep Q network (DQN) selects the most suitable scheduling rule according to the current environment. Through continuous learning, the Q value network can effectively measure the long-term returns of different actions. Among them, the urgency priority batch processing rule: when the workshop receives an urgent task or a priority order from a specific customer, this rule is selected and batched according to the urgency value; the shortest processing time batch processing rule: when the task volume is large and the production resources are tight, this rule can be selected to shorten the overall production cycle; the job load balancing batch processing rule: when the workshop task load is uneven and some machines are overloaded, this rule is selected to balance the workshop resource utilization; the real-time priority rule of multi-objective trade-off: when the workshop environment is complex (such as multiple orders with different delivery dates, different material requirements, and conflicting processing conditions), this rule is selected to dynamically adjust the priority by weighing different goals, balancing the processing time, material compatibility, urgency and other requirements; the adaptive scheduling rule based on the weighting of remaining time and urgency: when the workshop faces uneven remaining time of tasks and different urgency, this rule is selected. The adaptive scheduling rule can determine the priority by weighting according to the remaining time and urgency of the task.

[0026] It should be noted that DQN also involves an experience replay mechanism and a target network mechanism. The experience replay mechanism stores each scheduling decision in a memory pool and randomly samples a small batch of decision records for training to alleviate the unstable training problem caused by sample correlation. The target network mechanism updates the Q value through a target network with the same parameters as the main network but not shared, thereby stabilizing the training process. The target network parameters are synchronized with the main network at a certain training step.

[0027] In some embodiments, the reward function is formulated as: ; ; Tard represents the total delay time of all tasks, Load represents the load of the machine, changeCost represents the material switching cost between different tasks, and isSafe represents whether the current action is safe. Indicates the currently selected action. represents a set of unsafe actions, is an adjustment factor used to control the impact of delay time, equipment load, material switching and operation safety on the total reward.

[0028] In this embodiment, Tard represents the delay time of the task. If the task is not completed on time, a negative reward will be received. Load represents the load of the machine. If the equipment is overloaded, it will also be punished. ChangeCost represents the material switching cost between different tasks. The more frequent the material switching, the higher the cost and the greater the negative reward. IsSafe represents whether the current action is safe, that is, whether the machine will exceed the load. Indicates the currently selected action. Represents a set of unsafe actions. is the adjustment coefficient used to control the impact of delay time, equipment load, material switching and action safety on the total reward. The reward function quantifies the benefits of each action, and the reinforcement learning model will gradually learn to avoid high-penalty actions during training.

[0029] In this embodiment, the optimization process of the reinforcement learning strategy is completed through continuous iteration and multiple rounds of training. After each training cycle, the model will update its strategy to gradually approach the optimal scheduling strategy. Specifically including: 1) Model update: Through the training feedback of DQN and A3C, the scheduling strategy is continuously optimized to make the model more adaptable to the scheduling of different workshop states. 2) Loss function: The loss function measures the accuracy of the model output, including the original Q value and the loss of the policy value. The loss function is calculated as follows: For the DQN model, the loss function calculates the difference in Q values ​​and updates the network weights. For the A3C model, the loss function calculates the value difference of the critic network, the action advantage of the policy network, and updates the policy parameters. 3) Model parameter preservation and playback: After each iteration, the model saves the current parameters for calling in actual applications. In addition, new data is continuously input through the experience replay pool for secondary learning.

[0030] S130, using the improved A3C algorithm to select a machine, and assigning different batches of tasks to be scheduled to the target machine; the improved A3C algorithm is an A3C algorithm with a safety shield mechanism added.

[0031] The A3C (Asynchronous Advantage Actor-Critic) algorithm combines the policy gradient method with the learning of the value function to approximate the solution of the Markov decision process problem. It adopts the Actor-Critic structure, which consists of two networks, Actor and Critic, to learn the policy and value function respectively, to achieve fast and stable reinforcement learning training.

[0032] In some embodiments, S130, using the improved A3C algorithm to perform machine selection to determine the priorities of different subtasks, and assigning subtasks with priorities greater than a preset value to a target machine, includes: Create a policy network and a value network; the policy network is used to select actions through the policy gradient method, and the value network is used to evaluate the value of the current state; The policy network calculates the priority of each available machine and selects the available machine with the highest priority, and uses the value network to guide the policy network to select the target machine in order from high to low priority.

[0033] In this embodiment, the A3C model can use the output of the DQN model as input, and calculate the selection probability of each available machine through the policy network in the current state. The selection probability represents the priority of assigning tasks to a certain machine. The task is assigned to the machine according to the selected machine, and the environment enters a new state. Then, the selected machine is continuously adjusted according to the value network to achieve efficient additive manufacturing workshop scheduling.

[0034] In some embodiments, at S130, before the improved A3C algorithm is used to select a machine and the different batches of tasks to be scheduled are assigned to the target machine, the following steps are further included: The A3C algorithm is optimized based on the safety shield mechanism to ensure that the machine load does not exceed the maximum allowable load and the number of material switching times does not exceed the maximum allowable number of switching times.

[0035] In the actual scheduling of additive manufacturing workshops, it is particularly important to ensure the safety and stability of scheduling strategies. Here, each time scheduling is performed, the system will conduct a safety assessment on the selected action to ensure that it will not cause unsafe situations such as equipment overload or excessive material switching costs. The specific assessment content includes: equipment load safety check, that is, to ensure that the machine load does not exceed the maximum allowable load; material switching number check, that is, to limit the number of material switching times to ensure that the number of material switching times does not exceed the maximum allowed number of switching times. If an action is judged to be unsafe during the scheduling process, the system will record the specific information of the action and update the model to ensure that future scheduling is safer and more efficient.

[0036] In an alternative embodiment, see Figure 2 , Figure 2 Schematic diagram of the principle of the additive manufacturing workshop scheduling method based on safe reinforcement learning provided in the embodiment of the present application. After completing the training of the scheduling model, the reinforcement learning scheduling strategy can be applied to the actual workshop production. The specific execution steps are as follows: 1) Real-time status monitoring: The system monitors the status of tasks and equipment in the workshop in real time, and inputs the current state vector S into the reinforcement learning model. 2) Optimal action selection: According to the scheduling action output by the reinforcement learning model and combined with the shield mechanism, an optimal and safe scheduling plan is selected. 3) Task execution: Schedule the selected additive manufacturing workshop to the appropriate equipment for execution, ensure that the load of each equipment is within a safe range, and minimize material switching between tasks. 4) Feedback and learning: Record the execution effect of each additive manufacturing workshop scheduling, including task completion time, equipment load, material switching and other information. These data will be used to further optimize the reinforcement learning model. The schematic diagram of its application effect is shown in the figure. Figure 3 shown.

[0037] The embodiment of the present invention is based on the sample data of the workshop simulation environment. The deep Q network algorithm divides the tasks into different batches, and selects the batching method according to the workshop state vector S and the reward function to dynamically adjust the batch information. The A3C algorithm is used to select machines, and the machines for the tasks with higher priority in the batch are selected in batch processing. The machine allocation is dynamically adjusted according to the workshop state vector S and the reward function. By comprehensively considering the urgency of the task, the processing time and the machine load, the production scheduling is dynamically optimized, the scheduling efficiency and resource utilization are improved, and the multi-objective balance is achieved. It also reduces the extra time consumption and production costs caused by frequent material switching, and ensures the continuity of materials between batches. A safety shield mechanism is added to the model to prevent the model from entering an unsafe state, ensuring that the equipment load operates within the safety threshold, and improving the safety and stability of the production process.

[0038] In some embodiments, see Figure 4 , Figure 4 A structural diagram of an additive manufacturing workshop scheduling device based on secure reinforcement learning provided in an embodiment of the present application; an additive manufacturing workshop scheduling device 400 based on secure reinforcement learning is provided in an embodiment of the present application, including: a determination module 410, a division module 420 and an allocation module 430, wherein: The determination module 410 is configured to determine a workshop state vector S based on a workshop simulation environment parameter, ; Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled; A partitioning module 420 is configured to use a DQN algorithm to determine the priorities of the tasks to be scheduled in combination with the shop state vector S and the reward function, and determine the batches of the tasks to be scheduled based on the priorities; The allocation module 430 is configured to use the improved A3C algorithm to select machines and allocate different batches of the tasks to be scheduled to the target machines; the improved A3C algorithm is an A3C algorithm with a safety shield mechanism added.

[0039] In some embodiments, the additive manufacturing workshop scheduling device 400 based on secure reinforcement learning further includes an acquisition module; the acquisition module is configured as follows: Obtain workshop simulation environment parameters; workshop simulation environment parameters include task number, material type, task urgency, task area, task expiration time, available machines for each operation and corresponding running time.

[0040] In some embodiments, the partitioning module 420 is specifically configured as follows: The deep Q network algorithm is used to predict the Q value of each scheduling action. For each workshop state vector S, the deep Q network outputs an action A and selects the action with the largest Q value according to the following formula: ; Where A represents the set of all possible actions. represents the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, and the corresponding scheduling rules are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, multi-objective trade-off real-time priority rule, and adaptive scheduling rule based on remaining time and urgency weighting.

[0041] In some embodiments, the reward function is formulated as: ; ; Tard represents the total delay time of all tasks, Load represents the load of the machine, changeCost represents the material switching cost between different tasks, and isSafe represents whether the current action is safe. Indicates the currently selected action. represents a set of unsafe actions, is an adjustment factor used to control the impact of delay time, equipment load, material switching and operation safety on the total reward.

[0042] In some embodiments, the allocation module 430 is specifically configured as follows: Create a policy network and a value network; the policy network is used to select actions through the policy gradient method, and the value network is used to evaluate the value of the current state; The policy network calculates the priority of each available machine and selects the available machine with the highest priority, and uses the value network to guide the policy network to select the target machine in order from high to low priority.

[0043] In some embodiments, the allocation module 430 is specifically configured as follows: The A3C algorithm is optimized based on the safety shield mechanism to ensure that the machine load does not exceed the maximum allowable load and the number of material switching times does not exceed the maximum allowable number of switching times.

[0044] The additive manufacturing workshop scheduling device based on secure reinforcement learning provided in the embodiment of the present application can implement the various processes in the embodiments corresponding to the above-mentioned additive manufacturing workshop scheduling method based on secure reinforcement learning. To avoid repetition, they will not be repeated here.

[0045] It should be noted that the additive manufacturing workshop scheduling device based on safety reinforcement learning provided in the embodiment of the present application and the additive manufacturing workshop scheduling method based on safety reinforcement learning provided in the embodiment of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned additive manufacturing workshop scheduling method based on safety reinforcement learning, and the repeated parts will not be repeated.

[0046] The present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned additive manufacturing workshop scheduling method based on secure reinforcement learning is implemented. The computer-readable medium may be included in the device / apparatus / system described in the above-mentioned embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed, the method according to the embodiment of the present application is implemented.

[0047] According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above.

[0048] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-described embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for scheduling additive manufacturing workshops based on secure reinforcement learning, characterized in that: include: Determine the workshop state vector S based on the workshop simulation environment parameters, ; Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled; Using the DQN algorithm, combined with the workshop state vector S and the reward function, the priority of the tasks to be scheduled is determined, and the batch of the tasks to be scheduled is determined based on the priority; The improved A3C algorithm is used to select machines, and the tasks to be scheduled in different batches are allocated to the target machine; the improved A3C algorithm is an A3C algorithm with a safety shield mechanism added.

2. The additive manufacturing workshop scheduling method based on secure reinforcement learning according to claim 1 is characterized in that: Before determining the workshop state vector S based on the workshop simulation environment parameters, the method further includes: Acquire workshop simulation environment parameters; the workshop simulation environment parameters include task number, material type, task urgency, task area, task expiration time, number of operations, available machines for each operation and corresponding running time.

3. The additive manufacturing workshop scheduling method based on secure reinforcement learning according to claim 1 is characterized in that: The deep Q network algorithm is used to divide the tasks to be scheduled into subtasks of different batches in combination with the workshop state vector S and the reward function, including: The deep Q network algorithm is used to predict the Q value of each scheduling action. For each workshop state vector S, the deep Q network outputs an action A and selects the action with the largest Q value according to the following formula: ； Where A represents the set of all possible actions. represents the optimal action, A corresponds to five different scheduling rules, A={0, 1, 2, 3, 4}, and the corresponding scheduling rules are: urgency priority batch processing rule, shortest processing time batch processing rule, job load balancing batch processing rule, multi-objective trade-off real-time priority rule, and adaptive scheduling rule based on remaining time and urgency weighting.

4. The additive manufacturing workshop scheduling method based on secure reinforcement learning according to claim 1 is characterized in that: The formula of the reward function is: ； ； Tard represents the total delay time of all tasks, Load represents the load of the machine, changeCost represents the material switching cost between different tasks, and isSafe represents whether the current action is safe. Indicates the currently selected action. represents a set of unsafe actions, is an adjustment factor used to control the impact of delay time, equipment load, material switching and operation safety on the total reward.

5. The additive manufacturing workshop scheduling method based on secure reinforcement learning according to claim 1, characterized in that: The improved A3C algorithm is used to select a machine and allocate different batches of the tasks to be scheduled to the target machine, including: Create a policy network and a value network; the policy network is used to select actions through a policy gradient method, and the value network is used to evaluate the value of the current state; The priority of each available machine is calculated through the policy network and the available machine with the highest priority is selected, and the value network is used to guide the policy network to select the target machine in order from high to low priority.

6. The additive manufacturing workshop scheduling method based on secure reinforcement learning according to claim 1 is characterized in that: Before using the improved A3C algorithm to select a machine and assigning different batches of the tasks to be scheduled to the target machine, the method further includes: The A3C algorithm is optimized based on a safety shield mechanism to ensure that the load of the machine does not exceed the maximum allowable load and that the number of material switching times does not exceed the maximum allowable number of switching times.

7. An additive manufacturing workshop scheduling device based on secure reinforcement learning, characterized in that: include: Determine modules, divide modules and allocate modules, among which, The determination module is configured to determine the workshop state vector S based on the workshop simulation environment parameters, ; Where T represents the current time, L represents the number of tasks being processed in the current batch, Tard represents the total delay time of all tasks, C represents the total completion time of completed tasks, and Index represents the task currently being scheduled; The partitioning module is configured to use a DQN algorithm, in combination with a workshop state vector S and a reward function, to determine the priority of the tasks to be scheduled, and to determine the batches of the tasks to be scheduled based on the priority; The allocation module is configured to use an improved A3C algorithm to select machines and allocate different batches of the tasks to be scheduled to target machines; the improved A3C algorithm is an A3C algorithm with a safety shield mechanism added.

8. An electronic device comprising a processor and a memory; the memory stores a computer program, wherein: When the computer program is executed by the processor, the computer program implements the additive manufacturing workshop scheduling method based on secure reinforcement learning according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images