Intelligent Scheduling Method Driven by Hybrid Knowledge and Data with Batch Random Arrivals

Abstract

The present invention relates to the technical field of hybrid flow shop scheduling, and particularly belongs to an intelligent scheduling method driven by the hybrid of knowledge and data under the condition of random arrival of batches. Initialize the batches in the set of batches to be scheduled as batch operations, use the batch operation selection strategy to select a batch operation, execute the position-based insertion strategy to adjust the processing order of the batch operations on the selected machine, remove the batch operations that have completed processing from the set of unscheduled batch operations and the set of available batch operations, execute the strategy based on critical sub-batch operations for critical sub-batch operations, convert the randomly arriving batches into batch operations and add them to the set of unscheduled batch operations, use the double deep Q-network to train the policy network based on the above operation process, obtain an intelligent scheduling scheme under the condition of random arrival of batches by using the trained policy network, and terminate the scheduling. The present invention has the positive effects of realizing efficient, intelligent and flexible shop scheduling.

Description

Technical Field

[0001] The present invention relates to the technical field of hybrid flow shop scheduling, and particularly belongs to an intelligent scheduling method driven by the hybrid of knowledge and data under random arrival of batches. Background Art

[0002] With the increasing of personalized customer demands, enterprises must quickly respond to market demands. Under this background, the arrival of new batches is highly random, and these batches can be converted into one or more batches. In order to adapt to the dynamic environmental changes brought by these new batches, the scheduling must be adjusted in real time according to their arrival times. However, the problem of random arrival of batches has not been fully solved in existing research. Therefore, there is an urgent need to explore flexible and advanced scheduling methods to ensure that new batches can be inserted into the production plan in a timely manner while minimizing the delay of all batches.

[0003] Currently, the solution methods for scheduling problems can be mainly divided into three categories: mathematical programming models, heuristic algorithms, and meta-heuristic algorithms. Although mathematical programming models can obtain optimal solutions, the computational complexity of mathematical programming models will increase sharply with the growth of the problem scale, resulting in a significant extension of the solution time and an increase in resource consumption, which is unacceptable for a dynamic environment that requires real-time scheduling. Although meta-heuristic algorithms are more efficient than mathematical programming in solving large-scale problems, it may still take a long time to find the desired solution, making it difficult to meet the real-time scheduling requirements. Heuristic algorithms provide an effective alternative. Heuristic algorithms emphasize quick decision-making. Such algorithms utilize domain knowledge and heuristic rules and can quickly generate high-quality scheduling plans, thus being particularly suitable for dynamic scheduling environments. However, heuristic algorithms usually rely on static production data, and with the increase of the problem scale, the performance of heuristic algorithms may decline. In addition, heuristic algorithms are often designed for specific scenarios and are difficult to adapt to the changes in dynamic environments.

[0004] In summary, there are still certain limitations in the research on intelligent scheduling driven by the hybrid of knowledge and data under random arrival of batches. In order to better meet the actual needs of real production scenarios, it is necessary to conduct in-depth research on the intelligent scheduling problem driven by the hybrid of knowledge and data under random arrival of batches to find a more practical solution. Summary of the Invention

[0005] The present invention provides an intelligent scheduling method driven by the hybrid of knowledge and data under random arrival of batches, which solves the technical problems of the existing scheduling methods such as untimely scheduling response, lack of effective knowledge guidance, and insufficient strategy adaptability in dealing with dynamic batch environments, so as to achieve the purpose of improving the flexibility, adaptive ability and overall scheduling performance of the scheduling system.

[0006] The intelligent scheduling method driven by the hybrid of knowledge and data under the random arrival of batches provided by the present invention is characterized by including the following steps:

[0007] Step 1, define a policy network, a state space, and an action space, and control the policy network to select an action according to the current state. Each action corresponds to one of the rule combinations of the batch operation selection policy and the machine allocation policy.

[0008] Step 2, obtain the set of batches to be scheduled at the initial moment, and initialize the batches in the set of batches to be scheduled as batch operations. Batch operations refer to the processing operations performed by batches in each production stage. The batch operations in all production stages are initialized as an unscheduled batch operation set, and the batch operations in the first production stage are initialized as an available batch operation set.

[0009] Step 3, select a batch operation from the available batch operation set using the batch operation selection policy. If the selected batch operation is a batch operation in the first production stage, it means that the batch has not been split yet. Execute the greedy-based batch operation splitting policy to split the batch operation into several sub-batch operations, and then execute the machine allocation policy to allocate a machine for the selected batch operation and perform processing on the selected machine. If the selected batch operation is not a batch operation in the first production stage, use the splitting result of the batch operation in the first production stage, execute the machine allocation policy to allocate a machine for the selected batch operation, and perform processing on the selected machine.

[0010] Step 4, execute the position-based insertion policy to adjust the processing order of the batch operations on the selected machine, and the selected machine completes processing according to the adjusted batch operation order.

[0011] Step 5, remove the completed batch operations from the unscheduled batch operation set and the available batch operation set. If the removed batch operation is not a batch operation in the last production stage, add the batch operation in the next production stage of the removed batch operation to the available batch operation set.

[0012] Step 6, after all batch operations in all production stages are completed, find the critical sub-batch operation from all sub-batch operations. The critical sub-batch operation refers to the sub-batch operation with the least total processing time from the first production stage to the last production stage. Execute the policy based on the critical sub-batch operation to update the entire processing process.

[0013] Step 7, if there are randomly arriving batches from the start to the current moment, convert the randomly arriving batches into batch operations and add them to the unscheduled batch operation set, and add the batch operations in the first production stage to the available batch operation set.

[0014] Step 8: When the set of unscheduled batch operations is empty, all batch operations are scheduled;

[0015] Step 9: Use the dual-depth Q network to train the policy network based on steps 2-8. Use the trained policy network to obtain an intelligent scheduling solution under random batch arrival and terminate the scheduling.

[0016] Furthermore, the state space is composed of the general attributes of the overall scheduling state of the workshop and the specific attributes of each production stage. The general attributes have 9 dimensions, and the specific attributes of each production stage have 18 dimensions. That is, if the workshop contains m production stages, the total dimension of the state space is 9 + 18 × m;

[0017] The action space corresponds to all rule combinations of the batch operation selection strategy and the machine allocation strategy. There are eight rules for the batch operation selection strategy and five rules for the machine allocation strategy. There are 40 rule combinations of the batch operation selection strategy and the machine allocation strategy, which are used as the output of the policy network.

[0018] Furthermore, the batch operation selection strategy includes,

[0019] The earliest delivery date priority rule gives priority to batch operations with the earliest delivery date for scheduling;

[0020] The minimum slack time priority rule gives priority to scheduling the batch operation with the minimum slack time. The slack time is defined as the batch delivery time minus the remaining processing time.

[0021] The longest processing time priority rule gives priority to the batch operation with the longest processing time for scheduling;

[0022] The shortest processing time priority rule gives priority to the batch operation with the shortest processing time;

[0023] The shortest remaining processing time priority rule gives priority to the job with the shortest remaining processing time. The remaining processing time refers to the total time required for subsequent production stages after the current batch operation is completed.

[0024] The longest remaining processing time priority rule prioritizes the batch operations with the longest remaining processing time;

[0025] The earliest start time priority rule sorts batch operations according to their earliest start time, and schedules batch operations with the highest order first.

[0026] The latest start time priority rule sorts batch operations according to their latest start time, and schedules batch operations with the highest order first.

[0027] Furthermore, the machine allocation strategy includes,

[0028] The shortest waiting time priority rule gives priority to the machine that has the shortest waiting time for the current batch operation;

[0029] The load balancing priority rule gives priority to the machine with the smallest current load. The load is defined as the number of batch operations assigned to the machine.

[0030] The earliest idle time priority rule gives priority to the machine that becomes idle the earliest after completing the currently assigned batch operation;

[0031] The longest idle time priority rule: when there are idle machines, the machine with the longest idle time is given priority;

[0032] The earliest completion time is given priority. The machine with the earliest completion time is selected first, and the current batch operation is assigned to the machine with the earliest completion time.

[0033] Furthermore, the greedy batch operation splitting strategy includes the following operation process,

[0034] Get the size of the batch operation to be split, set the maximum number of sub-batch operations that can be split, set the initial size of each sub-batch operation to zero, and set the remaining sub-batch size. The batch operation size is allocated, and the loop is entered. Unit 1 is selected from the remaining unallocated batch operation size, and the selected unit 1 is allocated to each sub-batch operation in turn. The processing time corresponding to each sub-batch operation is calculated, and the processing time of all sub-batch operations after being allocated to the selected unit 1 is compared. The sub-batch operation with the smallest processing time is selected as the final allocation object of unit 1, the size of the sub-batch operation is updated, and the total amount of the remaining unallocated batch operations is reduced by 1. Then the next loop is performed until the remaining unallocated amount is 0, completing the batch operation greedy splitting process.

[0035] Furthermore, the position-based insertion strategy includes the following operation processes:

[0036] Define the parameters: represents a batch collection, , Represents the total number of batches, Represents a batch; represents a collection of production stages, , represents the total quantity of production stages, Represents a certain stage of production; Representative batch Batch operation;

[0037] Based on the selected batch operation, the selected machine, and the batches that the machine has currently completed scheduling, determine the positions on the machine that can serve as insertion points. Traverse each insertable position in sequence, and judge whether inserting the selected batch operation into the current position meets the feasibility conditions. The feasibility conditions are shown in Formula 1 and Formula 2.

[0038] Formula 1

[0039] Where, represents the length of the idle time period of the current machine; represents the batch operation size; represents the batch operation in the production stage unit processing time; represents the batch operation in the production stage machine setup time; Formula 1 ensures that the idle time of the selected machine at a certain position can accommodate the minimum processing time required for the selected batch operation. Meeting Formula 1 means that the idle time of the selected machine at a certain position can accommodate the selected batch operation;

[0040] Formula 2

[0041] Where, represents the latest time of the machine idle time; represents the maximum number of sub-batch operations; represents the batch operation maximum sub-batch operation in the production stage end time; represents a certain production stage time to transfer to the next production stage; represents the batch operation maximum sub-batch operation size; Formula 2 ensures that the latest time point of the idle time of the selected machine at a certain position can meet the processing requirements of the last sub-batch operation of the selected batch operation. Meeting Formula 2 means that moving the selected batch operation to a certain position on the selected machine for processing will not affect the arrangement of subsequent batch operations;

[0042] If the insertion position meets the above feasibility conditions, calculate the scheduling times of the sub-batch operations of the selected batch operation. From the sum of the end time after the previous batch operation is completed in the current production stage and the machine setup time, and the sum of the end time after the previous batch operation is processed and completed in the previous production stage and the transfer time, select the larger value as the start time of the first sub-batch. From the end time of the previous sub-batch operation of the selected batch operation, and the sum of the end time of the sub-batch operation in the previous production stage and the transfer time, select the larger value as the start time of the subsequent sub-batch operations. The end time of each sub-batch operation is the sum of the start time and the processing time of the sub-batch operation. The processing time of the sub-batch operation is the unit processing time multiplied by the size of the sub-batch operation. After calculating the times for all sub-batch operations, determine whether inserting the selected batch operation will cause delays to the subsequent sub-batch operations on the selected machine. If there are no delays, insert the selected batch operation at the current position and stop the position-based insertion strategy. Otherwise, skip the current position and continue to try the next insertion point until all positions are traversed or a feasible insertion position is found.

[0043] Further, the strategy based on the critical sub-batch operation includes the following operation process.

[0044] Calculate the reduced sizes of the critical sub-batch operation in each production stage except the first production stage. Traverse all production stages except the first production stage and calculate the reduced size of the sub-batch operation of the batch operation in the production stage , The calculation method is as follows.

[0045] Where the symbol represents rounding down; represents the start time of the sub-batch operation of the batch operation in the production stage ; represents the end time of the sub-batch operation of the batch operation in the production stage ;

[0046] Select the minimum value from the calculated reduced sizes of all production stages, remove the selected minimum reduced size from the critical sub-batch operation, and merge it into the previous sub-batch operation of the critical sub-batch operation to complete the adjustment of the sizes of the two sub-batch operations, and update the scheduling process of the two sub-batch operations in each production stage.

[0047] Furthermore, the method of training the policy network based on the double deep Q network from Step 2 to Step 8 is as follows:

[0048] The double deep Q network adopts a double network structure, including a policy network and a target network, which are used for action selection and Q value evaluation respectively. The double deep Q network combines an experience replay mechanism for stable training. The specific process is as follows:

[0049] Construct a training dataset, which contains several scheduling instances with a fixed number of production stages, and set the maximum number of training rounds. At the same time, initialize the current training round as the first round. Initialize two neural networks, namely the policy network and the target network, and establish an experience replay buffer. At the beginning of each round of training, randomly select an instance from the training dataset for training. During the training process, judge whether the maximum number of training rounds has been reached. If not, continue to execute this round of training. If so, output the trained policy network. In each round of training, the policy network obtains the current workshop state and checks whether the current scheduling task is completed. If it is completed, increment the current training round by one and start the next round. Otherwise, continue to execute the subsequent operations. The policy network adopts an ε-greedy strategy to select actions, randomly selects an action with probability ε, and selects the current optimal action with the remaining probability 1 - ε. After executing the action, the policy network obtains a new state and the corresponding reward value. The current state, the selected action, the reward value, and the next state are used as an experience data and stored in the experience replay buffer. The policy network randomly samples a small batch of experience data from the experience replay buffer and calculates the target Q value y for each experience data. The calculation method is as follows:

[0050]

[0051] where r represents the reward value obtained after executing action a in the current state s; γ is the discount factor, which is used to measure the importance of future rewards; s′ is the next state reached after executing the action; represents the action value estimated by the target network for state s′ and all actions a′; represents the maximum estimated action value among all actions in state s′;

[0052] Calculate the loss between the Q value obtained by the policy network and the target Q value, and calculate the gradient of the parameters in the policy network through the backpropagation algorithm to update the parameters in the policy network. After reaching the preset number of training rounds, synchronize the parameters of the target network with the parameters of the policy network. The training process is repeated continuously until the set maximum number of training rounds is reached, and the trained policy network is output;

[0053] Furthermore, the current state is input into the trained policy network to obtain the Q-values corresponding to all actions. Actions refer to the combined rules of batch operation selection policy and machine allocation policy. The rule combination with the maximum Q-value is selected as the current action, that is, the rule combinations of batch operation selection policy and machine allocation policy in the entire scheduling process are determined. The scheduling process is executed to obtain a scheduling plan, and the scheduling is completed.

[0054] The present invention proposes an intelligent scheduling method driven by the hybrid of knowledge and data under random batch arrivals, constructs a configurable constructive integrated solution framework, aiming to address the problems of uncertain batch arrival times and complex resource allocation in actual production. The configurable constructive integrated solution framework integrates multiple scheduling strategies, including batch operation selection strategy, greedy batch operation splitting strategy, machine allocation strategy, position-based insertion scheduling strategy, and optimization strategy based on critical sub-batch operations. On this basis, the present invention further introduces a double deep Q-network as a reinforcement learning model, and through the continuous interaction between the policy network and the scheduling environment, the policy network is iteratively trained. The double deep Q-network can effectively handle the problem of optimal action selection in a high-dimensional state space and avoid the phenomenon of overestimation of Q-values that may occur in traditional Q-learning. During the training process, the policy network selects an action combination with a higher Q-value according to the current state, thereby promoting the entire scheduling process to evolve towards a better direction. As the training progresses, the policy network can gradually learn a more efficient and flexible scheduling method, significantly improving the scheduling performance and response ability of the hybrid flow shop in a dynamic batch scenario. The configurable constructive integrated solution framework introduced with the double deep Q-network combines the intelligent optimization mechanism driven by the hybrid of knowledge and data, has good generalization and adaptability, is applicable to batch scheduling optimization problems in various complex manufacturing environments, and has the positive effects of realizing efficient, intelligent, and flexible workshop scheduling. Brief Description of the Drawings

[0055] Figure 1 is the implementation flowchart of the present invention;

[0056] Figure 2 is the flowchart of the present invention for training using the double deep Q-network;

[0057] Figure 3 is the data distribution diagram obtained from the experiments of the present invention and SPTCH_ UBLSS, NEHCH_ UBLSS, MCH_ UBLSS, and JCH_ UBLSS;

[0058] Figure 4 is the mean and confidence interval diagram of the data obtained from the experiments of the present invention and SPTCH_ UBLSS, NEHCH_ UBLSS, MCH_ UBLSS, and JCH_ UBLSS;

[0059] Figure 5 This is the batch interaction diagram obtained from the experiments of the present invention with SPTCH_ UBLSS, NEHCH_ UBLSS, MCH_ UBLSS, and JCH_ UBLSS;

[0060] Figure 6 This is the stage interaction diagram obtained from the experiments of the present invention with SPTCH_ UBLSS, NEHCH_ UBLSS, MCH_ UBLSS, and JCH_ UBLSS;

[0061] Figure 7 This is the data distribution diagram obtained from the experiments of the present invention with CVND, GAS2C1, and GAS2C2;

[0062] Figure 8 This is the mean and confidence interval diagram of the data obtained from the experiments of the present invention with CVND, GAS2C1, and GAS2C2. Detailed implementation manners

[0063] As Figure 1 - Figure 2 shown, the intelligent scheduling method driven by the mixture of knowledge and data under the random arrival of batches provided by the present invention is mainly implemented through the following steps.

[0064] Step 1: Define the policy network, state space, and action space, and control the policy network to select an action according to the current state. Each action corresponds to one of the rule combinations in the batch operation selection policy and the machine allocation policy. Specifically, the state space is composed of the general attributes of the overall workshop scheduling state and the specific attributes of each production stage. The general attributes are 9-dimensional, and the specific attribute of each production stage is 18-dimensional. That is, if the workshop contains m production stages, the total dimension of the state space is 9 + 18×m. The action space corresponds to all the rule combinations of the batch operation selection policy and the machine allocation policy. There are eight rules for the batch operation selection policy and five rules for the machine allocation policy. There are 40 rule combinations between the batch operation selection policy and the machine allocation policy, which are used as the output of the policy network. Among them, the rules of the batch operation selection policy and the machine allocation policy are as follows.

[0065] The batch operation selection policy includes:

[0066] Earliest due date first rule, which preferentially selects the batch operation with the earliest due date for scheduling;

[0067] Minimum slack time first rule, which preferentially selects the batch operation with the minimum slack time for scheduling. The slack time is defined as the batch due date minus the remaining processing time;

[0068] Longest processing time first rule, which preferentially selects the batch operation with the longest processing time for scheduling;

[0069] Shortest processing time first rule, which preferentially selects the batch operation with the shortest processing time;

[0070] Shortest remaining processing time first rule, which preferentially selects the job with the shortest remaining processing time. The remaining processing time refers to the total processing time required for the subsequent production stages after the current batch operation is completed;

[0071] Longest remaining processing time first rule, which preferentially schedules the batch operation with the longest remaining processing time;

[0072] Earliest start time first rule, which sorts according to the earliest start time of each batch operation and preferentially schedules the batch operations ranked higher;

[0073] Latest start time first rule, which sorts according to the latest start time of each batch operation and preferentially schedules the batch operations ranked higher.

[0074] Machine allocation strategies include:

[0075] Shortest waiting time first rule, which preferentially selects the machine with the shortest waiting time for the current batch operation;

[0076] Load balancing first rule, which preferentially selects the machine with the smallest current load. The load is defined as the number of batch operations assigned to the machine;

[0077] Earliest idle time first rule, which preferentially selects the machine that will be idle earliest after completing the currently assigned batch operations;

[0078] Longest idle time first rule, when there are idle machines, it preferentially selects the machine with the longest idle time;

[0079] Earliest completion time first, which preferentially selects the machine with the earliest completion time and assigns the current batch operation to the machine that will complete the current batch operation earliest.

[0080] Step 2: Obtain the set of batches to be scheduled at the initial moment and initialize the batches in the set of batches to be scheduled as batch operations. Batch operation refers to the processing operation performed by the batch in each production stage. The batch operations of all production stages are initialized as the set of unscheduled batch operations, and the batch operations of the first production stage are initialized as the set of available batch operations.

[0081] Step 3: Select a batch operation from the available batch operation set using the batch operation selection strategy. If the selected batch operation is a batch operation in the first production stage, which means the batch has not yet been split, the greedy batch operation splitting strategy is executed to split the batch operation into several sub-batch operations. Then, the machine allocation strategy is executed to allocate a machine to the selected batch operation for processing. If the selected batch operation is not a batch operation in the first production stage, the machine allocation strategy is executed using the split result of the batch operation in the first production stage, and the selected batch operation is allocated a machine for processing on the selected machine. The operation process of the greedy batch operation splitting strategy is as follows. Get the batch operation size that needs to be split, and set the maximum number of sub-batch operations that can be split. Set the initial size of each sub-batch operation to zero, and set the remaining batch operation size to be allocated. Enter the loop, select unit 1 from the remaining unallocated batch operation size, and allocate the selected unit 1 to each sub-batch operation in turn, calculate the processing time corresponding to each sub-batch operation, compare the processing time of all sub-batch operations after being allocated to the selected unit 1, select the sub-batch operation with the smallest processing time as the final allocation object of unit 1, update the size of the sub-batch operation, reduce the total amount of remaining unallocated batch operations by 1, and then perform the next loop until the remaining unallocated total amount is 0, completing the batch operation greedy splitting process.

[0082] Step 4: Execute the position-based insertion strategy to adjust the processing order of the batch operations on the selected machine, and the selected machine completes the processing according to the adjusted batch operation order. The position-based insertion strategy includes the following operation process.

[0083] Define the parameters: Represents a batch collection, , Represents the total number of batches, Represents a batch; represents a collection of production stages, , Represents the total quantity of production stage, Represents a certain stage of production; Representative batch Batch operation; Based on the selected batch operation, the selected machine, and the batches that have been scheduled by the machine, determine the position on the machine that can be used as the insertion point, traverse each possible insertion position in turn, and determine whether inserting the selected batch operation into the current position meets the feasibility conditions. The feasibility conditions are shown in Formula 1 and Formula 2.

[0084] Formula 1

[0085] in, Represents the length of the idle time period of the current machine, Represents the batch operation size; Represents the batch operation in the production stage unit processing time; Represents the batch operation in the production stage machine setup time; Formula 1 ensures that the idle time of the selected machine at a certain position can accommodate the minimum processing time required for the selected batch operation. Meeting Formula 1 means that the idle time of the selected machine at a certain position can accommodate the selected batch operation;

[0086] Formula 2

[0087] Where, Represents the latest time of the machine idle time; Represents the maximum number of sub - batch operations; Represents the batch operation maximum sub - batch operation in the production stage end time; Represents a certain production stage transfer time to the next production stage; Represents the batch operation maximum sub - batch operation size; Formula 2 ensures that the latest time point of the idle time of the selected machine at a certain position can meet the processing requirements of the last sub - batch operation of the selected batch operation. Meeting Formula 2 means that moving the selected batch operation to a certain position of the selected machine for processing will not affect the arrangement of subsequent batch operations;

[0088] If the insertion position meets the above feasibility conditions, calculate the scheduling times of the sub-batch operations of the selected batch operation. Select the larger value between the sum of the end time after the previous batch operation is completed in the current production stage and the machine setup time, and the sum of the end time after the previous batch operation is processed and completed in the previous production stage and the transfer time as the start time of the first sub-batch. Select the larger value between the end time of the previous sub-batch operation of the selected batch operation and the sum of the end time of the sub-batch operation in the previous production stage and the transfer time as the start time of the subsequent sub-batch. The end time of each sub-batch operation is the sum of the start time and the processing time of the sub-batch operation. The processing time of the sub-batch operation is the unit processing time multiplied by the size of the sub-batch operation. After calculating the times for all sub-batch operations, determine whether inserting the selected batch operation will cause delays to the subsequent sub-batch operations on the selected machine. If there are no delays, insert the selected batch operation at the current position and stop the position-based insertion strategy. Otherwise, skip the current position and continue to try the next insertion point until all positions are traversed or a feasible insertion position is found.

[0089] Step 5: Remove the batch operations that have completed processing from the unscheduled batch operation set and the available batch operation set. If the removed batch operation is not the batch operation in the last production stage, add the batch operation in the next production stage of the removed batch operation to the available batch operation set.

[0090] Step 6: After all batch operations in all production stages have completed processing, find the critical sub-batch operation from all sub-batch operations. The critical sub-batch operation refers to the sub-batch operation with the least total processing time from the first production stage to the last production stage. Execute the strategy based on the critical sub-batch operation to update the entire processing process. Among them, the strategy based on the critical sub-batch operation includes the following operation process.

[0091] Calculate the reduced size of the critical sub-batch operation in each production stage except the first production stage. Traverse all production stages except the first production stage and calculate the reduced size of the sub-batch operation of the batch operation in the production stage . The calculation method is as follows.

[0092] Among them, the symbol represents rounding down; represents the sub-batch operation of the batch operation at the start time in the production stage ; represents the sub-batch operation of the batch operation ​​In the production stage The end time on;

[0093] Select the minimum value from the reduced sizes of all production stages calculated, remove the selected minimum reduced size from the critical sub-batch operation, merge it into the previous sub-batch operation of the critical sub-batch operation, complete the adjustment of the sizes of the two sub-batch operations, and update the scheduling process of the two sub-batch operations in each production stage.

[0094] Step 7: If there are randomly arriving batches from the start to the current moment, convert the randomly arriving batches into batch operations and add them to the set of unscheduled batch operations, and add the batch operations in the first production stage to the set of available batch operations.

[0095] Step 8: When the set of unscheduled batch operations is empty, the batch operations of all batches are scheduled.

[0096] Step 9: Use the double deep Q-network to train the policy network based on Steps 2 - Step 8, obtain the intelligent scheduling scheme under random batch arrivals using the trained policy network, and terminate the scheduling. Among them, the method of using the double deep Q-network to train the policy network based on Steps 2 - Step 8 is as follows.

[0097] The double deep Q-network adopts a double-network structure, including a policy network and a target network, which are used for action selection and Q-value evaluation respectively. The double deep Q-network combines an experience replay mechanism for stable training. The specific process is as follows:

[0098] Construct a training dataset, which contains several scheduling instances with a fixed number of production stages, set the maximum number of training rounds, and initialize the current training round as the first round. Initialize two neural networks, namely the policy network and the target network, and establish an experience replay buffer. At the beginning of each round of training, randomly select an instance from the training dataset for training. During the training process, judge whether the maximum number of training rounds has been reached. If not, continue to execute this round of training. If so, output the trained policy network. In each round of training, the policy network obtains the current workshop state and checks whether the current scheduling task is completed. If it is completed, increment the current training round by one and start the next round. Otherwise, continue to execute the subsequent operations. The policy network adopts an ε-greedy strategy to select actions, randomly select an action with probability ε; select the current optimal action with the remaining probability 1 - ε. After executing the action, the policy network obtains a new state and the corresponding reward value, and the current state, the selected action, the reward value, and the next state are used as an experience data and stored in the experience replay buffer. The policy network randomly samples a small batch of experience data from the experience replay buffer and calculates the target Q-value y for each experience data. The calculation method is as follows.

[0099]

[0100] where r represents the reward value obtained after the current state s executes the action a; γ is the discount factor, which is used to measure the importance of future rewards; s′ is the next state reached after executing the action; represents the action value estimated by the target network for the state s′ and all actions a′; represents the action value with the largest estimated value among all actions in the state s′;

[0101] Calculate the loss between the Q value obtained by the policy network and the target Q value, and perform gradient calculation on the parameters in the policy network through the backpropagation algorithm to update the parameters in the policy network. After reaching the preset number of training rounds, synchronize the parameters of the target network to the parameters of the policy network. The training process is repeated continuously until the set maximum number of training rounds is reached, and then output the trained policy network;

[0102] The current state is input into the trained policy network to obtain the Q values corresponding to all actions. The actions refer to the rule combinations of batch operation selection strategies and machine allocation strategies. Select the rule combination with the largest Q value as the current action, that is, determine the rule combinations of batch operation selection strategies and machine allocation strategies in the entire scheduling process, execute the scheduling process to obtain a scheduling plan, and complete the scheduling.

[0103] To better prove the effectiveness of the present invention, the present invention will be further described and explained below through experimental analysis of a series of examples of the present invention.

[0104] The test data considered four different numbers of production stages, and the numbers of production stages were taken from the set {3, 5, 8, 10}. The four different numbers of production stages corresponded to four different models. In addition, to make the model applicable to different batch sizes, in each production stage, the number of batches was selected from the set {10, 20, 40, 60, 80, 100}. For each model, a total of 1200 different training instances were generated, with 200 instances corresponding to each batch size. Similarly, for each model, a total of 240 test instances were generated, with 40 instances corresponding to each batch size.

[0105] To simulate the arrival of batches in a dynamic environment, for each instance, the release time is generated as follows: for each batch, there is a 50% probability that its release time is 0, indicating that it can be put into production at the initial moment; otherwise, there is a 50% probability that its release time is randomly generated from the uniform distribution U[1, 100] (seconds). The unit quantity of each batch is selected from the uniform distribution U[50, 100]; the unit processing time is randomly generated from the uniform distribution U[1, 10]; the machine setup time and transfer time are drawn from the uniform distributions U[50, 100] and U[10, 20] (seconds), respectively. In addition, the maximum number of sub-batch operations is set to 5. Regarding the due date of each batch, it is selected from the uniform distribution where a represents the minimum completion time of the largest batch, b represents the sum of the minimum completion times of all batches, and represents the due date density parameter, which is set to 0.5 in this invention. represents the release time of the i-th batch. Among them, the calculation methods of a and b are as follows:

[0106]

[0107] In the double deep Q-network, the adopted deep Q-network consists of four fully connected layers: the input layer contains 256 units for receiving the state vector with the same dimension as the state; followed by two hidden layers, which contain 128 and 64 units respectively, and use the ReLU activation function; the output layer corresponds to 40 actions in the action space and outputs their respective Q-values. The training settings of the policy network are as follows: the maximum number of training rounds is 1000, the discount factor γ is set to 0.99 to balance short-term and long-term rewards; an attenuation factor of 0.8 is applied to the rewards; the Adam optimizer with a learning rate of 1e-4 is used to update the weights of the policy network; the buffer size of the experience replay mechanism is 5000 transfer records, and 32 samples are randomly drawn from it each time for network update. The exploration of the policy network starts from 1.0 and gradually decays with a coefficient of 0.995 until it reaches the minimum value of 0.01. To ensure the stability of the training process, the target network is updated every 100 training rounds.

[0108] To evaluate the performance of each comparative algorithm, this invention uses the average percentage deviation (APD) value as the performance metric for each instance.

[0109] The uniform batch splitting strategy (UBLSS) is introduced into the shortest processing time-based constructive heuristic (SPTCH), the NEH rule-based constructive heuristic (NEHCH), the memory mechanism-based constructive heuristic (MCH), and the Johnson rule-based constructive heuristic (JCH) respectively, to form four comparison algorithms, namely SPTCH_UBLSS, NEHCH_UBLSS, MCH_UBLSS, and JCH_UBLSS. The four comparison algorithms are compared with the knowledge and data hybrid-driven intelligent scheduling method (CCESF_DDQN) under random batch arrival of the present invention. The experimental results obtained are shown in Table 1:

[0110] Table 1 Experimental results obtained by the present invention and constructive heuristic algorithms

[0111]

[0112] Each number of production stages and number of batches in Table 1 constitutes a test problem. For example, 3 production stages and 10 batches will form a test problem 3×10. In the algorithm data in the table, 0.076(0.106), 0.076 represents the APD value obtained by the algorithm, and 0.106 represents the running time of the test problem, with the unit of seconds. It can be seen from the results in Table 1 that the CCESF_DDQN of the present invention has obtained the minimum APD value in all problems, showing the optimal scheduling performance. Through further statistical analysis, it can be concluded that when dealing with problems of different numbers of production stages and batches, the present invention has good stability and low time consumption while ensuring the solution quality, and the comprehensive performance is better than other comparison algorithms.

[0113] Through Figure 3 the violin plot, it can be seen that the data distribution range of the CCESF_DDQN of the present invention is smaller, and the values are mainly concentrated in the lower interval, especially close to zero. In contrast, the data distributions of the other four constructive heuristic algorithms are relatively similar and the range is wider, with greater volatility. In addition, the median of the CCESF_DDQN of the present invention is close to zero, and the interquartile range is smaller, indicating its lower volatility, while the medians of other comparison algorithms are generally higher, close to or exceeding 0.8. The above results show that the CCESF_DDQN of the present invention performs excellently in all problems and has strong stability.

[0114] From Figure 4 it can be seen that the CCESF_DDQN of the present invention is significantly superior to the other four comparison algorithms both in terms of mean and data distribution interval. Therefore, it can be concluded that the CCESF_DDQN of the present invention has more advantages than the constructive heuristic algorithms when solving the knowledge and data hybrid-driven intelligent scheduling problem under random batch arrival.

[0115] In addition, from Table 1, Figure 5 , Figure 6 it can be seen that as the number of production stages and batches increases, the time consumption of all algorithms shows an upward trend. In contrast, the time consumption of SPTCH_UBLSS is the least; however, according to the analysis of its APD value index, its performance is the worst. The CCESF_DDQN of the present invention has the second lowest time consumption among all algorithms, while its APD value performs the best. Generally speaking, the CCESF_DDQN of the present invention demonstrates excellent performance in terms of both effectiveness and efficiency.

[0116] The CCESF_DDQN of the present invention is compared with three metaheuristic algorithms, including the cooperative variable neighborhood descent algorithm (CVND), the single-point crossover genetic algorithm (GAS2C1), and the two-point crossover genetic algorithm (GAS2C2). The termination time is set to t×n×m (milliseconds), where t is taken from 20, 40, and 80 (milliseconds). When t is 80 milliseconds, CVND, GAS2C1, and GAS2C2 have basically converged on most test instances. Tables 2, 3, and 4 respectively show the APD values and the running times of the test problems of all algorithms on 24 different test problems:

[0117] Table 2 Experimental result table of the present invention and CVND

[0118]

[0119] Table 3 Experimental result table of the present invention and GAS2C1

[0120]

[0121] Table 4 Experimental result table of the present invention and GAS2C2

[0122]

[0123] In Table 2, CVND_t20 represents the experiment of CVND under the condition that t takes 20 milliseconds. The same applies to Tables 3 and 4. It can be seen from Table 2 that compared with CVND, the CCESF_DDQN of the present invention achieved the best performance in 23 out of 24 test problems. It can be seen from Table 3 that compared with GAS2C1, the CCESF_DDQN of the present invention achieved the best performance in 22 out of 24 test problems. It can be seen from Table 4 that compared with GAS2C2, the CCESF_DDQN of the present invention achieved the best performance in 17 out of 24 test problems. For the problems with 3 production stages, CVND obtained the minimum APD value in the instances with batch numbers of 10, 40, and 60. When the batch numbers were 40 and 60, CVND had reached the convergence state. In these problems, the CCESF_DDQN of the present invention performed sub-optimally. In the problems with more than 3 production stages, GAS2C1 achieved the best performance in 4 test problems with 5 production stages and batch numbers of 20 or 100, 8 production stages and batch number of 60, and 10 production stages and batch number of 20. The CCESF_DDQN of the present invention was second. Except for the above instances that were better than the CCESF_DDQN of the present invention, the CCESF_DDQN of the present invention performed best in most other instances. Therefore, the comprehensive performance of the present invention is the best.

[0124] Figure 7 As shown, the data distribution of the CCESF_DDQN of the present invention is more concentrated as a whole, indicating that its performance in multiple test problems is more consistent. Although other metaheuristic algorithms as comparison algorithms also show a trend of narrowing distribution when the termination time increases, indicating an improvement in the solution accuracy, their distribution ranges are still significantly wider than that of the CCESF_DDQN. By analyzing Figure 7 the widths of the violin plots obtained by each algorithm, it can be found that the CCESF_DDQN has a higher frequency in the lower APD value range, indicating its strong effectiveness in optimizing problems. At the same time, the median of the CCESF_DDQN is significantly lower than that of other metaheuristic algorithms, further indicating that the CCESF_DDQN of the present invention performs better in most cases. From Figure 8 it can be seen that the CCESF_DDQN of the present invention has the best performance in terms of APD value and the smallest error, indicating that the stability and diversity of the scheduling scheme obtained by the present invention are significantly better than those of other comparison algorithms. Through the above analysis, it can be concluded that the CCESF_DDQN of the present invention performs best in terms of accuracy and stability when solving the intelligent scheduling problem driven by the mixture of knowledge and data under random batch arrival.

Claims

1. A knowledge- and data-driven intelligent scheduling method under random batch arrival, characterized by: The following steps are included: Step 1: Define the policy network, state space, and action space. Select an action based on the current state of the control policy network. Each action corresponds to one of the rule combinations of the batch operation selection strategy and the machine allocation strategy. Step 2: At the initial moment, obtain the set of batches to be scheduled and initialize the batches in the set of batches to be scheduled as batch operations. Batch operations refer to the processing operations performed by batches in each production stage. The batch operations of all production stages are initialized as the set of unscheduled batch operations, and the batch operations of the first production stage are initialized as the set of available batch operations. Step 3: Select a batch operation from the available batch operation set using the batch operation selection strategy. If the selected batch operation is a batch operation in the first production stage, it means that the batch has not been split yet. The greedy batch operation splitting strategy is executed to split the batch operation into several sub-batch operations. Then, the machine allocation strategy is executed to allocate a machine to the selected batch operation and process it on the selected machine. If the selected batch operation is not a batch operation in the first production stage, the split result of the batch operation in the first production stage is used to execute the machine allocation strategy to allocate a machine to the selected batch operation and process it on the selected machine. The greedy batch operation splitting strategy includes the following operation processes: Get the batch operation size that needs to be split, and set the maximum number of sub-batch operations that can be split. Set the initial size of each sub-batch operation to zero, and set the remaining batch operation size to be allocated. Enter the loop, select unit 1 from the remaining unallocated batch operation size, and allocate the selected unit 1 to each sub-batch operation in turn, calculate the processing time corresponding to each sub-batch operation, compare the processing time of all sub-batch operations after being allocated to the selected unit 1, select the sub-batch operation with the smallest processing time as the final allocation object of unit 1, update the size of the sub-batch operation, reduce the total amount of remaining unallocated batch operations by 1, and then perform the next loop until the remaining unallocated amount is 0, completing the batch operation greedy splitting process; Step 4: Execute the position-based insertion strategy to adjust the processing order of the batch operations on the selected machine. The selected machine completes the processing according to the adjusted batch operation order. The position-based insertion strategy includes the following operation processes: Definition parameters: I represents the batch set, I = {1,…,i,…,n}, n represents the total number of batches, i represents a certain batch; J represents the set of production stages, J = {1,…,j,…,m}, m represents the total number of production stages, j represents a certain production stage; O i represents the batch operation of batch i; Based on the selected batch operation, the selected machine, and the batch that has been scheduled by the machine, determine the position on the machine that can be used as an insertion point. Then, traverse each possible insertion position in turn to determine whether inserting the selected batch operation into the current position meets the feasibility conditions. The feasibility conditions are shown in Formula 1 and Formula 2. IdleTime≥p i,j ×T i +Setup i,j ,i∈I,j∈J Formula 1 Among them, IdleTime represents the length of the current machine idle time period, T i Indicates batch operation O i The size of p i,j Indicates batch operation O i Unit processing time in production stage j; Setup i,j Indicates batch operation O i The machine setup time at production stage j; Formula 1 ensures that the idle time of the selected machine at a certain location can accommodate the minimum processing time required for the selected batch operation. If Formula 1 is satisfied, the idle time of the selected machine at a certain location can accommodate the selected batch operation; R≥E i,j-1,L +f i,j-1 +p i,j ×s i,L ,i∈I,j∈J Formula 2 Among them, R represents the latest time of machine idle time; L represents the maximum number of sub-batch operations; E i,j-1,L Indicates batch operation O i The end time of the maximum sub-batch operation L at the production stage j-1; f i,j-1 Indicates the time when a production stage j-1 transfers to the next production stage; s i,L Indicates batch operation O i The maximum sub-batch operation L is the size of the selected batch operation; Formula 2 ensures that the latest time point of the idle time of the selected machine at a certain position can meet the processing requirements of the last sub-batch operation of the selected batch operation. If Formula 2 is satisfied, the selected batch operation will not be affected when it is moved to a certain position of the selected machine for processing. If the insertion position meets the above feasibility conditions, the scheduling time of each sub-batch operation of the selected batch operation is calculated, and the larger value is selected as the start time of the first sub-batch from the sum of the end time of the preceding batch operation after the completion of the current production stage and the machine setup time, and the sum of the end time of the preceding batch operation after the completion of the previous production stage and the transfer time. The larger value is selected as the start time of the subsequent sub-batch from the end time of the preceding sub-batch operation of the selected batch operation and the sum of the end time and the transfer time of the sub-batch operation of the previous production stage. The end time of each sub-batch operation is the sum of the start time and the sub-batch operation processing time, and the sub-batch operation processing time is the unit processing time multiplied by the size of the sub-batch operation. After completing the time calculation of all sub-batch operations, it is determined whether the selected batch operation will cause delays to the subsequent sub-batch operations on the selected machine after insertion. If no delay is caused, the selected batch operation is inserted into the current position and the position-based insertion strategy is stopped. Otherwise, the current position is skipped and the next insertion point is tried until all positions are traversed or a feasible insertion position is found. Step 5: Remove the completed batch operation from the unscheduled batch operation set and the available batch operation set. If the removed batch operation is not the batch operation of the last production stage, add the batch operation of the next production stage of the removed batch operation to the available batch operation set. Step 6: After all batch operations in all production stages are processed, the key sub-batch operation is found from all sub-batch operations. The key sub-batch operation refers to the sub-batch operation with the shortest total processing time from the first production stage to the last production stage. The strategy based on the key sub-batch operation is executed for the key sub-batch operation, and the entire processing process is updated. Step 7: If there are batches that arrive randomly between the start and the current time, convert the batches that arrive randomly into batch operations and add them to the unscheduled batch operation set, and add the batch operations of the first production stage to the available batch operation set; Step 8: When the set of unscheduled batch operations is empty, all batch operations are scheduled; Step 9: Use the dual-depth Q network to train the policy network based on steps 2-8. Use the trained policy network to obtain an intelligent scheduling solution under random batch arrival and terminate the scheduling.

2. The method of intelligent scheduling driven by knowledge and data hybrid under random batch arrival according to claim 1 is characterized in that: The state space is composed of the general attributes of the overall scheduling state of the workshop and the specific attributes of each production stage. The general attributes have 9 dimensions, and the specific attributes of each production stage have 18 dimensions. That is, if the workshop contains m production stages, the total dimension of the state space is 9 + 18 × m. The action space corresponds to all rule combinations of the batch operation selection strategy and the machine allocation strategy. There are eight rules for the batch operation selection strategy and five rules for the machine allocation strategy. There are 40 rule combinations of the batch operation selection strategy and the machine allocation strategy, which are used as the output of the policy network.

3. The method of intelligent scheduling driven by knowledge and data hybrid under random batch arrival according to claim 2 is characterized in that: Batch operation selection strategies include, The earliest delivery date priority rule gives priority to batch operations with the earliest delivery date for scheduling; The minimum slack time priority rule gives priority to scheduling the batch operation with the minimum slack time. The slack time is defined as the batch delivery time minus the remaining processing time. The longest processing time priority rule gives priority to the batch operation with the longest processing time for scheduling; The shortest processing time priority rule gives priority to the batch operation with the shortest processing time; The shortest remaining processing time priority rule gives priority to the job with the shortest remaining processing time. The remaining processing time refers to the total time required for subsequent production stages after the current batch operation is completed. The longest remaining processing time priority rule prioritizes the batch operations with the longest remaining processing time; The earliest start time priority rule sorts batch operations according to their earliest start time, and schedules batch operations with the highest order first. The latest start time priority rule sorts batch operations according to their latest start time, and schedules batch operations with the highest order first.

4. The method of intelligent scheduling driven by knowledge and data hybrid under random batch arrival according to claim 3 is characterized in that: Machine allocation strategies include, The shortest waiting time priority rule gives priority to the machine that has the shortest waiting time for the current batch operation; The load balancing priority rule gives priority to the machine with the smallest current load. The load is defined as the number of batch operations assigned to the machine. The earliest idle time priority rule gives priority to the machine that becomes idle the earliest after completing the currently assigned batch operation; The longest idle time priority rule: when there are idle machines, the machine with the longest idle time is given priority; The earliest completion time is given priority. The machine with the earliest completion time is selected first, and the current batch operation is assigned to the machine with the earliest completion time.

5. The method of intelligent scheduling driven by knowledge and data hybrid under random batch arrival according to claim 4 is characterized in that: The strategy based on key sub-batch operation includes the following operation process, Calculate the reduction size of the key sub-batch operation in each production stage except the first production stage, traverse all production stages except the first production stage, and calculate the batch operation O in production stage j i The reduced size of the sub-batch operation e VSS i,j,e , VSS i,j,e The calculation method is as follows: Among them, the symbol B stands for round down; i,j,e Represents batch operation O i The start time of sub-batch operation e at production stage j-1; E i,j,e-1 Represents batch operation O i The end time of sub-batch operation e-1 at production stage j; The minimum value is selected from the calculated reduction sizes of all production stages, the selected minimum reduction size is removed from the key sub-batch operation, and merged into the previous sub-batch operation of the key sub-batch operation, completing the adjustment of the sizes of the two sub-batch operations, and updating the scheduling process of the two sub-batch operations in each production stage.

6. The method of intelligent scheduling driven by knowledge and data hybrid under random batch arrival according to claim 5, characterized in that: The method for training the policy network based on steps 2-8 using a dual-depth Q network is as follows: The dual-depth Q network adopts a dual network structure, including a policy network and a target network, which are used for action selection and Q value evaluation respectively. The dual-depth Q network combines the experience replay mechanism for stable training. The specific process is to construct a training data set, which contains several scheduling instances with a fixed number of production stages, and set the maximum number of training rounds. At the same time, the current number of training rounds is initialized to the first round, and two neural networks are initialized, namely the policy network and the target network, and an experience replay buffer is established. At the beginning of each round of training, an instance is randomly selected from the training data set for training. During the training process, it is judged whether the maximum number of training rounds has been reached. If not, the training will continue. If it has been reached, the training is output as completed. In each round of training, the policy network obtains the current workshop state and checks whether the current scheduling task is completed. If it is completed, the current training round number is increased by one and the next round is started. Otherwise, the subsequent operations are continued. The policy network adopts the ε-greedy strategy to select actions and randomly selects an action with probability ε; the current optimal action is selected with the remaining probability 1-ε. After executing the action, the policy network obtains the new state and the corresponding reward value. The current state, selected action, reward value and next state are stored as an experience data in the experience replay buffer. The policy network randomly samples a small batch of experience data from the experience replay buffer and calculates the target Q value y for each experience data. The calculation method is as follows, y=r+γ×max a′ Q target (s′,a′) Among them, r represents the reward value obtained after the current state s performs action a; γ is the discount factor used to measure the importance of future rewards; s′ is the next state reached after performing the action; Q target (s′, a′) represents the action value estimated by the target network for state s′ and all actions a′; max a 'Q target (s′, a′) represents the action value with the largest estimated value among all actions in state s′; Calculate the loss between the Q value obtained by the policy network and the target Q value, and perform gradient calculation on the parameters in the policy network through the back-propagation algorithm to update the parameters in the policy network. After reaching the preset number of training rounds, synchronize the parameters of the target network with the parameters of the policy network. The training process is repeated until the set maximum number of training rounds is reached, and the trained policy network is output.

7. The method of intelligent scheduling driven by knowledge and data hybrid under random batch arrival according to claim 6 is characterized in that: The current state is input into the trained policy network to obtain the Q values corresponding to all actions. The action refers to the rule combination of the batch operation selection strategy and the machine allocation strategy. The rule combination with the largest Q value is selected as the current action, which means that the rule combination of the batch operation selection strategy and the machine allocation strategy in the entire scheduling process is determined. The scheduling process is executed to obtain the scheduling plan and complete the scheduling.

Images