Flexible job shop batch scheduling optimization method, device and electronic equipment

Abstract

The application provides a flexible job shop batch scheduling optimization method and device and electronic equipment, and the method comprises the following steps: acquiring multiple workpieces, multiple machines and multiple processes of each workpiece in a workshop; determining multiple machining batches of various workpieces based on a preset batch generation strategy; constructing a flexible job shop allocation scheduling optimization objective function and a constraint condition based on the multiple workpieces, the multiple machines, the multiple processes and the multiple machining batches; solving the flexible job shop allocation scheduling optimization objective function based on a reinforcement learning-artificial bee colony algorithm and the constraint condition to obtain a flexible job shop allocation scheduling scheme. The application improves the efficiency of obtaining a flexible job shop allocation scheduling scheme.

Description

Technical Field

[0001] This invention relates to the field of workshop scheduling technology, specifically to a method, apparatus, and electronic equipment for optimizing batch scheduling in flexible work workshops. Background Technology

[0002] In recent years, with the advancement of technologies such as intelligent optimization algorithms and machine learning, various advanced technologies are gradually being applied to the engineering field, leading to the concept of "intelligent manufacturing." This production model, which combines traditional manufacturing with advanced technologies, is playing an indispensable role in industrial production. In actual manufacturing production, scheduling holds a very important position; a company's operation and production capacity largely depend on its ability to rationally schedule production lines.

[0003] Invention patent CN111985672A proposes a single-piece job shop scheduling method based on multi-agent deep reinforcement learning. It solves the flexible job shop scheduling problem through modeling, state extraction, action selection mechanism design, and reward function design. Invention patent CN112987665A proposes a flow shop scheduling method based on reinforcement learning. This method consists of five stages: processing the processing time matrix, constructing the reward function, building a neural network model, training the model, and applying the model. It also solves the flow shop scheduling problem. Other inventions involving reinforcement learning include the following aspects: generation of the environment space, design of the action set, design of the reward function, and criteria for determining whether an agent has reached its optimal state.

[0004] However, existing workshop scheduling solutions do not consider batch scheduling, the most common and practical type of scheduling problem. Due to various constraints, batch scheduling is very complex and has a huge solution space. Traditional batch scheduling methods often fail to find an ideal solution within a limited time. Therefore, there is an urgent need to provide a method, device, and electronic equipment for optimizing batch scheduling in flexible workshops, so as to quickly and reliably solve batch scheduling schemes in flexible workshops and improve solution efficiency. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, apparatus and electronic equipment for optimizing the batch scheduling of flexible workshops, so as to solve the technical problems of slow solution efficiency and unreliable solution results in the existing flexible workshop batch scheduling optimization scheme.

[0006] To address the aforementioned technical problems, this invention provides a flexible workshop batch scheduling optimization method, comprising:

[0007] Acquire various workpieces, multiple machines, and multiple processes for each workpiece in the workshop;

[0008] Multiple processing batches for various workpieces are determined based on a preset batch generation strategy.

[0009] Based on the various workpieces, the various machines, the various processes, and the various processing batches, construct the objective function and constraints for the flexible workshop allocation and scheduling optimization;

[0010] The objective function for optimizing the allocation and scheduling of the flexible job workshop is solved based on the reinforcement learning-artificial bee colony algorithm and the aforementioned constraints, thereby obtaining the flexible job workshop allocation and scheduling scheme.

[0011] In some possible implementations, the objective function for optimizing the allocation and scheduling of the flexible job shop is:

[0012] F = min(makespan)

[0013]

[0014] In the formula, F is the objective function for optimizing the allocation and scheduling of flexible workshops; min() is the minimum value function; makespan is the completion time; max[] is the maximum value function; O i,j,k For the j-th operation of the k-th processing sub-batch of the i-th workpiece; FT(O i,j,k ) is O i,j,k The processing end time; n is the total number of workpieces; p is the number of operations per workpiece; N i Let be the number of processing sub-batches for the i-th workpiece.

[0015] In some possible implementations, the constraint is:

[0016] N i ={N i ∈[1,sublot UB ]|N i ∈N +}

[0017]

[0018] PT(O i,j,k ) = PT i,j,l ·L i,k ·X(O i,j,k M l )

[0019] l = 1, 2, ..., m

[0020] FT(O i,j,k ) = TT(O i,j,k-1 O i,j,k )+ST(O i,j,k )+PT(O i,j,k )

[0021] FT(O i,j,1 ) = ST(O i,j,1 )+PT(O i,j,1 )

[0022] MT l =FT(O i,j,k )

[0023] ST(O i,j,k )≥max{FT(O i,j-1,k ),MT l}

[0024] ST(O i,1,k )≥max{0,MT l}

[0025]

[0026]

[0027] In the formula, sublot UB N represents the maximum value of the processed sub-batch; + It is a positive integer; TL i L represents the total number of batches processed for the i-th workpiece; i,k PT(O) represents the quantity of the k-th processing sub-batch for the i-th workpiece. i,j,k ) is O i,j,k Processing time; m is the total number of machines; PT i,j,l For the j-th process of the i-th workpiece in M l Processing time on M; l For the l-th machine; X(O) i,j,k M l ) is O i,j,k and M l Decision variables; TT(O i,j,k-1 O i,j,k ) is from O i,j,k-1 To O i,j,k Transportation time; ST(O) i,j,k ) is O i,j,k The start time of the processing; MT l Let be the processing time for the l-th machine.

[0028] In some possible implementations, the step of solving the flexible job shop allocation and scheduling optimization objective function based on the reinforcement learning-artificial bee colony algorithm and the constraints to obtain the flexible job shop allocation and scheduling scheme includes:

[0029] Obtain initial artificial bee colony parameters, which include initial hired bees, initial scout bees, initial observer bees, and initial food sources corresponding to the initial hired bees;

[0030] The initial food source is encoded using a two-segment integer encoding method, generating a chromosome that includes a first segment of chromosome information and a second segment of chromosome information. The first segment of chromosome information is the equipment selection segment representing the machine selected in the current process, and the second segment of chromosome information is the process sorting segment representing the process order.

[0031] A flexible workshop scheduling scheme is determined based on the artificial bee colony algorithm and the aforementioned constraints.

[0032] The flexible job shop allocation and scheduling scheme is determined based on the reinforcement learning algorithm and the flexible job shop scheduling scheme.

[0033] In some possible implementations, the determination of the flexible job shop scheduling scheme based on the artificial bee colony algorithm and the constraints includes:

[0034] Step 1: The initial spectator bees evaluate the quality of the initial food source, determine the optimal food source, and re-divide the artificial bee colony into new hired bees and new scout bees based on the roulette wheel method;

[0035] Step 2: The newly hired bees use a neighborhood search strategy to update the best food source and obtain an updated food source. The new scout bees obtain candidate food sources based on a global search strategy and optimize the best food source based on a local search strategy to obtain an optimized food source. The updated food source, the candidate food source, and the optimized food source constitute a new generation of food sources.

[0036] Step 3: Determine whether the artificial bee colony iteration has ended based on the preset iteration judgment conditions. If it has ended, the new generation of food source is a flexible operation workshop scheduling scheme. If it has not ended, return to step 2 and repeat steps 1-3.

[0037] In some possible implementations, determining the flexible job shop allocation and scheduling scheme based on the reinforcement learning algorithm and the flexible job shop scheduling scheme includes:

[0038] Step 5: Obtain initial reinforcement learning parameters and generate three initial environments based on three preset environment space generation strategies;

[0039] Step 6: Based on the flexible workshop allocation and scheduling scheme and the initial environment, create an action set, an initial Q-value table, and a state set;

[0040] Step 7: Randomly select a state from the set of states, and select the action with the best expected return from the set of actions based on a greedy strategy;

[0041] Step 8: Update the initial Q-value table according to the Q-learning formula, the state, and the action to obtain the updated Q-value table;

[0042] Step 9: Determine if the number of training sessions is less than the preset number of training sessions. If it is less, return to step 7 and repeat steps 7-9. If it is greater than or equal to the preset number of training rounds, determine if the number of training rounds is less than the preset number of training rounds. If it is less, return to step 7 and repeat steps 7-9. If it is greater than or equal to the preset number of training rounds, the flexible workshop allocation and scheduling scheme is obtained.

[0043] In some possible implementations, the Q-learning formula is:

[0044] Q * (s,a)←Q(s,a)+α*[γ(r+max(Q(s',a'))-Q(s,a)]

[0045] Q * (s,a)←Q(s,a)+α*[R t+1 +γ(max(Q(s',a'))-Q(s,a)]

[0046] In the formula, R t+1 γ is the immediate reward / punishment; α is the discount factor; α is the learning rate; s' is the next state; a' is the next action; s is the current state; a is the current action; Q(s,a) is the Q function value of executing the current action in the current state; Q(s',a') is the Q function value of executing the next action in the next state; Q*(s,a) is the updated Q value.

[0047] In some possible implementations, the flexible workshop batch scheduling optimization method further includes:

[0048] The initial artificial bee colony parameters and initial reinforcement learning parameters were determined based on the Taguchi experiment.

[0049] On the other hand, the present invention also provides a flexible workshop batch scheduling optimization device, comprising:

[0050] The data acquisition unit is used to acquire information about various workpieces, multiple machines, and multiple processes of each workpiece in the workshop.

[0051] A batch division unit is used to determine multiple processing batches of various workpieces based on a preset batch generation strategy.

[0052] The objective function construction unit is used to construct the flexible workshop allocation and scheduling optimization objective function and constraints based on the multiple workpieces, the multiple machines, the multiple processes, and the multiple processing batches.

[0053] The function solving unit is used to solve the objective function of the flexible job shop allocation and scheduling optimization based on the reinforcement learning-artificial bee colony algorithm and the constraints, so as to obtain the flexible job shop allocation and scheduling scheme.

[0054] On the other hand, the present invention also provides an electronic device, including a memory and a processor, wherein,

[0055] The memory is used to store programs;

[0056] The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the flexible job shop batch scheduling optimization method described in any of the above possible implementations.

[0057] The beneficial effects of the above implementation method are as follows: The flexible job shop batch scheduling optimization method provided by this invention determines multiple processing batches of various workpieces based on a preset batch generation strategy, and solves the flexible job shop allocation and scheduling optimization objective function based on reinforcement learning-artificial bee colony algorithm and constraints to obtain a flexible job shop allocation and scheduling scheme. Incorporating the allocation of processing batches into the flexible job shop allocation and scheduling scheme improves the accuracy and reliability of the obtained scheme. Furthermore, by using the reinforcement learning-artificial bee colony algorithm to solve the flexible job shop allocation and scheduling optimization objective function, this invention improves the solution efficiency, thereby increasing the efficiency of obtaining the flexible job shop allocation and scheduling scheme. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A schematic flowchart of an embodiment of the flexible workshop batch scheduling optimization method provided by the present invention;

[0060] Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S104;

[0061] Figure 3 A schematic diagram of an embodiment of the two-segment frame number encoding method provided by the present invention;

[0062] Figure 4 For the present invention Figure 2 A schematic diagram of an embodiment of S203;

[0063] Figure 5 A schematic diagram of an embodiment of the local search strategy provided by the present invention;

[0064] Figure 6 A schematic diagram of an embodiment of the neighborhood search strategy provided by the present invention;

[0065] Figure 7 For the present invention Figure 2 A schematic diagram of an embodiment of S204;

[0066] Figure 8 A schematic diagram of an embodiment of the Taguchi experiment calculation results provided by the present invention;

[0067] Figure 9 A schematic diagram of an embodiment of the flexible workshop batch scheduling optimization device provided by the present invention;

[0068] Figure 10 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0070] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0071] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0072] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0073] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0074] This invention provides a method, apparatus, and electronic equipment for optimizing batch scheduling in flexible workshops, which will be described below.

[0075] Figure 1 This is a schematic flowchart of an embodiment of the flexible workshop batch scheduling optimization method provided by the present invention, as shown below. Figure 1 As shown, the batch scheduling optimization method for flexible workshops includes:

[0076] S101. Obtain various workpieces, multiple machines, and multiple processes for each workpiece from the workshop;

[0077] S102. Determine multiple processing batches for various workpieces based on a preset batch generation strategy;

[0078] S103. Construct the objective function and constraints for flexible workshop allocation and scheduling optimization based on multiple workpieces, multiple machines, multiple processes, and multiple processing batches;

[0079] S104. Based on the reinforcement learning-artificial bee colony algorithm and constraints, the objective function for the flexible job shop allocation and scheduling optimization is solved to obtain the flexible job shop allocation and scheduling scheme.

[0080] Compared with existing technologies, the flexible job shop batch scheduling optimization method provided in this invention determines multiple processing batches for various workpieces based on a preset batch generation strategy, and solves the flexible job shop allocation and scheduling optimization objective function based on reinforcement learning-artificial bee colony algorithm and constraints to obtain a flexible job shop allocation and scheduling scheme. By considering the allocation of processing batches in the flexible job shop allocation and scheduling scheme, the accuracy and reliability of the obtained scheme are improved. Furthermore, the use of reinforcement learning-artificial bee colony algorithm to solve the flexible job shop allocation and scheduling optimization objective function improves the solution efficiency, thereby increasing the efficiency of obtaining the flexible job shop allocation and scheduling scheme.

[0081] In some embodiments of the present invention, the batch generation strategy in step S102 may be an equal batch strategy or a consistent batch strategy.

[0082] To simplify the function solving process, the following assumptions are made in some embodiments of the present invention:

[0083] (1) At time t=0, all workpieces can be scheduled;

[0084] (2) For all workpieces, the next process can only be carried out if and only if the current process is completed;

[0085] (3) If the processes in the same sub-processing batch need to be processed on different machines, then the transportation time needs to be considered;

[0086] (4) For all machines, at any given time, only one process can be processed;

[0087] (5) For all machines, once processing begins, it cannot be stopped or interrupted;

[0088] (6) All workpieces have the same priority.

[0089] In some embodiments of the present invention, the objective function for optimizing the allocation and scheduling of flexible workshops is:

[0090] F = min(makespan)

[0091]

[0092] In the formula, F is the objective function for optimizing the allocation and scheduling of flexible workshops; min() is the minimum value function; makespan is the completion time; max[] is the maximum value function; O i,j,k For the j-th operation of the k-th processing sub-batch of the i-th workpiece; FT(O i,j,k ) is O i,j,k The processing end time; n is the total number of workpieces; p is the number of operations per workpiece; N i Let be the number of processing sub-batches for the i-th workpiece.

[0093] The constraints are as follows:

[0094] N i ={N i ∈[1,sublot UB ]|N i ∈N +}

[0095]

[0096] PT(O i,j,k ) = PT i,j,l ·L i,k ·X(O i,j,k M l )

[0097] l = 1, 2, ..., m

[0098] FT(O i,j,k ) = TT(O i,j,k-1 O i,j,k )+ST(O i,j,k )+PT(O i,j,k )

[0099] FT(O i,j,1 ) = ST(O i,j,1 )+PT(O i,j,1 )

[0100] MT l =FT(O i,j,k )

[0101] ST(O i,j,k )≥max{FT(O i,j-1,k ),MT l}

[0102] ST(O i,1,k )≥max{0,MT l}

[0103]

[0104]

[0105] In the formula, sublot UB N represents the maximum value of the processed sub-batch; + It is a positive integer; TL i L represents the total number of batches processed for the i-th workpiece; i,k PT(O) represents the quantity of the k-th processing sub-batch for the i-th workpiece. i,j,k ) is O i,j,k Processing time; m is the total number of machines; PT i,j,l For the j-th process of the i-th workpiece in M l Processing time on M; l For the l-th machine; X(O) i,j,k M l ) is O i,j,k and M l The decision variable, if O i,j,k Arranged in M l Above, then X(O) i,j,k M l If X(O) is 1, otherwise X(O) is 1. i,j,k M l ) is 0; TT(O i,j,k-1 O i,j,k ) is from O i,j,k-1 To Oi,j,k Transportation time; ST(O) i,j,k ) is O i,j,k The start time of the processing; MT l Let be the processing time for the l-th machine.

[0106] In some embodiments of the present invention, such as Figure 2 As shown, step S104 includes:

[0107] S201. Obtain the initial artificial bee colony parameters. The initial artificial bee colony parameters include the initial mercenary bee, the initial scout bee, the initial observer bee, and the initial food source corresponding to the initial mercenary bee.

[0108] S202. The initial food source is encoded using a two-segment integer encoding method to generate a chromosome including a first segment of chromosome information and a second segment of chromosome information. The first segment of chromosome information is the Machines Selection (MS) segment representing the machine selected in the current process, and the second segment of chromosome information is the Operations Sequencing (OS) segment representing the process sequence.

[0109] S203. Determine the flexible workshop scheduling scheme based on the artificial bee colony algorithm and constraints;

[0110] S204. Determine the flexible job shop allocation and scheduling scheme based on reinforcement learning algorithm and flexible job shop scheduling scheme.

[0111] This invention breaks down the flexible job workshop allocation and scheduling scheme into two sub-problems: solving the scheduling scheme and solving the sub-processing batch division scheme. Correspondingly, a two-form integer encoding method is used to encode individuals in the artificial bee colony, generating individual chromosomes including a first chromosome segment and a second chromosome segment. The first chromosome segment represents the equipment selection segment indicating the machine selected for the current process, and the second chromosome segment represents the process ordering segment. Furthermore, the flexible job workshop scheduling scheme is determined based on the artificial bee colony algorithm, and the flexible job workshop allocation and scheduling scheme is determined based on the reinforcement learning algorithm, thus improving the solution efficiency and consequently the efficiency of obtaining the flexible job workshop allocation and scheduling scheme.

[0112] In some embodiments of the present invention, the two-segment integer encoding method is as follows: Figure 3As shown, the first and second chromosome segments follow different encoding rules. In the MS segment, each gene represents the machine selected for the current process. If the current process cannot be processed by a machine, the corresponding machine number will not be entered in the gene, thus ensuring that the generated MS chromosome segment is valid. In the OS segment, the workpiece number needs to be entered in the gene. The total number of each workpiece entered in the OS equals the total number of processes, ensuring that the generated chromosome is valid. Figure 3 The code in the code represents the following meanings: the first process of the first workpiece is processed by the first machine; the second and third processes of the first workpiece are processed by the second machine; the first process of the second workpiece is processed by the second machine; the second process of the second workpiece is processed by the third machine; and the third process of the second workpiece is processed by the first machine. The process order is as follows: the first process of the first workpiece, the first process of the second workpiece, the second process of the first workpiece, the third process of the first workpiece, the second process of the second workpiece, and the third process of the second workpiece.

[0113] In some embodiments of the present invention, such as Figure 4 As shown, step S203 includes:

[0114] S401. Initial observer bees evaluate the quality of the initial food source, determine the optimal food source, and re-divide the artificial bee colony into new hired bees and new scout bees based on the roulette wheel method.

[0115] S402. The new hired bee uses a neighborhood search strategy to update the best food source and obtain an updated food source. The new scout bee obtains a candidate food source based on a global search strategy and optimizes the best food source based on a local search strategy to obtain an optimized food source. The updated food source, the candidate food source, and the optimized food source together form a new generation of food sources.

[0116] S403. Based on the preset iterative judgment conditions, determine whether the artificial bee colony iteration has ended. If it has ended, the new generation of food source is the flexible operation workshop scheduling scheme. If it has not ended, return to step two and repeat steps S401-S402.

[0117] In this embodiment of the invention, by setting the process of updating the best food source for newly hired bees in the artificial bee colony algorithm as a neighborhood search strategy, the ability to further optimize local optima can be improved. By setting the process of obtaining candidate food sources for new scout bees as a global search strategy, the ability to escape local optima can be improved. By setting the optimization of the best food source as a local search strategy, the learning ability of the current best solution can be improved, thereby improving the reliability and rationality of the generated results.

[0118] It should be understood that the iteration judgment condition can be the maximum number of iterations.

[0119] In some embodiments of the present invention, such as Figure 5 As shown, local search strategies can include three strategies: swap strategy, insertion strategy, and mutation strategy.

[0120] In a specific embodiment of the present invention, the exchange strategy is as follows: randomly select two gene positions in the OS segment and exchange their contents. For example... Figure 5 The old food source is coded as 122231J1J2J1J1J2J2. After a swapping strategy (swapping the 9th and 12th positions of the chromosome), the new food source is coded as 122231J1J2J2J1J2J1.

[0121] In a specific embodiment of the present invention, the insertion strategy is as follows: two genes in the OS segment are randomly selected as the start gene and the stop gene. First, the start gene is moved to the position of the stop gene, and then each gene after the start gene is moved forward one position in sequence. Figure 5 The old food source is coded as 122231J1J2J1J1J2J2. After an insertion strategy (using the 9th position as the start gene and the 12th position as the stop gene), the new food source is coded as 122231J1J2J2J1J1J2.

[0122] In a specific embodiment of the present invention, the mutation strategy is as follows: randomly select a gene for mutation. For example... Figure 5 The old food source was coded as 122231J1J2J1J1J2J2. After a mutation strategy (mutating the third position of the chromosome from 2 to 3), the new food source was coded as 123231J1J2J2J1J1J2.

[0123] In some embodiments of the present invention, such as Figure 6 As shown, the neighborhood search strategy is as follows: newly hired bees are randomly divided into two groups. Individuals in the first group are swapped with another individual in their group, and individuals in the second group are swapped with the current best individual. After all individuals in both groups have performed neighborhood searches, a new population will be formed. Both groups perform neighborhood searches simultaneously to efficiently find better food sources.

[0124] In some embodiments of the present invention, such as Figure 7 As shown, step S204 includes:

[0125] S701. Obtain initial reinforcement learning parameters and generate three initial environments based on three preset environment space generation strategies;

[0126] S702. Based on the flexible workshop allocation and scheduling scheme and the initial environment, create an action set, an initial Q-value table and a state set;

[0127] S703. Randomly select a state from the state set, and select the action with the best expected return from the action set based on a greedy strategy;

[0128] S704. Update the initial Q-value table based on the Q-learning formula, state, and action to obtain the updated Q-value table;

[0129] S705. Determine if the number of training sessions is less than the preset number of training sessions. If it is less, return to step S703 and repeat step S703-step S705. If it is greater than or equal to the preset number of training rounds, determine if the number of training rounds is less than the preset number of training rounds. If it is less, return to step S703 and repeat step S703-step S705. If it is greater than or equal to the preset number of training rounds, obtain the flexible workshop allocation and scheduling scheme.

[0130] The embodiments of the present invention generate three initial environments based on three preset environmental space generation strategies, which can improve the richness of the solution space, thereby improving the reliability and rationality of the obtained flexible workshop allocation and scheduling scheme.

[0131] It should be noted that: the state represents the sub-processing batch division scheme, the environment represents the scheme set, the feedback represents the completion time, if the feedback of the next state is greater than the feedback of the current state, the feedback value is 1, if the feedback of the next state is less than the feedback of the current state, the feedback value is -1, and if the feedback of the next state is equal to the feedback of the current state, the feedback value is 0; the action represents the jump between different schemes.

[0132] It should be understood that the Q-value table includes multiple Q-values. A negative Q-value indicates that the action is not conducive to optimization. If the Q-value is 0 or positive, the action is allowed to be performed in this state. The larger the Q-value, the greater the probability of performing the action, which is more beneficial for optimization. By combining the Q-values ​​of the three actions, the state can be determined. If one of them is significantly greater than the other two, the corresponding action is selected, the action is performed by a proxy, and the process transitions to the next state. Based on the above method, the Q-value table is used for state transitions, and finally, the solution with the largest feedback value is found as the batching solution.

[0133] It should also be noted that the three preset environment space generation strategies are Consistent Batch Generation Strategy (CS-RL), Equal Batch Generation Strategy (ES-RL), and Hybrid Generation Strategy (HRL). In the actual solution process, the algorithm is executed three times using each strategy. When using the first two strategies, the environment is randomly generated according to the principles of consistent sub-batches and equal sub-batches, respectively. When using the third strategy, the environment space is generated by combining a certain number of environments that perform well under the first two strategies, and the action space changes from all workpieces sharing the same batching scheme to using different batching schemes.

[0134] In some embodiments of the present invention, the Q-learning formula is:

[0135] Q * (s,a)←Q(s,a)+α*[γ(r+max(Q(s',a'))-Q(s,a)]

[0136] Q * (s,a)←Q(s,a)+α*[R t+1 +γ(max(Q(s',a'))-Q(s,a)]

[0137] In the formula, R t+1 γ is the immediate reward / punishment; α is the discount factor; α is the learning rate; s' is the next state; a' is the next action; s is the current state; a is the current action; Q(s,a) is the Q function value of executing the current action in the current state; Q(s',a') is the Q function value of executing the next action in the next state; Q*(s,a) is the updated Q value.

[0138] Since the efficiency and results of solving the objective function are closely related to the setting of the initial parameters (initial artificial bee colony parameters and initial reinforcement learning parameters), in order to improve the efficiency and reliability of the solution results, in some embodiments of the present invention, the flexible job shop batch scheduling optimization method further includes:

[0139] The initial artificial bee colony parameters and initial reinforcement learning parameters were determined based on the Taguchi experiment.

[0140] The Taguchi experiment included two factors: controllable factors and noise factors. Controllable factors refer to process or design parameters that can be controlled; in the algorithm, these include the number of roulette wheel bets, the probability of being classified as a scout bee, the search probability using the current best food source as a neighborhood, the greediness of reinforcement learning, the learning rate, and the discount factor. Noise factors refer to parameters that are difficult to control during the production process; in the algorithm, this refers to the makespan.

[0141] To analyze experimental numerical control and find the optimal level of controllable factor combinations, a signal-to-noise ratio formula is defined here:

[0142]

[0143] In the formula, S / N is the signal-to-noise ratio; w is the number of runs; y i For the i-th run of makespan.

[0144] The values ​​and levels of controllable factors are shown in Table 1:

[0145] Table 1. Values ​​and Levels of Controllable Factors

[0146]

[0147] Based on the above parameter values, the calculation results are as follows: Figure 8 As shown, according to Figure 8 The parameters are set to the level with the maximum signal-to-noise ratio, namely: 1500 roulette wheel attempts, a probability of being classified as a scout bee of 0.3, a search probability of using the current best food source as a neighborhood of 0.7, a greediness level of reinforcement learning of 0.8, a learning rate of 0.1, and a discount factor of 1.

[0148] To verify the superiority of the reinforcement learning-artificial bee colony algorithm proposed in this embodiment of the invention, it was compared with the traditional artificial bee colony algorithm, tabu search algorithm, reinforcement learning-based genetic algorithm, improved artificial bee colony algorithm, and discrete evolutionary weed optimization algorithm. The algorithm in this embodiment of the invention has high efficiency and high robustness compared with other existing algorithms.

[0149] To better implement the flexible workshop batch scheduling optimization method in this embodiment of the invention, based on the flexible workshop batch scheduling optimization method, correspondingly, as follows: Figure 9 As shown, this embodiment of the invention also provides a flexible workshop batch scheduling optimization device 900, comprising:

[0150] The data acquisition unit 901 is used to acquire information about various workpieces, multiple machines, and multiple processes of each workpiece in the workshop.

[0151] Batch division unit 902 is used to determine multiple processing batches of various workpieces based on a preset batch generation strategy;

[0152] Objective function construction unit 903 is used to construct the objective function and constraints for flexible workshop allocation and scheduling optimization based on multiple workpieces, multiple machines, multiple processes and multiple processing batches;

[0153] The function solving unit 904 is used to solve the objective function of flexible job shop allocation and scheduling optimization based on reinforcement learning-artificial bee colony algorithm and constraints, so as to obtain the flexible job shop allocation and scheduling scheme.

[0154] The flexible workshop batch scheduling optimization device 900 provided in the above embodiments can realize the technical solutions described in the above flexible workshop batch scheduling optimization method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above flexible workshop batch scheduling optimization method embodiments, which will not be repeated here.

[0155] like Figure 10 As shown, the present invention also provides an electronic device 1000. The electronic device 1000 includes a processor 1001, a memory 1002, and a display 1003. Figure 10Only some components of the electronic device 1000 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0156] In some embodiments, memory 1002 may be an internal storage unit of electronic device 1000, such as a hard disk or memory of electronic device 1000. In other embodiments, memory 1002 may also be an external storage device of electronic device 1000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 1000.

[0157] Furthermore, the memory 1002 may include both internal storage units of the electronic device 1000 and external storage devices. The memory 1002 is used to store application software and various types of data installed on the electronic device 1000.

[0158] In some embodiments, processor 1001 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 1002 or process data, such as the flexible workshop batch scheduling optimization method of the present invention.

[0159] In some embodiments, display 1003 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 1003 is used to display information from electronic device 1000 and to display a visual user interface. Components 1001-1003 of electronic device 1000 communicate with each other via a system bus.

[0160] In some embodiments of the present invention, when the processor 1001 executes the flexible job shop batch scheduling optimization program in the memory 1002, the following steps can be implemented:

[0161] Acquire various workpieces, multiple machines, and multiple processes for each workpiece in the workshop;

[0162] Multiple processing batches for various workpieces are determined based on a preset batch generation strategy.

[0163] Based on multiple workpieces, multiple machines, multiple processes, and multiple processing batches, construct the objective function and constraints for the allocation and scheduling optimization of flexible workshops;

[0164] The objective function for optimizing the allocation and scheduling of flexible workshops is solved based on the reinforcement learning-artificial bee colony algorithm and constraints, and a flexible workshop allocation and scheduling scheme is obtained.

[0165] It should be understood that when the processor 1001 executes the flexible job shop batch scheduling optimization program in the memory 1002, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0166] Furthermore, the embodiments of the present invention do not specifically limit the type of the electronic device 1000 mentioned. The electronic device 1000 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 1000 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0167] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0168] The above provides a detailed description of the flexible workshop batch scheduling optimization method, device, and electronic equipment provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for optimizing batch scheduling in a flexible workshop, characterized in that, include: Acquire various workpieces, multiple machines, and multiple processes for each workpiece in the workshop; Multiple processing batches for various workpieces are determined based on a preset batch generation strategy. Based on the various workpieces, the various machines, the various processes, and the various processing batches, construct the objective function and constraints for the flexible workshop allocation and scheduling optimization; The objective function for optimizing the allocation and scheduling of the flexible workshop is solved based on the reinforcement learning-artificial bee colony algorithm and the aforementioned constraints, thereby obtaining the flexible workshop allocation and scheduling scheme. The optimization objective function for the flexible job shop allocation and scheduling is solved based on the reinforcement learning-artificial bee colony algorithm and the constraints to obtain the flexible job shop allocation and scheduling scheme, including: Obtain initial artificial bee colony parameters, which include initial hired bees, initial scout bees, initial observer bees, and initial food sources corresponding to the initial hired bees; The initial food source is encoded using a two-segment integer encoding method, generating a chromosome that includes a first segment of chromosome information and a second segment of chromosome information. The first segment of chromosome information is the equipment selection segment representing the machine selected in the current process, and the second segment of chromosome information is the process sorting segment representing the process order. A flexible workshop scheduling scheme is determined based on the artificial bee colony algorithm and the aforementioned constraints. The flexible job shop allocation and scheduling scheme is determined based on the reinforcement learning algorithm and the aforementioned flexible job shop scheduling scheme. The determination of the flexible job shop scheduling scheme based on the artificial bee colony algorithm and the constraints includes: Step 1: The initial spectator bees evaluate the quality of the initial food source, determine the optimal food source, and re-divide the artificial bee colony into new hired bees and new scout bees based on the roulette wheel method; Step 2: The newly hired bees use a neighborhood search strategy to update the best food source and obtain an updated food source. The new scout bees obtain candidate food sources based on a global search strategy and optimize the best food source based on a local search strategy to obtain an optimized food source. The updated food source, the candidate food source, and the optimized food source constitute a new generation of food sources. Step 3: Determine whether the artificial bee colony iteration has ended based on the preset iteration judgment conditions. If it has ended, the new generation of food source is a flexible operation workshop scheduling scheme. If it has not ended, return to step 2 and repeat step 2-step 3. The determination of the flexible job shop allocation and scheduling scheme based on the reinforcement learning algorithm and the flexible job shop scheduling scheme includes: Step 5: Obtain initial reinforcement learning parameters and generate three initial environments based on three preset environment space generation strategies; Step 6: Based on the flexible workshop allocation and scheduling scheme and the initial environment, create an action set, an initial Q-value table, and a state set; Step 7: Randomly select a state from the set of states, and select the action with the best expected return from the set of actions based on a greedy strategy; Step 8: Update the initial Q-value table according to the Q-learning formula, the state, and the action to obtain the updated Q-value table; Step 9: Determine if the number of training sessions is less than the preset number of training sessions. If it is less, return to step 7 and repeat steps 7-9. If it is greater than or equal to the preset number of training rounds, determine if the number of training rounds is less than the preset number of training rounds. If it is less, return to step 7 and repeat steps 7-9. If it is greater than or equal to the preset number of training rounds, obtain the flexible workshop allocation and scheduling scheme.

2. The flexible workshop batch scheduling optimization method according to claim 1, characterized in that, The objective function for optimizing the allocation and scheduling of the flexible workshop is: In the formula, Assign a scheduling optimization objective function to the flexible workshop; It is a minimum value function; For completion time; It is a function for maximizing the value; For the j-th operation of the k-th processing sub-batch of the i-th workpiece; for The processing end time; n is the total number of workpieces; p is the number of operations per workpiece; N i Let be the number of processing sub-batches for the i-th workpiece.

3. The flexible workshop batch scheduling optimization method according to claim 2, characterized in that, The constraints are as follows: In the formula, This represents the maximum value of the processed sub-batch; It is a positive integer; Let be the total number of batches processed for the i-th workpiece; The quantity of the kth processing sub-batch for the i-th workpiece; for The processing time is ; m is the total number of machines; For the j-th process of the i-th workpiece in M l Processing time on M; l For the first l One machine; for and Decision variables; From arrive The delivery time; for The start time of processing; For the first l Processing time of each machine.

4. The flexible workshop batch scheduling optimization method according to claim 1, characterized in that, The Q-learning formula is: In the formula, For immediate rewards and punishments; γ Discount factor; The learning rate; s’ The next state; a’ For the next action; s This is the current state; a For the current action; Q(s,a) To perform the current action in the current state Q Function value; Q(s',a') The Q-function value for executing the next action in the next state; Q (s,a) For updates Q value.

5. The flexible workshop batch scheduling optimization method according to claim 1, characterized in that, The flexible workshop batch scheduling optimization method also includes: The initial artificial bee colony parameters and initial reinforcement learning parameters were determined based on the Taguchi experiment.

6. A flexible workshop batch scheduling optimization device, characterized in that, The apparatus applicable to the flexible workshop batch scheduling optimization method according to any one of claims 1-5 includes: The data acquisition unit is used to acquire information about various workpieces, multiple machines, and multiple processes of each workpiece in the workshop. A batch division unit is used to determine multiple processing batches of various workpieces based on a preset batch generation strategy. The objective function construction unit is used to construct the flexible workshop allocation and scheduling optimization objective function and constraints based on the multiple workpieces, the multiple machines, the multiple processes, and the multiple processing batches. The function solving unit is used to solve the objective function of the flexible job shop allocation and scheduling optimization based on the reinforcement learning-artificial bee colony algorithm and the constraints, so as to obtain the flexible job shop allocation and scheduling scheme.

7. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the flexible workshop batch scheduling optimization method according to any one of claims 1 to 5.

Images