Q learning-based same-cluster batch scheduling method for three-stage energy-saving hybrid flow shop

Abstract

The invention discloses a same-cluster batch scheduling method for a three-stage energy-saving hybrid flow shop based on Q learning, and relates to the technical field of green scheduling in the manufacturing industry. Aiming at the scheduling problem of the three-stage hybrid flow shop in scenes such as automobile engine manufacturing, the method comprises the following steps: firstly, constructing a scheduling model comprising three stages of processing, surface treatment and final testing; determining a dual-objective function and multiple constraints for minimizing the maximum completion time and the total energy consumption; and then proposing a hybrid ant colony optimization algorithm QHACO solving model based on Q learning, dynamically selecting workpiece and machine strategies through Q learning, and generating a scheduling scheme with low energy consumption and high efficiency in combination with pheromone updating and heuristic search of the ant colony algorithm. The method can effectively cope with complex constraints such as a parallel machine, a batch processor, a limited buffer area and same-cluster group batch, and provides technical support for green transformation of manufacturing enterprises.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of mixed flow shop scheduling, and in particular to a three-stage energy-saving mixed flow shop same cluster batch scheduling method based on Q learning. BACKGROUND

[0002] With the global manufacturing industry transforming towards green and low carbon, energy-saving scheduling has become an important research direction in the field of intelligent manufacturing. The energy-saving mixed flow shop scheduling problem, as a core optimization problem in manufacturing systems, widely exists in process manufacturing scenarios such as steel continuous casting, automobile welding, glass annealing, etc. How to minimize energy consumption while ensuring production efficiency has become a major challenge for manufacturing enterprises.

[0003] In the existing technology, some schemes use multi-objective optimization algorithms to arrange and schedule the production of mixed flow shops. However, the scenarios applied by these schemes are too simple, while the current actual production scenarios are more complex, involving more realistic production scenarios, such as flow shops containing parallel batch machines, where the batch machine can process multiple workpieces at a time without exceeding its own capacity. This makes the problem more complex and belongs to a more complex NP-Hard problem. The currently available algorithms are difficult to find more abundant and reasonable production and processing schemes within a limited search time. In addition, these methods do not effectively handle the distribution characteristics of the solution, and fail to fully utilize the useful information provided by these characteristics to guide the evolution direction of the population, resulting in the production and processing scheme obtained ultimately often failing to meet the requirements of enterprise managers. SUMMARY

[0004] The present application is to solve the above-mentioned deficiencies in the prior art, and proposes a three-stage energy-saving mixed flow shop same cluster batch scheduling method based on Q learning, in order to obtain a time-consuming and energy-saving production scheme for production and processing, thereby improving production and processing efficiency and reducing energy consumption, to reduce the manufacturing cycle and manufacturing cost of manufacturing enterprises.

[0005] To achieve the above-mentioned application purposes, the following technical solutions are adopted: The three-stage energy-saving mixed flow shop same cluster batch scheduling method based on Q learning has the characteristics that it is applied to a three-stage mixed flow shop composed of a processing stage, a surface treatment stage and a final testing stage, and the processing stage, the surface treatment stage and the final testing stage are sequentially recorded as the first stage, the second stage and the third stage, and the serial number of any one of the first stage and the second stage is recorded as ; the processing stage contains parallel machines, the surface treatment stage contains parallel machines, and the final testing stage contains A batch processor, each workpiece in the first and second stages is processed by a corresponding machine according to the machine eligibility constraints, and the batch processor can process all workpieces in the third stage; let the set of workpieces to be processed be , let any workpiece in be denoted as , , and let the total number of workpieces in be , the workpieces are processed in turn through the three stages, and a first finite buffer with a capacity of is set between the first and second stages, and a second finite buffer with a capacity of is set between the second and third stages, the workpieces are batch-processed according to workpiece cluster classes in the third stage, and the total size of each batch does not exceed the capacity of the batch processor ; the parallel machine and the batch processor both include working states with different energy consumptions, standby states, and shutdown states; the three-stage energy-saving hybrid flow shop same-cluster batch scheduling method is executed according to the following steps: Step 1, a double-objective function of the three-stage energy-saving hybrid flow shop same-cluster batch scheduling model is constructed, including: minimizing the maximum completion time and minimizing the total energy consumption , and any objective function in and is denoted as { , }; Step 2, the constraint conditions of the three-stage energy-saving hybrid flow shop same-cluster batch scheduling model are constructed; Step 3, a hybrid ant colony optimization algorithm based on Q-learning is used to solve the three-stage energy-saving hybrid flow shop same-cluster batch scheduling model to generate an optimal scheduling scheme.

[0006] The three-stage energy-saving hybrid flow shop same-cluster batch scheduling method based on Q-learning has the characteristics that in step 1, formula (1) and formula (2) are used to construct and respectively: (1) (2) In formula (1) and formula (2), denotes the current batch; denotes the batch set; denotes the completion time of the current batch ; denotes the current stage number, when =1, it denotes the first stage, and when , it denotes the second stage; Indicates the index of the parallel machine in phase 1 and phase 2; Indicates the first The total number of parallel machines in a stage; Indicates time; Indicates the total duration; This represents the current state of the machine, when... "When" indicates the working status; when When, it indicates standby mode. When the time is specified, it indicates that the machine is in a stopped state; Indicates the first stage The energy consumption of a parallel machine in a given state; Indicates the first Phase 1 Parallel machines at time Is it in If the state is such, then let Otherwise, let ; Indicates the index of the batch processing machine; Indicates in Energy consumption of the batch processing machine under certain conditions; Indicates the first Taiwan batch processing machine at time Is it in If the state is such, then let Otherwise, let 0; This indicates the total number of batch processing machines.

[0007] Furthermore, step 2 includes: Step 2.1: Construct workpiece and machine assignment constraints using equation (3): (3) In equation (3), Indicates workpiece Is it in the first Phase 1 Processing on the machine; if so, then... Otherwise, let 0; Indicates workpiece Is it in the first Phase 1 Processing on a parallel machine; if so, then... Otherwise, let ; Indicates workpiece Is it in the first The current batch of the batch processing machine If so, then let , otherwise, let ; indicates whether the current batch is processed on the i-th batch processor, if yes, let , otherwise, let ; ; Step 2.2, construct the same cluster and batch capacity constraints by using formula (4): (4) In formula (4), indicates whether the workpiece is in the current batch of the i-th batch processor, if yes, let , otherwise, let ; ; indicates the cluster to which the workpiece belongs; indicates the cluster to which the workpiece belongs; indicates that if the condition is met, let , otherwise, let ; indicates the size of the workpiece ; indicates the capacity of the i-th batch processor; indicates whether the current batch contains at least one workpiece, if yes, let , otherwise, let ; ; indicates whether the i-th batch contains at least one workpiece, if yes, let , otherwise, let ; ; Step 2.3, construct the time logic constraints by using formula (5): (5) In formula (5), indicates the start processing time of the workpiece in the first stage; indicates the arrival time of the workpiece in the first stage; indicates the start processing time of the workpiece in the second stage; indicates the completion time of the workpiece in the first stage; indicates the waiting time of the workpiece in the first finite buffer; indicates the workpiece the completion time of the job in the first stage; the completion time of the job in the first stage; the start processing time of the job in the first stage; the completion time of the job in the first stage; the processing time of the job on the first stage parallel machine; the start processing time of the job in the first stage; the completion time of the job in the first stage; the processing time of the job on the first stage parallel machine; the start processing time of the job in the first stage; the completion time of the job in the first stage; the processing time of the job on the first stage parallel machine; the completion time of the job in the first stage; the processing time of the job on the first stage parallel machine; whether the job is processed on the first stage parallel machine in the first stage, if yes, let , otherwise, let ; whether the job is processed on the first stage parallel machine in the first stage, if yes, let , otherwise, let ; a positive number; the start processing time of the job in the third stage; the completion time of the job in the second stage; the waiting time of the job in the second finite buffer; the completion time of the current batch; the processing time of the job on the first batch processor; the completion time of the current batch; the processing time of the job on the first batch processor; Step 2.4, construct the buffer capacity constraint by using formula (6): (6) In formula (6), whether the job is located in the first buffer at time , if yes, let , otherwise, let ; the capacity of buffer 1; whether the job is located in the second buffer at time , if yes, let , otherwise, let ; the capacity of buffer 2. whether the job is located in the second buffer at time , if yes, let , otherwise, let ; the capacity of buffer 2.

[0008] Step 2.5, construct the machine state constraint by using formula (7): (7) In formula (7),​ Indicates the first Phase 1 Parallel machines at time Is it in a state? If so, then let Otherwise, let ; Indicates the first Phase 1 Parallel machines at time Is it in standby mode? If so, then... Otherwise, let ; Indicates the first Phase 1 Parallel machines at time Is the machine in a shutdown state? If so, then... Otherwise, let ; Indicates the first Phase 1 The machine is at all times Is the machine in a shutdown state? If so, then... Otherwise, let ; Indicates the first Taiwan batch processing machine at time Is it in a state? If so, then let Otherwise, let ; Indicates the first Taiwan batch processing machine at time Is it in standby mode? If so, then... Otherwise, let .

[0009] Furthermore, step 3 includes: Step 3.1, Parameter Initialization: Step 3.1.1: Initialize the parameters of the ant colony optimization algorithm: local pheromone weights. Heuristic information weights Local pheromone volatility coefficient ; Step 3.1.2: Initialize Q-learning parameters: learning rate Discount Factor Exploration rate ; Step 3.1.3: Set running parameters: total number of iterations Total number of ants Current iteration number The current number Iterative optimal scheduling scheme set ; Step 3.1.4: Define and initialize the current ant sequence number. = 1; Step 3.2, Define and Initialize the first In the nth iteration Only ants in the objective function Next Phase pheromone matrix , No. In the nth iteration Only ants in the objective function The third stage of the workpiece batch pheromone matrix and the In the nth iteration Only ants in the objective function Batch and workpiece pheromone matrix ; Step 3.3, Define the first In the nth iteration The state of an ant ,action , No. In the nth iteration Only one ant takes action The reward function after And the Q-value update mechanism; Step 3.4: Define and initialize the first... In the nth iteration The taboo list matrix for an ant in three stages, and the initialization of the first stage. The initial workpiece capacity of the finite buffer is set to 0; Step 3.5: Based on the pheromone matrix , 2nd The taboo list matrix for each stage is learned through Q-learning based on the state. ,action Reward function And the Q-value update mechanism, for the first In the nth iteration Only one ant performed the first Stage scheduling to obtain workpieces In the Processing plan in stages; Step 3.6: Based on two pheromone matrices , and the The taboo list matrix for each stage is learned through Q-learning based on the state. ,action Reward function And the Q-value update mechanism, for the first In the nth iteration Only one ant performs the third stage of scheduling to obtain a plan for workpiece batching and processing; Step 3.7, the In the nth iteration After an ant completes the three-stage scheduling, calculate its maximum completion time. Total energy consumption ,Will And the corresponding scheduling scheme is stored in the first... Iterative optimal scheduling scheme set ; Step 3.8, if Then Assign to Then, return to step 3.2; otherwise, it indicates that the final result has been obtained. Iterative optimal scheduling scheme set ; Step 3.9, if Then Assign to If the previous step is not executed, proceed to step 3.1.4 sequentially; otherwise, it indicates that the desired outcome has been achieved. Then, the optimal scheduling schemes of all generations are merged and redundant schemes are removed to obtain the optimal scheduling scheme set, which is the optimal scheme set for the same-cluster batch scheduling of the three-stage energy-saving mixed flow workshop.

[0010] Furthermore, step 3.2 includes: Order No. In the nth iteration Only ants will carry the workpiece Assigned to the Phase 1 When using a parallel machine, the objective function Pheromones expected to be denoted as Thus, the first In the nth iteration Only ants in the objective function Next Phase pheromone matrix Random initialization ; Indicates the initial concentration of pheromones; Order No. In the nth iteration Only ants will carry the workpiece With workpiece When assigned to the same batch of workpieces in stage 3, the objective function Pheromones expected to be denoted as Thus, the first In the nth iteration Only ants in the objective function The third stage of the workpiece batch pheromone matrix Initialize to ; Order No. In the nth iteration Only ants will carry the workpiece Add to current batch When, the objective function Pheromones expected to be denoted as And through equation (8), we obtain the first... In the nth iteration Only ants in the objective function Batch and workpiece pheromone matrix ; (8) In equation (8), For the first In the nth iteration The current batch of ants The set of workpieces already included in the list. for The number of workpieces in the process.

[0011] 6. The three-stage energy-saving hybrid flow workshop batch scheduling method based on Q-learning according to claim 5, characterized in that step 3.3 includes: Definition of the first In the nth iteration The state of an ant ,in, ∈{1,2,3} represents the current stage of processing. Indicates the first In the nth iteration Only ants at the current stage The number of machines that are blocked. ∈[0,100] represents the first... In the nth iteration The first ant Utilization of a limited buffer; Definition of the first In the nth iteration The movement of an ant ,in, ∈{1,2,3,4} represents the first... In the nth iteration The workpiece selection strategy for ants: when This indicates that the workpiece with the shortest arrival time will be selected first. This indicates that the workpiece with the shortest processing time will be selected first. This indicates that the workpiece with the longest processing time will be selected first. This indicates that the largest workpiece will be selected first. ∈{1,2,3} represents the first... In the nth iteration Machine selection strategy for ants: when This indicates selecting the machine with the lowest energy consumption per unit time. This indicates that the machine with the lowest current load is selected. This indicates that the earliest available machine will be selected; Construct the first equation using equation (9) In the nth iteration Only one ant takes action The reward function after : (9) In equation (9), Indicates the first The two target preference weights of the ant, and ; Indicates the first In the nth iteration The maximum completion time improvement rate for a single ant is obtained through equation (10); Indicates the first In the nth iteration The total energy consumption improvement rate of a single ant is obtained through equation (11); (10) In equation (10), Indicates the first In the nth iteration The maximum completion time for a single ant; Indicates the first In the nth iteration The maximum completion time for a single ant; (11) In equation (11), Indicates the first In the nth iteration The total energy consumption of an ant Indicates the first In the nth iteration The total energy consumption of a single ant; The first is constructed by defining equation (12). In the nth iteration The Q-value update mechanism of an ant: (12) In equation (12), Indicates the first In the nth iteration Only ants are in the state Take action below Updated state action function value; Indicates the first In the nth iteration Only ants are in the state Take action below State action function value; Indicates the first In the nth iteration Only one ant takes action The state after; Indicates the first In the nth iteration Only ants are in the state The optimal action to take; Indicates the first In the nth iteration Only ants are in the state Choose the optimal action The expected reward.

[0012] Furthermore, step 3.4 includes: Define and initialize the first In the nth iteration The taboo list matrix for ants in Phase 1 Define and initialize the first In the nth iteration Ant's taboo list matrix in phase 2 Define and initialize the first In the nth iteration Ant's Taboo List Matrix in Stage 3 , respectively used for the In the nth iteration Each ant records whether there are any workpieces to be processed in stages 1, 2, and 3. If an element in the taboo list matrix is ​​not 0, it means that the workpiece at the corresponding position has not been processed. The workpiece with non-zero elements constitutes the first... In the nth iteration A collection of unprocessed workpieces from an ant ;Depend on The workpiece with non-zero elements constitutes the first... In the nth iteration A collection of unprocessed workpieces from an ant ;Depend on The workpiece with non-zero elements constitutes the first... In the nth iteration A collection of unprocessed workpieces from an ant .

[0013] 8. A three-stage energy-saving hybrid flow workshop batch scheduling method based on Q-learning according to claim 7, characterized in that step 3.5 includes: Step 3.5.1, according to the first In the nth iteration The state of an ant , No. In the nth iteration Only ants determine the workpiece selection strategy Machine selection strategy Then, the determined workpiece selection strategy is used. For the In the nth iteration Only ants in the first The set of unprocessed workpieces in the stage After filtering, the first one was obtained. In the nth iteration Only ants in the first Candidate workpieces for the stage ; Utilizing the determined machine selection strategy For those with Set of idle parallel machines qualified for processing After filtering, the first one was obtained. In the nth iteration Only ants in the first The first stage of selection Parallel machine; Step 3.5.2: Construct the first equation using equations (13) and (14). In the nth iteration Only ants in the first First workpiece candidate list for the stage Second workpiece candidate list ; (13) (14) In equations (13) and (14), Indicates the first In the nth iteration Workpieces recorded by only ants In the Can the stage be processed? If so, then let... Otherwise, let ; Indicates the first In the nth iteration Only ants determine the workpiece Is it possible in the first In the nth iteration Only ants in the first The first stage of selection Processing on a parallel machine; if so, then... Otherwise, let ; Indicates the first In the nth iteration Workpieces recorded by only ants In the The arrival time of the stage; Indicates the first In the nth iteration Only ants in the first The first stage of selection The current completion time of the parallel machines; Step 3.5.3, prioritize from Select the workpiece, if Then from Choose from; and calculate using equation (15). or medium workpiece by In the nth iteration Only ants in the first The first stage of selection Probability of parallel machining Therefore, the workpiece with the highest processing probability is selected as the first... In the nth iteration Only ants in the first Stage of workpiece processing ; (15) In equation (15), Indicates the first Only ants targeting the objective function Preference weights; Indicates the first In the nth iteration The workpiece selection strategy of an ant for Select workpiece at time In the In the nth iteration Only ants in the first The first stage of selection The heuristic information processed on the parallel machine is obtained through equation (16); Indicates the first In the nth iteration Only ants will carry the workpiece Assigned to in In the nth iteration Only ants in the first The first stage of selection When using a parallel machine, the objective function Pheromones expected; (16) In equation (16), Indicates the first In the nth iteration Workpieces recorded by only ants In the In the nth iteration Only ants in the first The first stage of selection Processing time on a parallel machine; Indicates the first In the nth iteration Workpieces recorded by only ants Dimensions; Step 3.5.4: Obtain the first result through equation (17). In the nth iteration Only ants will carry the workpiece Assigned to the In the nth iteration Only ants in the first The first stage of selection When using a parallel machine, the objective function Updated pheromone expectations : (17) Step 3.5.5, when When =1, if the remaining capacity of the first finite buffer is greater than that of the second finite buffer... In the nth iteration Workpieces recorded by only ants If the dimensions are such that the workpiece is... Put it into the first finite buffer, and middle Set the corresponding element to 0, and middle The corresponding element is set to The workpiece number, and release the first In the nth iteration Only ants in the first The first stage of selection A parallel machine; otherwise, it indicates the first parallel machine. In the nth iteration The ant selected in the first stage The parallel machine enters a blocked state and waits for the remaining capacity of the first finite buffer to be greater than or equal to... After determining the dimensions, return to step 3.5.5 to execute; when When =2, if the remaining capacity of the second finite buffer is greater than or equal to the first... In the nth iteration Workpieces recorded by only ants If the dimensions are such that the workpiece is... Put it into the second finite buffer, and middle Set the corresponding element to 0, and middle The corresponding element is set to The workpiece number, and release the first In the nth iteration Only ants in the first The first stage of selection A parallel machine; otherwise, it indicates the first parallel machine. In the nth iteration Only ants in the first The first stage of selection The parallel machine enters a blocked state and waits for the remaining capacity of the first finite buffer to be greater than or equal to... After determining the dimensions, return to step 3.5.5 to execute; Step 3.5.6, if If there are any non-zero elements in the workpiece, return to step 3.5.1 and continue executing step 2. Phase scheduling.

[0014] Furthermore, step 3.6 includes: Step 3.6.1, based on the first In the nth iteration The state of an ant Q-learning dynamically selects the workpiece selection strategy. Machine selection strategy ; In the workpiece set In the middle, through workpiece selection strategy Candidate workpieces were obtained through screening. ; In idle batch processing machine collection In the middle, through The first was obtained by filtering. In the nth iteration Only ants in the first The first stage of selection Taiwan batch processing machine; Build batch , to the workpiece As a batch The first workpiece, and the batch The cluster class is set as the workpiece. Clusters; Step 3.6.2, for batches The first equation is constructed using equations (18) and (19). In the nth iteration The first list of artifact candidates for ants Second workpiece candidate list ; (18) (19) In equations (18) and (19), Indicates the first In the nth iteration Workpieces recorded by only ants Can it be processed in stage 3? If so, then let Otherwise, let ; Indicates the first In the nth iteration Workpieces recorded by only ants Arrival time in stage 3; Indicates the first In the nth iteration Batch recorded by ants Start processing time; Indicates the first In the nth iteration Only ants in the first The first stage of selection The capacity of the batch processing machine; Indicates the first In the nth iteration Batch recorded by ants The current total size; Indicates the first In the nth iteration Workpieces recorded by only ants Clusters; Step 3.6.3, prioritize from Select the workpiece, if Then from Select from the options; calculate the batch addition of candidate workpieces using equation (20). Given the probability, select the workpiece with the highest probability as the first... In the nth iteration Add only ants to the batch workpiece ; (20) In equation (20), Indicates the first In the nth iteration Only ants will carry the workpiece Add to batch When, the objective function Pheromones expected; Indicates the first In the nth iteration The heuristic information of the ant in the third stage, and obtained from equation (21), Indicates the first In the nth iteration Only ants will carry the workpiece Add to batch When, the objective function Pheromones expected; (twenty one) In equation (21), Indicates the first In the nth iteration Workpieces recorded by only ants Add to batch Preparation time; Indicates the first In the nth iteration Batch recorded by ants In the In the nth iteration Only ants in the first The first stage of selection Processing time on a batch processing machine; express In the In the nth iteration Only ants in the first The first stage of selection Processing time on a batch processing machine; Step 3.6.4, Settings The local pheromone in the third stage is updated using equation (22). (twenty two) In equation (22), Indicates the first In the nth iteration Only ants will carry the workpiece With workpiece When allocated to the same batch, the objective function Updated pheromone expectations; Step 3.6.5, if There are workpieces in Meet the conditions and If so, return to step 3.6.2 and continue to batch. Add a workpiece; otherwise, proceed to step 3.6.6. Step 3.6.6: Batch Put into the first In the nth iteration Only ants in the first The first stage of selection Processed by batch processing machine; if If there is an item with element 1, return to step 3.6.1; otherwise, proceed to step 3.7.

[0015] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing the three-stage energy-saving hybrid flow workshop co-cluster batch scheduling method, and the processor is configured to execute the program stored in the memory.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention addresses the urgent need for synergistic optimization of green and efficient processes in modern manufacturing. Based on the complex actual production conditions of hybrid assembly line workshops (such as multi-stage heterogeneous equipment, machine qualification constraints, limited buffer zones, and clustered batch processing requirements), comprehensive constraints are established, and a dual-objective optimization model is constructed with the goal of minimizing maximum completion time and total energy consumption. Furthermore, an innovative hybrid ant colony optimization algorithm, QHACO, based on Q-learning, is proposed to solve this complex model, outputting a set of Pareto optimal production solutions that demonstrate excellent performance in both efficiency and energy consumption. This solution uses intelligent methods to replace or assist scheduling personnel in decision-making, thereby significantly reducing energy consumption during the production process while ensuring production efficiency, achieving green and efficient operation of the workshop.

[0017] 2. The QHACO proposed in this invention has significant advantages over traditional optimization algorithms. This algorithm, through a deep fusion mechanism, uses Q-learning as a high-level intelligent decision-maker, rather than a simple parameter adjustment tool, enabling it to dynamically select the optimal workpiece allocation and machine selection strategy based on the real-time status during the scheduling process. This method effectively overcomes the limitations of standard ant colony algorithms in solving such complex problems, namely high randomness and susceptibility to local optima. QHACO possesses both excellent global exploration capabilities and efficient local exploitation capabilities, exhibiting faster convergence speed and superior solution quality. It is particularly suitable for solving large-scale, strongly constrained three-stage energy-efficient hybrid flow shop cluster batch scheduling problems, thus providing a more effective scheduling solution for practical process manufacturing industries such as automotive engine manufacturing. Attached Figure Description

[0018] Figure 1 This is a sequence diagram of the method of the present invention; Figure 2 This is a flowchart of the automotive engine manufacturing process; Figure 3 This is a flowchart of the hybrid ant colony optimization algorithm based on Q-learning of this invention. Detailed Implementation

[0019] In this embodiment, a three-stage energy-saving mixed flow workshop co-cluster batch scheduling method based on Q-learning is described, referring to... Figure 2 This allows us to capture the precision manufacturing process of core components such as cylinder blocks, pistons, and valves in the actual production of automotive engines. First, these components undergo precision machining such as milling, drilling, and boring on multiple high-precision CNC machine tools in the machining workshop. Due to differences in the material and geometry of different components, each machine tool can only process specific types of components, and the finished components need to be temporarily stored in a limited-capacity intermediate warehouse awaiting the next process. Subsequently, the components are sent to the surface treatment workshop for processes such as coating, plating, or heat treatment, again subject to equipment specialization and storage space constraints. Finally, in the quality inspection workshop, the surface-treated components are grouped and batch-tested according to the similarity of the test items. Components in the same batch must belong to the same workpiece type, each component must have a certain size, and the batch size cannot exceed the capacity limit of the testing equipment. Specifically, for example... Figure 1 As shown, for this production scheduling, the three-stage energy-saving hybrid flow shop cluster batch scheduling method in this embodiment studies the three-stage energy-saving hybrid flow shop cluster batch scheduling problem. This problem comprehensively considers multiple complex features such as parallel machine processing, batch processing machine batching, finite buffer constraints, and cluster batching rules, aiming to simultaneously minimize the maximum completion time and total energy consumption. The method is executed according to the following steps: Step 1: Construct the dual objective function of the three-stage energy-saving hybrid flow workshop batch scheduling model, including: minimizing the maximum completion time as described in equation (1). and the minimum total energy consumption as shown in equation (2) and will and Any objective function in the given information is denoted as . { , }: (1) (2) In equations (1) and (2), Indicates the current batch; Represents a batch set; Indicates the current batch Completion time; Indicates the current stage number, when When =1, it indicates the first stage. "Time" indicates the second stage; Indicates the index of the parallel machine in phase 1 and phase 2; Indicates the first The total number of parallel machines in a stage; Indicates time; Indicates the total duration; This represents the current state of the machine, when... "When" indicates the working status; when When, it indicates standby mode. When the time is specified, it indicates that the machine is in a stopped state; Indicates the first stage The energy consumption of a parallel machine in a given state; Indicates the first Phase 1 Parallel machines at time Is it in If the state is such, then let Otherwise, let ; Indicates the index of the batch processing machine; Indicates in Energy consumption of the batch processing machine under certain conditions; Indicates the first Taiwan batch processing machine at time Is it in If the state is such, then let Otherwise, let 0; This indicates the total number of batch processing machines.

[0020] Step 2: Constructing the constraints for the three-stage energy-saving hybrid flow workshop cluster batch scheduling model: Step 2.1: Construct the four assignment constraints between the workpiece and the machine using equation (3): (3) In equation (3), Indicates workpiece Is it in the first Phase 1 Processing on the machine; if so, then... Otherwise, let 0; Indicates workpiece Is it in the first Phase 1 Processing on a parallel machine; if so, then... Otherwise, let ; Indicates workpiece Is it in the first The current batch of the batch processing machine If so, then let Otherwise, let ; Indicates the current batch Is it in the first Processed on a batch processing machine in Taiwan; if so, then... Otherwise, let The first allocation constraint is used to guarantee the workpiece. In the The first stage must be processed on one and only one machine; the second assignment constraint is used to ensure that the workpiece... In the The stage can only be completed in the first stage when processing qualifications are available. The machining is performed on a parallel machine; the third allocation constraint is used to guarantee the workpiece. It must be assigned to one batch only; the fourth assignment constraint is used to guarantee that the workpiece It can only be assigned to a batch that is actually being processed on a batch processor.

[0021] Step 2.2: Construct five constraints on cluster and batch processing capacity using equation (4): (4) In equation (4), Indicates workpiece Is it in the first The current batch of the batch processing machine If so, then let Otherwise, let ; Indicates workpiece The cluster to which it belongs; Indicates workpiece The cluster to which it belongs; Indicates if the condition is met Then let Otherwise, let ; Indicates workpiece Dimensions; Indicates the first The capacity of the batch processing machine; Indicates the current batch Does it contain at least one workpiece? If so, then let... Otherwise, let ; Indicates the first If a batch contains at least one workpiece, then let... Otherwise, let The first constraint is used to ensure that in the batch In the first constraint, only workpieces belonging to the same cluster can exist simultaneously; the second constraint is used to ensure batch processing. The total dimensions of the workpieces must not exceed the machine capacity for processing that batch; the third constraint is used to ensure the workpieces... It can only be assigned to the batch that is being used; the fourth constraint is to ensure that each batch processor can be assigned to at most one batch; the fifth constraint is to ensure that the total size of the workpieces in each batch cannot exceed the capacity of the machine that processes that batch.

[0022] Step 2.3: Construct 7 time logic constraints using equation (5): (5) In equation (5), Indicates workpiece The start time of processing in stage 1; Indicates the first stage of the workpiece Arrival time; Indicates workpiece The start time of processing in stage 2; Indicates workpiece Completion time of Phase 1; Indicates workpiece Waiting time in the first finite buffer; Indicates workpiece In the Completion time of each stage; Indicates workpiece In the Phase 1 Processing time on a parallel machine; Indicates workpiece In the The start time of the processing stage; Indicates workpiece In the Phase 1 Processing time on a parallel machine; Indicates workpiece Is it in the first Phase 1 Processing on a parallel machine; if so, then... Otherwise, let ; Represents positive numbers; Indicates workpiece The start time of processing in stage 3; Indicates workpiece Completion time of Phase 2; Indicates workpiece Waiting time in the second finite buffer; Indicates the current batch Completion time; Indicates workpiece In the The processing time on the batch processing machine; where the first time logic constraint is used to guarantee the workpiece The start time of the first stage cannot be earlier than its arrival time; the second time logic constraint is used to ensure the workpiece... The start time of stage 2 must be later than its completion time in stage 1 plus the waiting time in the first finite buffer; a third time logic constraint is used to guarantee the workpiece. The completion time is equal to the start time plus the time on the [number]th day. Phase 1 Processing time on a parallel machine; the fourth timing logic constraint is used to ensure that the processing time on the first parallel machine is... Phase 1 Workpiece on a parallel machine The start time must be later than the workpiece. The completion time; the 5th time logic constraint is used to guarantee the completion time of the workpiece. The start time of phase 3 equals the completion time of phase 2 plus the waiting time in the second finite buffer; the sixth time logic constraint is for the current batch. The start time is the current batch The maximum start time of all workpieces in the batch; the 7th time logic constraint is used to guarantee the current batch The completion time is the start time plus the current batch. The longest processing time for any workpiece.

[0023] Step 2.4: Construct the four capacity constraints of the buffer using equation (6): (6) In equation (6), Indicates workpiece At any moment Is it located in the first buffer? If so, then let... Otherwise, let ; This indicates the capacity of buffer 1; Indicates workpiece At any moment Is it located in the second buffer? If so, then let... Otherwise, let ; This represents the capacity of buffer 2. The first capacity constraint is used for the workpiece. The waiting time in the first finite buffer is the difference between the start time of the second phase and the completion time of the first phase; the second capacity constraint is used to ensure that at time [time value missing]... The total size of the workpieces within the first finite buffer cannot exceed its capacity limit; the third capacity constraint is used to ensure that the workpieces... The waiting time in the second finite buffer is the difference between the start time of the third phase and the completion time of the second phase; the fourth capacity constraint is used to ensure that at time [time value missing]... The total size of the workpieces in the second finite buffer cannot exceed its capacity limit.

[0024] Step 2.5: Construct the five state constraints of the machine using equation (7): (7) In equation (7), Indicates the first Phase 1 Parallel machines at time Is it in a state? If so, then let Otherwise, let ; Indicates the first Phase 1 Parallel machines at time Is it in standby mode? If so, then... Otherwise, let ; Indicates the first Phase 1 Parallel machines at time Is the machine in a shutdown state? If so, then... Otherwise, let ; Indicates the first Phase 1 The machine is at all times Is the machine in a shutdown state? If so, then... Otherwise, let ; Indicates the first Taiwan batch processing machine at time Is it in a state? If so, then let Otherwise, let ; Indicates the first Taiwan batch processing machine at time Is it in standby mode? If so, then... Otherwise, let The first state constraint is used to guarantee the... Phase 1 Parallel machines at time It can only be in one of the following states: working, standby, or off; the second state constraint is used to guarantee the first state. Phase 1 Parallel machines at time and time It cannot be in standby mode; the third state constraint is used for the first... Phase 1 Once the parallel machine is shut down, it will not be restarted; the fourth state constraint is used to guarantee the... Taiwan batch processing machine at time It can only be in one of the following states: working, standby, or off; the fifth state constraint is used to guarantee the... Taiwan batch processing machine at time and time It cannot be in standby mode.

[0025] Step 3: Use a Q-learning-based hybrid ant colony optimization algorithm to solve the three-stage energy-saving hybrid flow workshop cluster batch scheduling model, and generate the optimal scheduling scheme. Specifically, as follows: Figure 3 As shown: Step 3.1, Parameter Initialization: Step 3.1.1: Initialize the parameters of the ant colony optimization algorithm: local pheromone weights. Heuristic information weights Local pheromone volatility coefficient ; Step 3.1.2: Initialize Q-learning parameters: learning rate Discount Factor Exploration rate ; Step 3.1.3: Set running parameters: total number of iterations Total number of ants Current iteration number The current number Iterative optimal scheduling scheme set ; Step 3.1.4, Current Ant Sequence Number = 1.

[0026] Step 3.2, Define and initialize the core matrix: Step 3.2.1, let the first... In the nth iteration Only ants will carry the workpiece Assigned to the Phase 1 When using a parallel machine, the objective function Pheromones expected to be denoted as Thus, the first In the nth iteration Only ants in the objective function Next Phase pheromone matrix Random initialization ; Indicates the initial concentration of pheromones; Order No. In the nth iteration Only ants will carry the workpiece With workpiece When assigned to the same batch of workpieces in stage 3, the objective function Pheromones expected to be denoted as Thus, the first In the nth iteration Only ants in the objective function The third stage of the workpiece batch pheromone matrix Initialize to .

[0027] Order No. In the nth iteration Only ants will carry the workpiece Add to current batch When, the objective function Pheromones expected to be denoted as And through equation (8), we obtain the first... In the nth iteration Only ants in the objective function Batch and workpiece pheromone matrix ; (8) In equation (8), For the first In the nth iteration The current batch of ants The set of workpieces already included in the list. for The number of workpieces in the process.

[0028] Step 3.2.2, Define the first In the nth iteration The state of an ant ,in, ∈{1,2,3} represents the current stage of processing. Indicates the first In the nth iteration Only ants at the current stage The number of machines that are blocked. ∈[0,100] represents the first... In the nth iteration The first ant Utilization of a limited buffer; Definition of the first In the nth iteration The movement of an ant ,in, ∈{1,2,3,4} represents the first... In the nth iteration The workpiece selection strategy for ants: when This indicates that the workpiece with the shortest arrival time will be selected first. This indicates that the workpiece with the shortest processing time will be selected first. This indicates that the workpiece with the longest processing time will be selected first. This indicates that the largest workpiece will be selected first. ∈{1,2,3} represents the first... In the nth iteration Machine selection strategy for ants: when This indicates selecting the machine with the lowest energy consumption per unit time. This indicates that the machine with the lowest current load is selected. This indicates that the earliest available machine will be selected.

[0029] Construct the first equation using equation (9) In the nth iteration Only one ant takes action The reward function after : (9) In equation (9), Indicates the first The two target preference weights of the ant, and ; Indicates the first In the nth iteration The maximum completion time improvement rate for a single ant is obtained through equation (10); Indicates the first In the nth iteration The total energy consumption improvement rate of a single ant is obtained through equation (11); (10) In equation (10), Indicates the first In the nth iteration The maximum completion time for a single ant; Indicates the first In the nth iteration The maximum completion time for a single ant; (11) In equation (11), Indicates the first In the nth iteration The total energy consumption of an ant Indicates the first In the nth iteration The total energy consumption of a single ant.

[0030] The first is constructed by defining equation (12). In the nth iteration The Q-value update mechanism of an ant: (12) In equation (12), Indicates the first In the nth iteration Only ants are in the state Take action below Updated state action function value; Indicates the first In the nth iteration Only ants are in the state Take action below State action function value; Indicates the first In the nth iteration Only one ant takes action The state after; Indicates the first In the nth iteration Only ants are in the state The optimal action that can be taken; Indicates the first In the nth iteration Only ants are in the state Choose the optimal action The expected reward.

[0031] Step 3.3: Initialize scheduling pre-parameters: Step 3.3.1: Define and initialize the first... In the nth iteration The taboo list matrix for ants in Phase 1 Define and initialize the first In the nth iteration Ant's taboo list matrix in phase 2 Define and initialize the first In the nth iteration Ant's Taboo List Matrix in Stage 3 , respectively used for the In the nth iteration Each ant records whether there are any workpieces to be processed in stages 1, 2, and 3. If an element in the taboo list matrix is ​​not 0, it means that the workpiece at the corresponding position has not been processed. The workpiece with non-zero elements constitutes the first... In the nth iteration A collection of unprocessed workpieces from an ant ;Depend on The workpiece with non-zero elements constitutes the first... In the nth iteration A collection of unprocessed workpieces from an ant ;Depend on The workpiece with non-zero elements constitutes the first... In the nth iteration A collection of unprocessed workpieces from an ant .

[0032] Step 3.3.2, Initialize the first The initial workpiece capacity of the finite buffer is set to 0; Step 3.4, the In the nth iteration Only one ant performed the first Phase scheduling: Step 3.4.1, according to the first In the nth iteration The state of an ant , No. In the nth iteration Only ants determine the workpiece selection strategy Machine selection strategy Then, the determined workpiece selection strategy is used. For the In the nth iteration Only ants in the first The set of unprocessed workpieces in the stage After filtering, the first one was obtained. In the nth iteration Only ants in the first Candidate workpieces for the stage ; Utilizing the determined machine selection strategy For those with Set of idle parallel machines qualified for processing After filtering, the first one was obtained. In the nth iteration Only ants in the first The first stage of selection Parallel machine.

[0033] Step 3.4.2: Construct the first equation using equations (13) and (14). In the nth iteration Only ants in the first First workpiece candidate list for the stage Second workpiece candidate list ; (13) (14) In equations (13) and (14), Indicates the first In the nth iteration Workpieces recorded by only ants In the Can the stage be processed? If so, then let... Otherwise, let ; Indicates the first In the nth iteration Only ants determine the workpiece Is it possible in the first In the nth iteration Only ants in the first The first stage of selection Processing on a parallel machine; if so, then... Otherwise, let ; Indicates the first In the nth iteration Workpieces recorded by only ants In the The arrival time of the stage; Indicates the first In the nth iteration Only ants in the first The first stage of selection The current completion time of the parallel machine.

[0034] Step 3.4.3, prioritize from Select the workpiece, if Then from Choose from; and calculate using equation (15). or medium workpiece by In the nth iteration Only ants in the first The first stage of selection Probability of parallel machining Therefore, the workpiece with the highest processing probability is selected as the first... In the nth iteration Only ants in the first Stage of workpiece processing ; (15) In equation (15), Indicates the first Only ants targeting the objective function Preference weights; Indicates the first In the nth iteration The workpiece selection strategy of an ant for Select workpiece at time In the In the nth iteration Only ants in the first The first stage of selection The heuristic information processed on the parallel machine is obtained through equation (16); Indicates the first In the nth iteration Only ants will carry the workpiece Assigned to in In the nth iteration Only ants in the first The first stage of selection When using a parallel machine, the objective function Pheromones expected; Indicates the first In the nth iteration The workpiece selection strategy of an ant for Select workpiece at time In the In the nth iteration Only ants in the first The first stage of selection Heuristic information processed on a parallel machine.

[0035] (16) In equation (16), Indicates the first In the nth iteration Workpieces recorded by only ants In the In the nth iteration Only ants in the first The first stage of selection Processing time on a parallel machine; Indicates the first In the nth iteration Workpieces recorded by only ants The size.

[0036] Step 3.4.4: Obtain the first result through equation (17). In the nth iteration Only ants will carry the workpiece Assigned to the In the nth iteration Only ants in the first The first stage of selection When using a parallel machine, the objective function Updated pheromone expectations : (17) Steps 3, 4, and 5, when When =1, if the remaining capacity of the first finite buffer is greater than that of the second finite buffer... In the nth iteration Workpieces recorded by only ants If the dimensions are such that the workpiece is... Put it into the first finite buffer, and middle Set the corresponding element to 0, and middle The corresponding element is set to The workpiece number, and release the first In the nth iteration Only ants in the first The first stage of selection A parallel machine; otherwise, it indicates the first parallel machine. In the nth iteration The ant selected in the first stage The parallel machine enters a blocked state and waits for the remaining capacity of the first finite buffer to be greater than or equal to... After determining the dimensions, return to step 3.4.5 to execute.

[0037] when When =2, if the remaining capacity of the second finite buffer is greater than or equal to the first... In the nth iteration Workpieces recorded by only ants If the dimensions are such that the workpiece is... Put it into the second finite buffer, and middle Set the corresponding element to 0, and middle The corresponding element is set to The workpiece number, and release the first In the nth iteration Only ants in the first The first stage of selection A parallel machine; otherwise, it indicates the first parallel machine. In the nth iteration Only ants in the first The first stage of selection The parallel machine enters a blocked state and waits for the remaining capacity of the first finite buffer to be greater than or equal to... After determining the dimensions, return to step 3.4.5 to execute.

[0038] Steps 3.4.6, if If there are any non-zero elements in the workpiece, return to step 3.4.1 and continue executing step 3.4.1. Phase scheduling.

[0039] Step 3.5, the In the nth iteration Only one ant performs the third phase of scheduling: Step 3.5.1, based on the first In the nth iteration The state of an ant Q-learning dynamically selects the workpiece selection strategy. Machine selection strategy In the workpiece set In the middle, through workpiece selection strategy Candidate workpieces were obtained through screening. In the idle batch processing machine set In the middle, through The first was obtained by filtering. In the nth iteration Only ants in the first The first stage of selection Taiwan batch processing machine; build batch , to the workpiece As a batch The first workpiece, and the batch The cluster class is set as the workpiece. Clusters.

[0040] Step 3.5.2, for batches The first equation is constructed using equations (18) and (19). In the nth iteration The first list of artifact candidates for ants Second workpiece candidate list ; (18) (19) In equations (18) and (19), Indicates the first In the nth iteration Workpieces recorded by only ants Can it be processed in stage 3? If so, then let Otherwise, let ; Indicates the first In the nth iteration Workpieces recorded by only ants Arrival time in stage 3; Indicates the first In the nth iteration Batch recorded by ants Start processing time; Indicates the first In the nth iteration Only ants in the first The first stage of selection The capacity of the batch processing machine; Indicates the first In the nth iteration Batch recorded by ants The current total size; Indicates the first In the nth iteration Workpieces recorded by only ants Clusters.

[0041] Step 3.5.3, prioritize from Select the workpiece, if Then from Select from the options; calculate the batch addition of candidate workpieces using equation (20). Given the probability, select the workpiece with the highest probability as the first... In the nth iteration Add only ants to the batch workpiece ; (20) In equation (20), Indicates the first In the nth iteration Only ants will carry the workpiece Add to batch When, the objective function Pheromones expected; Indicates the first In the nth iteration Only ants in the first Stage selection of workpiece Add to batch The heuristic information is obtained from equation (21), Indicates the first In the nth iteration Only ants will carry the workpiece Add to batch When, the objective function Pheromones expected; Indicates the first In the nth iteration Only ants in the first Stage selection of workpiece Add to batch Heuristic information.

[0042] (twenty one) In equation (21), Indicates the first In the nth iteration Workpieces recorded by only ants Add to batch Preparation time; Indicates the first In the nth iteration Batch recorded by ants In the In the nth iteration Only ants in the first The first stage of selection Processing time on a batch processing machine; Indicates workpiece In the In the nth iteration Only ants in the first The first stage of selection Processing time on a batch processing machine.

[0043] Step 3.5.4, Settings The local pheromone in the third stage is updated using equation (22). (twenty two) In equation (22), Indicates the updated number In the nth iteration Only ants will carry the workpiece With workpiece When allocated to the same batch, the objective function Pheromones expected.

[0044] Step 3.5.5, if There are workpieces in Meet the conditions and Then return to step 3.5.2 and continue to batch. Add a workpiece; Step 3.5.6: Batch Put into the first In the nth iteration Only ants in the first The first stage of selection Processed by batch processing machine; if If there is an item with element 1, return to step 3.6.1; otherwise, proceed to step 3.6.

[0045] Step 3.6: Iterative optimization and update of the optimal scheduling scheme set: Step 3.6.1, the In the nth iteration After an ant completes the three-stage scheduling, calculate its maximum completion time. Total energy consumption The solution And the corresponding scheduling scheme is stored in the current number. Iterative optimal scheduling scheme set ; Step 3.6.2, if Then Return to step 3.2 and execute sequentially; otherwise, it indicates that the desired outcome has been achieved. Proceed to step 3.6.3; Step 3.6.3, if Then Assign to Return to step 3.3 and execute sequentially; otherwise, it indicates that the desired outcome has been achieved. Then, the optimal scheduling schemes of all generations are merged and redundant schemes are removed to obtain the optimal scheduling scheme set, which is the optimal scheme set for the same-cluster batch scheduling of the three-stage energy-saving mixed flow workshop.

[0046] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

Claims

1. A three-stage energy-efficient hybrid flow shop identical batch scheduling method based on Q-learning, characterized in that, The application is applied to a three-stage hybrid flow shop composed of a processing stage, a surface treatment stage and a final test stage, and the processing stage, the surface treatment stage and the final test stage are sequentially recorded as the first stage, the second stage and the third stage, and the serial number of any stage in the first stage and the second stage is recorded as The processing stage contains parallel machines, the surface treatment stage contains parallel machines, and the final test stage contains batch machines, each workpiece in the first stage and the second stage is processed by a corresponding machine according to the machine eligibility constraint, and the batch machine in the third stage can process all workpieces; let the set of workpieces to be processed be , let any workpiece in be recorded as , , and let the total number of workpieces in be workpieces are processed in turn through the three stages, a first finite buffer with a capacity of is arranged between the first stage and the second stage, a second finite buffer with a capacity of is arranged between the second stage and the third stage, the workpieces in the third stage are batch-processed according to workpiece cluster classes, and the total size of each batch does not exceed the capacity of the batch machine Parallel machines and batch machines both contain working states, standby states and shutdown states with different energy consumptions; the same-cluster batch scheduling method of the three-stage energy-saving hybrid flow shop is executed according to the following steps: Step 1, constructing a bi-objective function of three-stage energy-saving hybrid flow shop cluster scheduling model, including: minimizing the maximum completion time and minimizing the total energy consumption , and recording any one of the objective functions in and as { , } ; Step 2, constructing constraint conditions of the three-stage energy-saving hybrid flow shop same cluster batch scheduling model; Step 3, solving the three-stage energy-saving hybrid flow shop same cluster batch scheduling model by using a hybrid ant colony optimization algorithm based on Q-learning to generate an optimal scheduling scheme.

2. The three-stage energy-efficient hybrid flow shop identical batch scheduling method based on Q-learning according to claim 1, wherein, The step 1 is to construct respectively by using formula (1) and formula (2) and : (1) (2) In formula (1) and formula (2), represents the current batch; represents the set of batches; represents the current batch ; represents the completion time of the current batch; represents the first stage when represents the second stage when represents the index of the parallel machine in the first stage and the second stage; represents the total number of parallel machines in the stage; represents the time; represents the total duration; represents the current state of the machine, and when , it represents the working state; when , it represents the standby state; and when , it represents the shutdown state; represents the energy consumption of the parallel machine in the state in the stage; represents whether the th parallel machine in the stage is in the state at the time , and if so, then , otherwise ; represents the index of the batch processor; represents the energy consumption of the batch processor in the state; represents whether the th batch processor is in the state at the time , and if so, then , otherwise 0; represents the total number of batch processors.

3. The three-stage energy-efficient hybrid flow shop identical batch scheduling method based on Q-learning according to claim 2, characterized in that, The step 2 comprises: Step 2.1, constructing workpiece and machine allocation constraints by using formula (3): (3) in formula (3), indicates whether the workpiece is processed on the first stage of the first machine, if yes, then let , otherwise let 0; ; indicates whether the workpiece is processed on the first stage of the first parallel machine, if yes, then let , otherwise let ; ; indicates whether the workpiece is processed in the current batch of the first batch processor, if yes, then let , otherwise let ; ; indicates whether the current batch is processed on the first batch processor, if yes, then let , otherwise let ; Step 2.2, constructing same cluster and batch processing capacity constraints by using formula (4): (4) in formula (4), indicates the workpiece whether in the first batch of the current batch , if yes, let , otherwise, let ; indicates the workpiece belongs to the cluster; indicates the workpiece belongs to the cluster; indicates if the condition is met, let , otherwise, let ; indicates the size of the workpiece ; indicates the capacity of the first batch processor; indicates whether the current batch contains at least one workpiece, if yes, let , otherwise, let ; indicates whether the first batch contains at least one workpiece, if yes, let , otherwise, let ; Step 2.3, constructing time logic constraints by using formula (5): (5) In equation (5), Indicates workpiece The start time of processing in Phase 1; Indicates the first stage of the workpiece Arrival time; Indicates workpiece The start time of processing in stage 2; Indicates workpiece Completion time of Phase 1; Indicates workpiece Waiting time in the first finite buffer; Indicates workpiece In the Completion time of each stage; Indicates workpiece In the Phase 1 Processing time on a parallel machine; Indicates workpiece In the The start time of the processing stage; Indicates workpiece In the Phase 1 Processing time on a parallel machine; Indicates workpiece Is it in the first Phase 1 Processing on a parallel machine; if so, then... Otherwise, let ; Represents positive numbers; Indicates workpiece The start time of processing in stage 3; Indicates workpiece Completion time of Phase 2; Indicates workpiece Waiting time in the second finite buffer; Indicates the current batch Completion time; Indicates workpiece In the Processing time on a batch processing machine; Step 2.4, constructing buffer capacity constraints by using formula (6): (6) In formula (6), indicates the workpiece at time whether it is located in the first buffer, if yes, then let , otherwise, let ; indicates the capacity of the buffer 1; indicates the workpiece at time whether it is located in the second buffer, if yes, then let , otherwise, let ; indicates the capacity of the buffer 2; Step 2.5, constructing machine state constraints by using formula (7): (7) In formula (7), represents the stage the th parallel machine is in the state at time , if yes, then let , otherwise let ; represents the stage the th parallel machine is in the standby state, if yes, then let , otherwise let ; ; represents the stage the th parallel machine is in the shutdown state, if yes, then let , otherwise let ; ; represents the stage the th machine is in the shutdown state, if yes, then let , otherwise let ; ; represents the th batch machine is in the state at time , if yes, then let , otherwise let ; represents the th batch machine is in the standby state, if yes, then let , otherwise let ; .

4. The three-stage energy-efficient hybrid flow shop identical batch scheduling method based on Q-learning according to claim 3, wherein, The step 3 comprises: Step 3.1, parameter initialization: Step 3.1.1, initialization of parameters of the ant colony optimization algorithm: local pheromone weight heuristic information weight local pheromone evaporation coefficient ; Step 3.1.2, initialize parameters of Q-learning: learning rate , discount factor , exploration rate ; Step 3.1.3, setting running parameters: total number of iterations total number of ants current iteration number current iteration number set of optimal scheduling schemes for the current iteration ; Step 3.1.4, determine and initialize current ant number = 1; Step 3.2, defining and initializing the first iteration of the first ant in the objective function under the first phase pheromone matrix , the first iteration of the first ant in the objective function under the third phase inter-job intra-batch pheromone matrix and the first iteration of the first ant in the objective function under the batch and job pheromone matrix ; Step 3.3, defining the first iteration of the second ant only state , action , first iteration of the second ant only action reward function and Q-value update mechanism Step 3.4, define and initialize the first iteration of the second ant only the 3-phase tabu list matrix, and initialize the first The initial job capacity of the limited buffer is set to 0; Step 3.5: Based on the pheromone matrix , 2nd The taboo list matrix for each stage is learned through Q-learning based on the state. ,action Reward function And the Q-value update mechanism, for the first In the nth iteration Only one ant performed the first Stage scheduling to obtain workpieces In the Processing plan in stages; Step 3.6, based on 2 pheromone matrices , and the tabu list matrix of the 1st phase, the 3rd phase scheduling is performed for the 1st ant in the 1st iteration by Q-learning according to the state , action , reward function and Q-value updating mechanism, to obtain the solution of workpiece group batching and processing; Step 3.7, the In the nth iteration After an ant completes the three-stage scheduling, calculate its maximum completion time. Total energy consumption ,Will And the corresponding scheduling scheme is stored in the first... Iterative optimal scheduling scheme set ; Step 3.8, if then set to After that, go to Step 3.2; otherwise, the final set of iterative optimal scheduling schemes is obtained. ; Step 3.9, if , then set to , return to Step 3.1.4 for sequential execution, otherwise, it indicates that the is reached, and all the optimal scheduling scheme sets of the generations are combined and the redundant schemes are deleted, thereby obtaining the optimal scheduling scheme set, i.e. the optimal scheme set of the three-stage energy-saving hybrid flow shop homogenous batch scheduling.

5. The three-stage energy-efficient hybrid flow shop identical batch scheduling method based on Q-learning according to claim 4, wherein, The step 3.2 comprises: Order No. In the nth iteration Only ants will carry the workpiece Assigned to the Phase 1 When using a parallel machine, the objective function Pheromones expected to be denoted as Thus, the first In the nth iteration Only ants in the objective function Next Phase pheromone matrix Random initialization ; Indicates the initial concentration of pheromones; Let the th iteration be the th iteration Only ants will allocate the same batch of jobs between the third stage jobs, the pheromone expectation of the objective function is recorded as , so as to obtain the th iteration of the th iteration of the third stage job between the same batch of jobs under the objective function ; initialized to ;​ Order No. In the nth iteration Only ants will carry the workpiece Add to current batch When, the objective function Pheromones expected to be denoted as And through equation (8), we obtain the first... In the nth iteration Only ants in the objective function Batch and workpiece pheromone matrix ; (8) In formula (8), is the th iteration the set of artifacts already included in the current batch of ants, is the number of artifacts in ​​ 6. The three-stage energy-efficient hybrid flow shop identical batch scheduling method based on Q-learning according to claim 5, wherein, The step 3.3 comprises: Definition of the first Definition of the first Status of the only ant wherein, ∈{1,2,3} represents the current phase being processed, represents the first Definition of the first Current phase of the only ant Number of machines blocked downstream, ∈[0,100] represents the first Definition of the first Utilization of the first buffer of the only ant; Definition of the first Definition of the first Action of the first wherein, ∈{1,2,3,4} indicates the Definition of the first Workpiece selection strategy of the first indicates the preference for the shortest arrival time of the workpiece, when indicates the preference for the shortest processing time of the workpiece, when indicates the preference for the longest processing time of the workpiece, when indicates the preference for the largest size of the workpiece; ∈{1,2,3} indicates the Definition of the first Machine selection strategy of the first indicates the selection of the machine with the lowest energy consumption per unit of time, when indicates the selection of the machine with the smallest current load, when indicates the selection of the machine available earliest; The first iteration in the second iteration is constructed by equation (9) The reward function after the ants take action (9) In formula (9), denotes the th iteration ; denotes the th iteration th ant's maximum completion time improvement rate, and is obtained by formula (10); denotes the th iteration th ant's total energy consumption improvement rate, and is obtained by formula (11); (10) In formula (10), denotes the first iteration the maximum completion time of the first iteration the maximum completion time of the first iteration the (11) In formula (11), represents the first iteration in the first total energy consumption of the only ant, represents the first iteration in the first total energy consumption of the only ant; The first iteration is constructed by defining the first The second iteration is constructed by defining the second The Q-value update mechanism for the ant only: (12) In Equation (12), represents the first ant in the state takes action in the state after the state-action function value is updated; represents the first ant in the state takes action in the state after the state-action function value is updated; represents the first ant in the state takes action after the state-action function value is updated; represents the first ant in the state takes the optimal action in the state represents the first ant in the state selects the optimal action in the state with the expected reward.

7. The three-phase energy-saving hybrid flow shop identical batch scheduling method based on Q-learning according to claim 6, wherein, The step 3.4 comprises: define and initialize the first iteration of the first ant's tabu list matrix in phase 1 , define and initialize the first iteration of the first ant's tabu list matrix in phase 2 , define and initialize the first iteration of the first ant's tabu list matrix in phase 3 , respectively, for the first iteration of the first ant to record whether there are unprocessed workpieces in phase 1, phase 2, and phase 3; if the element in the tabu list matrix is not 0, it indicates that the workpiece at the corresponding position has not been processed; the unprocessed workpiece set of the first iteration of the first ant is composed of workpieces whose elements in the tabu list matrix are not 0 ; the unprocessed workpiece set of the first iteration of the first ant is composed of workpieces whose elements in the tabu list matrix are not 0 ; the unprocessed workpiece set of the first iteration of the first ant is composed of workpieces whose elements in the tabu list matrix are not 0 ; the unprocessed workpiece set of the first iteration of the first ant is composed of workpieces whose elements in the tabu list matrix are not 0 .

8. The three-phase energy-saving hybrid flow shop identical batch scheduling method based on Q-learning according to claim 7, characterized in that, The step 3.5 comprises: Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant State of the only ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Step 3.5.1, according to the first iteration of the first ant Utilizing the determined machine selection policy To possess A set of idle parallel machines with processing eligibility After screening, a first In a first iteration, a first Only ants in a first Stage selection of a first Parallel machine; Step 3.5.2, constructing the first iteration of the first ant in the first stage of the first and second candidate lists ; (13) (14) In equations (13) and (14), Indicates the first In the nth iteration Workpieces recorded by only ants In the Can the stage be processed? If so, then let... Otherwise, let ; Indicates the first In the nth iteration Only ants determine the workpiece Is it possible in the first In the nth iteration Only ants in the first The first stage of selection Processing on a parallel machine; if so, then... Otherwise, let ; Indicates the first In the nth iteration Workpieces recorded by only ants In the The arrival time of the stage; Indicates the first In the nth iteration Only ants in the first The first stage of selection The current completion time of the parallel machines; Step 3.5.3, prioritize from Select the workpiece, if Then from Choose from; and calculate using equation (15). or medium workpiece by In the nth iteration Only ants in the first The first stage of selection Probability of parallel machining Therefore, the workpiece with the highest processing probability is selected as the first... In the nth iteration Only ants in the first Stage of workpiece processing ; (15) In equation (15), Indicates the first Only ants targeting the objective function Preference weights; Indicates the first In the nth iteration The workpiece selection strategy of an ant for Select workpiece at time In the In the nth iteration Only ants in the first The first stage of selection The heuristic information processed on the parallel machine is obtained through equation (16); Indicates the first In the nth iteration Only ants will carry the workpiece Assigned to in In the nth iteration Only ants in the first The first stage of selection When using a parallel machine, the objective function Pheromones expected; Indicates the first In the nth iteration The workpiece selection strategy of an ant for Select workpiece at time In the In the nth iteration Only ants in the first The first stage of selection Heuristic information processed on a parallel machine; (16) In formula (16), represents the first iteration recorded by the first ant in the first iteration first ant in the first stage machining time on the first iteration first iteration recorded by the first ant Step 3.5.

4. Obtaining the first iteration of the first ant by means of formula (17) Assigning the workpiece to the first ant in the first stage of the selection of the first machine in parallel and updating the pheromone expectation of the objective function : (17) Step 3.5.5, when =1, if the remaining capacity of the first finite buffer is greater than the size of the workpiece selected by the i-th ant in the j-th iteration in the first stage, the workpiece is put into the first finite buffer, the element corresponding to the workpiece in the first finite buffer is set to 0, the element corresponding to the workpiece in the second finite buffer is set to the workpiece number of the workpiece, and the first finite buffer is released; otherwise, it indicates that the i-th ant in the j-th iteration in the first stage selects the j-th parallel machine into the blocking state, and waits until the remaining capacity of the first finite buffer is greater than or equal to the size of the workpiece selected by the i-th ant in the j-th iteration in the first stage, and then returns to step 3.5.5 for execution. Step 3.5.5, when =1, if the remaining capacity of the first finite buffer is greater than the size of the workpiece selected by the i-th ant in the j-th iteration in the first stage, the workpiece is put into the first finite buffer, the element corresponding to the workpiece in the first finite buffer is set to 0, the element corresponding to the workpiece in the second finite buffer is set to the workpiece number of the workpiece, and the first finite buffer is released; otherwise, it indicates that the i-th ant in the j-th iteration in the first stage selects the j-th parallel machine into the blocking state, and waits until the remaining capacity of the first finite buffer is greater than or equal to the size of the workpiece selected by the i-th ant in the j-th iteration in the first stage, and then returns to step 3.5.5 for execution.​​​​​​​​​​​​​​ When =2, if the remaining capacity of the second finite buffer is greater than or equal to the size of the job selected by the first ant in the second iteration at the (i-1)th stage, the job is put into the second finite buffer, the element corresponding to the job in the first iteration is set to 0, the element corresponding to the job in the second iteration is set to the job number of the job, and the first parallel machine selected by the first ant in the second iteration at the (i-1)th stage is released; otherwise, the first parallel machine selected by the first ant in the second iteration at the (i-1)th stage enters a blocked state and waits until the remaining capacity of the first finite buffer is greater than or equal to the size of the job selected by the first ant in the second iteration at the (i-1)th stage, and then returns to step 3.5.5 for execution. ​​​​​​​​​​​​​​​​​​ Step 3.5.6, if If there are workpieces with non-zero elements in the middle, go to Step 3.5.1 and continue to execute the first Stage scheduling.

9. The three-phase energy-saving hybrid flow shop identical batch scheduling method based on Q-learning according to claim 8, characterized in that, The step 3.6 comprises: Step 3.6.1, based on the first iteration of the second iteration of the state of the only ants dynamically selecting a job selection policy by Q-learning and a machine selection policy ; In the set of workpieces , a workpiece selection strategy is used to filter out candidate workpieces ; In the idle batch processor set , the first iteration is obtained by screening the first ant in the first stage selection of the first batch processor; build batch , the workpiece is constructed as the first workpiece of the batch , and the cluster class of the batch is set to the cluster class of the workpiece ; Step 3.6.2, for batches The first equation is constructed using equations (18) and (19). In the nth iteration The first list of artifact candidates for ants Second workpiece candidate list ; (18) (19) In equations (18) and (19), Indicates the first In the nth iteration Workpieces recorded by only ants Can it be processed in stage 3? If so, then let Otherwise, let ; Indicates the first In the nth iteration Workpieces recorded by only ants Arrival time in stage 3; Indicates the first In the nth iteration Batch recorded by ants Start processing time; Indicates the first In the nth iteration Only ants in the first The first stage of selection The capacity of the batch processing machine; Indicates the first In the nth iteration Batch recorded by ants The current total size; Indicates the first In the nth iteration Workpieces recorded by only ants Clusters; Step 3.6.

3. Select the workpiece from , if , select from ; calculate the probability of the candidate workpiece joining the batch by equation (20), select the workpiece with the largest probability as the workpiece added to the batch by the th ant in the th iteration ; (20) In formula (20), denotes the first ant in the ith iteration; the heuristic information for the first ant to add the job to the batch at the th stage, and is given by formula (21), denotes the first ant in the ith iteration; the heuristic information for the first ant to add the job to the batch at the th stage, and is given by formula (21), denotes the first ant in the ith iteration; the heuristic information for the first ant to add the job to the batch at the th stage, and is given by formula (21), denotes the first ant in the ith iteration; the heuristic information for the first ant to add the job to the batch at the th stage, and is given by formula (21), (21) in formula (21), indicates the first iteration only ant record of the job joining the batch preparation time; indicates the first iteration only ant record of the batch in the first iteration only ant at stage selected for the first batch processor indicates the first iteration only ant record of the job at stage selected for the first batch processor Step 3.6.4, Setup updating the local pheromone of the 3rd stage by equation (22); (22) In formula (22), the first iteration the first iteration Only ants assign workpieces to the same batch as workpieces The objective function updated pheromone expectation; Step 3.6.5, if There are workpieces in Meet the conditions and Then return to step 3.6.2 and continue to batch. Add a workpiece; otherwise, proceed to step 3.6.

6. Step 3.6.6: Batch Put into the first In the nth iteration Only ants in the first The first stage of selection Processed by batch processing machine; if If there is an item with element 1, return to step 3.6.1; otherwise, proceed to step 3.

7.

10. An electronic device comprising a memory and a processor, characterized in that The memory is used for storing a program supporting the processor to execute the three-stage energy-saving hybrid flow shop same cluster batch scheduling method in any one of claims 1-9, and the processor is configured to execute the program stored in the memory.

Images