Scheduling method for distributed assembly wait-free flow shop based on Q learning

Abstract

The invention discloses a scheduling method of a distributed assembly wait-free flow shop based on Q learning. The scheduling method comprises the following steps: 1, constructing an objective function of a distributed assembly wait-free flow shop model; 2, constructing constraint conditions of a distributed assembly wait-free flow shop scheduling model; and 3, solving the distributed assembly wait-free flow shop scheduling model by using a dual-population co-evolution algorithm based on Q learning, and generating a distributed assembly wait-free flow shop scheduling scheme. According to the method, the optimal production and processing scheme with short time consumption and low carbon emission can be obtained in a distributed assembly wait-free flow workshop scene, so that the production and processing efficiency is improved, and the carbon emission is reduced.

Description

Technical Field

[0001] This invention belongs to the field of assembly line scheduling technology, specifically a scheduling method for a distributed assembly line with no waiting time based on Q-learning. Background Technology

[0002] With the deepening of economic globalization, multi-factory distributed manufacturing systems are widely used in industrial production and scheduling. Distributed manufacturing, through collaborative production among multiple enterprises, not only effectively reduces production costs but also plays a crucial role in improving product quality. At the same time, against the backdrop of vigorous development of the manufacturing industry, environmental issues are receiving increasing attention. As a major global energy consumer, manufacturing currently accounts for approximately half of the world's total energy consumption. This necessitates improving energy efficiency and reducing carbon emissions while simultaneously increasing production efficiency, making a green and sustainable development path particularly essential.

[0003] The Distributed Assembly No-Wait Flow Shop Scheduling Problem (DANWFSP) originates from the Permutation Flow Shop Scheduling Problem (PFSP) and has a wide range of applications. Current research largely assumes factory isomorphism, lacking in-depth understanding of the diverse heterogeneous characteristics prevalent in real-world production scenarios. Meanwhile, with increasing societal environmental awareness, green scheduling is gaining traction, and incorporating carbon emissions into optimization objectives has become a significant trend in this field. Regarding algorithms for solving such problems, existing metaheuristic algorithms have limitations in optimizing complex scenarios. In contrast, hybrid algorithms combining metaheuristics and reinforcement learning exhibit superior performance and have thus become the mainstream research trend in this area. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a scheduling method for distributed assembly line workshops with no waiting time based on Q-learning. The aim is to obtain production plans with short processing times and low carbon emissions, thereby improving production efficiency and reducing carbon emissions. This will help manufacturing enterprises reduce manufacturing costs and achieve energy conservation and emission reduction.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0006] The present invention discloses a scheduling method for a distributed assembly line workshop with no waiting time based on Q-learning, characterized in that the distributed assembly line workshop with no waiting time includes: a processing stage and an assembly stage, wherein the processing stage is... A processing plant uses processing machines to collect the workpieces to be processed. After processing, the materials are transported to the assembly plant. During the assembly stage, one assembly plant uses one assembly machine to... Individual workpieces are assembled into a product assembly. ;in, Indicates the number to be processed One workpiece, express The total number of workpieces; express The Middle One product Represents a product set The total number of products in the process ensures that each processing plant has A set of processing machines is provided, with each machine having a state including processing, idle, and off. The power output of each machine varies depending on its state, allowing workpieces to be processed on any machine in any processing plant, and ensuring that at any given time, a workpiece is processed on only one machine. The scheduling method is performed as follows:

[0007] Step 1: Construct the objective function of the distributed assembly line scheduling model with no waiting time, including: minimizing the maximum completion time. and total carbon emissions ;

[0008] Step 2: Construct the constraints for the distributed assembly line scheduling model with no waiting time;

[0009] Step 3: Use the Q-learning-based dual-population co-evolutionary algorithm to solve the distributed assembly line workshop scheduling model with no waiting time, and generate a distributed assembly line workshop scheduling scheme with no waiting time.

[0010] The characteristic of the distributed assembly wait-free shop scheduling method based on Q-learning described in this invention is that the objective function is obtained using equation (1) in step 1:

[0011] (1)

[0012] In equation (1), Represents a product set The Middle Product Assembly completion time; Indicates the total number of moments; Indicates the processing stage The first processing plant Is the processing machine in the first... If it is in a processing state at any given moment, then let =1, otherwise, let =0; Indicates the first The first processing plant The power of a processing machine when it is in processing mode; Indicates the processing stage The first processing plant Is the processing machine in the first... If the current state is idle, then let... =1, otherwise, let =0; Indicates the first The first processing plant The power of a processing machine when it is idle; Indicates the first Carbon emission conversion factor of a processing plant.

[0013] Furthermore, step 2 includes:

[0014] Step 2.1: Use equation (2) to construct the allocation relationship constraints between the processing plant and the workpiece;

[0015] (2)

[0016] In equation (2), Indicates the index of the workpiece. Indicates the workpiece's machining position index; This represents the j-th workpiece. Is it assigned to the first The first processing plant If it is at a processing position, then let Otherwise, let ; Indicates the first In a processing plant, the g-th workpiece Is it in the j-th workpiece? If it was processed previously, then... Otherwise, let ; Indicates the first In a processing plant, the j-th workpiece Is it in the g-th workpiece? If it was processed previously, then... Otherwise, let ; Indicates the first In a processing plant, the j-th workpiece Is it in a virtual workpiece? If it was processed previously, then... Otherwise, let ;

[0017] Step 2.2: Use equation (3) to construct the time constraints of the processing plant's no-waiting assembly line workshop;

[0018] (3)

[0019] In equation (3), Indicates the index of the processing machine. Index representing time, Indicates the first One workpiece In the The first processing plant The time when processing begins on the processing machine; Indicates the first One workpiece In the The first processing plant The processing time completed on the processing machine; Indicates the first One workpiece In the The first processing plant The processing time completed on the processing machine; Indicates the first One workpiece In the The first processing plant Processing time on the machine; Indicates the first In the processing plant, the first One workpiece Is it in the first The moment is placed at the first Processing on a Taiwanese processing machine; if so, then... Otherwise, let L represents a positive integer;

[0020] Step 2.3: Use equation (4) to construct the state constraints of the processing machines in the processing plant;

[0021] (4)

[0022] In equation (4), Indicates the processing time of the processing machine; Indicates the processing stage The first processing plant Is the processing machine in the first... If a process is in a processing state at any given moment, then let... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If a processing time is idle, then let =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If the processing time is currently in a closed state, then... =1, otherwise, let =0; Indicates the next processing stage The first processing plant Is the processing machine in the first... If a process is in a processing state at any given moment, then let... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine idle at the initial processing time? If so, then... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If a processing time is idle, then let =1, otherwise, let =0; A binary auxiliary variable representing the processing stage;

[0023] Step 2.4: Use equation (5) to construct the assembly sequence constraints of products in the assembly plant;

[0024] (5)

[0025] In equation (5), Represents a product set Index of products in China; Indicates the first Product Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Indicates virtual products Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Indicates the first Product Is it in the first Product If the assembly has already begun, then... Otherwise, let ;

[0026] Step 2.5: Use equation (6) to construct the assembly time constraints for products in the assembly plant;

[0027] (6)

[0028] In equation (6), Indicates the first The workpiece in the first The first processing plant The processing time of a single processing machine; Indicates the first One workpiece Belongs to the Product ; Indicates the first Product The time when assembly begins on the assembly machine; Indicates the first Transportation time from the processing plant to the assembly plant; Indicates the first Product Assembly time on the assembly machine; Indicates the first Product In the If the assembly is being performed on the assembly machine at a given time, then... Otherwise, let ; Indicates virtual products The time when assembly begins on the assembly machine.

[0029] Furthermore, step 3 is performed as follows;

[0030] Step 3.1: Define the current iteration number as... and initialize The maximum number of iterations is defined as ;

[0031] Step 3.2, Set the first Generation population Each individual in the algorithm consists of a three-layer coding scheme, where the first layer of coding is composed of... The first layer of coding consists of a sequence of workpiece numbers to characterize the processing order of the workpieces; the second layer of coding consists of... A factory sequence consisting of the serial numbers of each processing plant is used to characterize the allocation of workpieces. The third layer of coding consists of... A product sequence composed of the serial numbers of each product is used to represent the assembly order of the products;

[0032] Some of the individuals are selected based on total processing time and total carbon emissions. The algorithm initializes the first and second layer codes; and uses the shortest processing time first algorithm to initialize the third layer code. The remaining individuals are initialized using a random generation algorithm; thus obtaining the first individual. Generation population = ,in, express The first in Individual; express The scale;

[0033] Step 3.3, for After normalization, we obtain the first... Generational normalized population ,in, express After normalization, the first individual;

[0034] Step 3.4: Calculate the first step using equation (1). individual Corresponding maximum completion time and total carbon emissions ,get Corresponding solution space coordinates ( );in, Indicates the maximum completion time , Indicates total carbon emissions ;

[0035] like Then Classified as the Left subpopulation ,like Then Classified as the upper right subpopulation ; thereby Divided into the first Subpopulation of generations ,make and Any subpopulation in the middle is denoted as , ;

[0036] Step 3.5: Initialize the first equation using equations (7) and (8). The generation Subspecies Reference point ;

[0037] , (7)

[0038] , (8)

[0039] In equations (7) and (8), express The scale; express The first in individual Total carbon emissions, express The first in individual Maximum completion time; express Maximum completion time; express Total carbon emissions;

[0040] Step 3.6, Define the first The generation Subspecies Improved threshold set ,in, express The first in An improved threshold, express The scale;

[0041] Definition of the first The generation Reference points The set of moving steps ,in, express The first in Reference points The step size of movement, express The scale;

[0042] from Randomly select an improved threshold as Improved threshold ;from Randomly select a movement step size as Movement step size ;

[0043] Step 3.7, determine the first offspring population The Middle individual Solution space coordinates ( ) and the offspring population Any number in the middle individual Solution space coordinates ( Does it satisfy equation (8) or equation (9)? If it does, then it means... for The non-dominant individual, otherwise, indicates To dominate individuals; thereby All non-dominant individuals constitute the first Substitute non-dominated solution set ,in, express The first in Individual, express The scale;

[0044] (9)

[0045] and (10)

[0046] Step 3.8, based on the reference point Using an improved genetic algorithm to Optimize to obtain the first The generation Optimized subpopulation ;

[0047] Step 3.9: Calculate the first step using equation (13). Optimized subpopulation The Middle individual and the individual Solution space distance between Thus, the dimension is triangular distance matrix and from The parameter obtained from the solution space distance is less than the population maintenance distance. The Individuals of a generation to a set Then, proceed according to step 3.9. Each individual in the process optimizes against the individuals with poor fitness, thus obtaining the [number]th [unit]. The updated set of individual pairs ; and based on renew , obtained the The generation One maintenance subpopulation ;

[0048] = (13)

[0049] In equation (13), express The first in individual Maximum completion time express The first in individual Total carbon emissions; express The first in individual Maximum completion time express The first in individual Total carbon emissions;

[0050] Step 3.10, place the first Sub-population maintenance Non-dominated solution set After merging and removing duplicates, the optimized top-left subpopulation is obtained. and the optimized lower right subpopulation ,Will and Any optimized subpopulation is denoted as the i-th An optimized subpopulation Then, proceed according to step 3.8. and Process the solution to obtain the optimized set of non-dominated solutions. ,in, express The first in Individual, express The scale;

[0051] Step 3.11: Calculate using equations (14) and (15) Normalized hypervolume value and Normalized hypervolume value ,

[0052] (14)

[0053] (15)

[0054] In equations (14) and (15), Indicates the first An optimized subpopulation Nondominated solution set The first in individual Maximum completion time; Indicates the first An optimized subpopulation Nondominated solution set The first in individual Total carbon emissions; Indicates the first An optimized subpopulation Optimized nondominated solution set The first in individual Maximum completion time; Indicates the first An optimized subpopulation Optimized nondominated solution set The first in individual Total carbon emissions;

[0055] Step 3.12, using equation (16) to obtain and degree of improvement ,like Then, using equations (17) and (18) to apply the first... Reference point Update to obtain the first Reference point , order the An optimized subpopulation Comparison reference points = Even if the comparison reference point Maximum completion time = ,make Total carbon emissions = Otherwise, directly Assign to ;

[0056] (16)

[0057] (17)

[0058] (18)

[0059] In equations (17) and (18), express Maximum completion time; express Total carbon emissions;

[0060] Step 3.13, if =0, then set the first The generation Subspecies Q-learning controller and the Q-learning controller Initialize the parameters;

[0061] Step 3.14, if the first Reference point With the Reference point If they are different, use the Q-learning controller. Get the The generation Subgroup Reference point Movement step size and improved threshold Otherwise, directly Assign to ,Will Assign to ;

[0062] Step 3.15, if G Then output All non-dominated individuals are considered as the final production and processing scheme; otherwise, the first... An optimized subpopulation As the first Subpopulation of generations ,make +1 is assigned to Then, return to step 3.7.

[0063] Furthermore, step 3.8 includes:

[0064] Step 3.8.1 Based on the reference point Calculate using equation (11) The Middle 1 individual fitness value ;

[0065] (11)

[0066] Step 3.8.2 will After sorting all individuals in ascending order of fitness value, the top few individuals are placed as elite individuals in the next group. The generation Elite Pool Middle; Order The Middle 1 individual The sorting number is ;

[0067] Step 3.8.3 Calculate using equation (12) The Middle individual Probability of being selected and according to The first parent individual is obtained by using a roulette wheel. Second parent individual ;

[0068] (12)

[0069] In equation (12), Indicates the probability of choosing to regulate. express of Power;

[0070] Step 3.8.4 Select and The first layer of encoding randomly generates the intersection interval [ , ], and copy The first layer of coding interval [ , The workpiece sequence is used as the offspring individual. The first layer of coding interval [ , The sequence of workpieces;

[0071] Establish and The first layer of coding interval [ , Mapping relationship, take In the first layer of coding, non-range [ , The workpiece serial number is assigned to the child individual through a mapping relationship. The first layer of coding interval [ , The position of ] is used to obtain the offspring individuals. The first-level encoding of the offspring;

[0072] Take respectively and After the second layer of encoding and two-point crossover, offspring individuals are obtained. The second layer of encoding;

[0073] Use the shortest processing time first algorithm to evaluate offspring individuals. The first and second layer codes are processed to obtain offspring individuals. The third layer of encoding, thus completing the offspring individual Three-layer encoding;

[0074] Step 3.8.5 Mutate the offspring individual C with a certain probability. If mutation occurs, randomly select a workpiece from the first layer of C's encoding and move it to a random position. Also, randomly select a workpiece from the second layer of C's encoding and move it to another processing plant, thus obtaining a new offspring individual C'. Otherwise, directly... Assign it to C';

[0075] Step 3.8.6 Perform a local search on C' to obtain the new individual #1. The new second individual The new third individual The new fourth individual ; and calculate respectively , , The fitness value of the new individual is compared with the fitness value of C'. If the fitness value of the new individual exceeds that of C', then the corresponding new individual replaces C'; otherwise, it is not replaced.

[0076] Randomly select a workpiece serial number from within the factory from C' and exchange it to obtain... ;

[0077] Randomly select a workpiece number from C' within the factory sequence and insert it into the position of other workpiece sequences to obtain... ;

[0078] Randomly select any two factory workpiece serial numbers from C' and swap them to obtain... ;

[0079] Randomly select a workpiece serial number from C' within a factory and insert it into a random position in another factory, to obtain... ;

[0080] Step 3.8.7 Add C' to the first... The generation intermediate offspring population Then, and judge Has the required scale been reached? If so, proceed to step 3.8.8; otherwise, return to step 3.8.3 and execute sequentially.

[0081] Step 3.8.8 will and After merging, select the quantity as The most fit individual forms .

[0082] Furthermore, in step 3.14, the Q-learning controller is used according to the following process:

[0083] Step a: Obtain the comparison reference point using equation (19) With the current iteration number Deviation between ;

[0084] (19)

[0085] Step b: Define the Gth generation offspring population state set Let the state set Any state is ,in, ;

[0086] when hour, For state ;

[0087] when hour, For state ;

[0088] when )hour, For state ;

[0089] Step c: Calculate the first step using equation (20). offspring population State interval value ;

[0090] (20)

[0091] Step d, will and After orthogonalization, the first... The generation Action set ,in, express The first in Each action includes improving the threshold. and moving step size , express The scale;

[0092] Step e: Calculate the first step using equations (21) and (22). The generation Reward Value and according to To update the Q table; thereby utilizing Greedy strategy to obtain the first The generation Subgroup action ,make Improved threshold and Movement step size ;

[0093] (twenty one)

[0094] In equations (21) and (22), Let be the standard deviation in a normal distribution, and we have:

[0095] (twenty two).

[0096] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program that supports the processor in executing the Q-learning-based distributed assembly waiting flow shop scheduling method, and the processor is configured to execute the program stored in the memory.

[0097] The present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, performs the steps of the low-Q-based learning-based distributed assembly waiting flow shop scheduling method.

[0098] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0099] 1. This invention addresses the practical application scenario of distributed assembly line workshops with no waiting time. Considering heterogeneous factors such as machine differences and factory geographical location, it establishes a multi-objective optimization model aimed at minimizing the maximum assembly completion time and total carbon emissions. A dual-population evolutionary algorithm based on Q-learning is used to solve this model, providing decision-makers with a set of production and processing schemes that are time-efficient and have low carbon emissions. This improves production efficiency and reduces carbon emissions in practical applications.

[0100] 2. The dual-population evolutionary algorithm based on Q-learning in this invention sets a dynamic reference point for each population based on the dual-objective optimization characteristics, which is dynamically adjusted by Q-learning. This provides a precise search direction for the evolutionary algorithm, thereby realizing differentiated exploration between the two populations, effectively improving the global search efficiency and solution quality, and thus obtaining a more comprehensive production and processing solution for distributed assembly line workshops without waiting time.

[0101] 3. The population maintenance mechanism based on solution space distance in this invention calculates the distance between individuals in the normalized solution space to compare the degree of similarity between individuals, and optimizes individuals with high similarity but poor quality in a targeted manner, thereby effectively ensuring the stability of population diversity and quality, and thus obtaining a better distributed assembly line workshop production and processing solution. Attached Figure Description

[0102] Figure 1 This is a flowchart of the method of the present invention;

[0103] Figure 2 This is a flowchart of the Q-learning-based dual-population co-evolutionary algorithm involved in this invention;

[0104] Figure 3 This is a schematic diagram of the three-layer encoding and decoding of an individual population in this invention;

[0105] Figure 4 This is a flowchart of the improved genetic algorithm involved in this invention. Detailed Implementation

[0106] In this embodiment, a scheduling method for a distributed assembly line with no waiting time based on Q-learning is proposed, which uses a dual-population cooperative evolution algorithm based on Q-learning to solve the problem. The problem of scheduling in a distributed assembly line with no waiting time. This involves... Each workpiece forms a set of products. Need to be Distributed assembly line production with no waiting time. Any workpiece can be assigned to... In any one of the processing plants, and through the plant The machines process the workpiece sequentially. After processing, the workpiece is transported to the assembly plant for assembly. The power of the machines differs between processing and idle states, and the processing time for the same workpiece varies between different plants. The geographical and technological differences of distributed factories may result in different carbon emission conversion factors for each plant. The entire production process comprises three stages: production, transportation, and assembly. Specifically, as... Figure 1 As shown, the method is performed according to the following steps:

[0107] Step 1: Construct the objective function of the distributed assembly line scheduling model with no waiting time using equation (1), including: minimizing the maximum completion time. and total carbon emissions ;

[0108] (1)

[0109] In equation (1), Represents a product set The Middle Product Assembly completion time; Indicates the total number of moments; Indicates the processing stage The first processing plant Is the processing machine in the first... If it is in a processing state at any given moment, then let =1, otherwise, let =0; Indicates the first The first processing plant The power of a processing machine when it is in processing mode; Indicates the processing stage The first processing plant Is the processing machine in the first... If the current state is idle, then let... =1, otherwise, let =0; Indicates the first The first processing plant The power of a processing machine when it is idle; Indicates the first Carbon emission conversion factor of a processing plant.

[0110] Step 2: Construct the constraints for the distributed assembly line scheduling model with no waiting time;

[0111] Step 2.1: Construct six allocation relationship constraints between the processing plant and the workpiece using equation (2);

[0112] (2)

[0113] In equation (2), Indicates the index of the workpiece. Indicates the workpiece's machining position index; This represents the j-th workpiece. Is it assigned to the first The first processing plant If it is at a processing position, then let Otherwise, let ; Indicates the first In a processing plant, the g-th workpiece Is it in the j-th workpiece? If it was processed previously, then... Otherwise, let ; Indicates the first In a processing plant, the j-th workpiece Is it in the g-th workpiece? If it was processed previously, then... Otherwise, let ; Indicates the first In a processing plant, the j-th workpiece Is it in a virtual workpiece? If it was processed previously, then... Otherwise, let The first and second allocation constraints ensure that each workpiece j is assigned to a fixed factory f and that the position k of the workpiece's feasible scheduling sequence within that factory is unique. The third allocation constraint ensures that a virtual workpiece in the processing factory f has only one successor workpiece j. The fourth allocation constraint ensures that each workpiece j in the processing factory f has at most one direct predecessor workpiece g. The fifth allocation constraint ensures that each workpiece g in the processing factory f has at most one direct successor workpiece j. The sixth allocation constraint ensures that each workpiece j in the processing factory f cannot be both a predecessor and a successor workpiece of another workpiece g at the same time.

[0114] Step 2.2: Use equation (3) to construct 6 time constraints for the no-waiting flow workshop in the processing plant;

[0115] (3)

[0116] In equation (3), Indicates the index of the processing machine. Index representing time, Indicates the first One workpiece In the The first processing plant The time when processing begins on the processing machine; Indicates the first One workpiece In the The first processing plant The processing time completed on the processing machine; Indicates the first One workpiece In the The first processing plant The processing time completed on the processing machine; Indicates the first One workpiece In the The first processing plant Processing time on the machine; Indicates the first In the processing plant, the first One workpiece Is it in the first The moment is placed at the first Processing on a Taiwanese processing machine; if so, then... Otherwise, let L represents a positive integer; where the first time constraint is used to ensure that the workpiece in the processing plant f... In the next machine The time to start processing on the current machine is equal to the time on the previous machine. The first time constraint is the completion time of the workpiece; the second time constraint is used to ensure the completion time of the workpiece in the processing sequence in the processing plant f. In the machine The start processing time for the workpiece is not small. The completion time on the machine; the third time constraint is used to guarantee the completion time of the workpiece in the processing plant f. The completion time equals the start time plus the processing time; the fourth time constraint is used to ensure that the machine in the processing plant f completes its work. The machining time for workpiece j is equal to the actual machining time for workpiece j; the 5th and 6th time constraints are used to ensure the workpiece... At the processing plant The processing will not be interrupted.

[0117] Step 2.3: Use equation (4) to construct the eight state constraints of the processing machines in the processing plant;

[0118] (4)

[0119] In equation (4), Indicates the processing time of the processing machine; Indicates the processing stage The first processing plant Is the processing machine in the first... If a process is in a processing state at any given moment, then let... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If a processing time is idle, then let =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If the processing time is currently in a closed state, then... =1, otherwise, let =0; Indicates the next processing stage The first processing plant Is the processing machine in the first... If a process is in a processing state at any given moment, then let... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine idle at the initial processing time? If so, then... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If a processing time is idle, then let =1, otherwise, let =0; Let represent the binary auxiliary variables for the processing stage; where the first state constraint is used to ensure that the machine in the processing plant f... At the same time The machine can only be in one of three states; the second state constraint is used to ensure that the first machine in processing plant f is not idle at the initial moment; the third state constraint is used to ensure that the machine in processing plant f is in one of three states. It is not in an idle state at the last moment; the fourth state constraint is used to ensure that the machine in the processing plant f is in an idle state. At processing time t, the processing state is defined; the 5th and 6th state constraints define the processing plant. China Machinery The idle state; the 5th and 6th state constraints are used to define the processing plant. China Machinery It is in the off state.

[0120] Step 2.4: Use equation (5) to construct the four assembly sequence constraints for the products in the assembly plant;

[0121] (5)

[0122] In equation (5), Represents a product set Index of products in China; Indicates the first Product Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Indicates virtual products Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Indicates the first Product Is it in the first Product If the assembly has already begun, then... Otherwise, let The first assembly sequence constraint ensures that the virtual product has only one successor product s; the second assembly sequence constraint ensures that each product... There can be at most one direct pre-product. The third assembly sequence constraint is used to ensure that each product... There can be at most one direct successor product. The fourth assembly sequence constraint is used to ensure that each product... It cannot be another product at the same time. The predecessor and successor products.

[0123] Step 2.5: Use equation (6) to construct the seven assembly time constraints for the products in the assembly plant;

[0124] (6)

[0125] In equation (6), Indicates the first The workpiece in the first The first processing plant The processing time of a single processing machine; Indicates the first One workpiece Belongs to the Product ; Indicates the first Product The time when assembly begins on the assembly machine; Indicates the first Transportation time from the processing plant to the assembly plant; Indicates the first Product Assembly time on the assembly machine; Indicates the first Product In the If the assembly is being performed on the assembly machine at a given time, then... Otherwise, let ; Indicates virtual products The time at which assembly begins on the assembly machine; where the first assembly time constraint ensures that the virtual product's completion time is 0; the second assembly time constraint ensures that each product... Assembly can only begin after all required parts have been machined; the third assembly time constraint is used to ensure that the preceding product in a feasible product sequence... The assembly of the next product can only begin after the first assembly is completed. The fourth assembly time constraint is used to ensure product quality. The completion time equals the start time plus the assembly time; the fourth and fifth assembly time constraints are used to ensure the product's completion time. The assembly process is uninterrupted in the assembly plant; the fifth assembly time constraint ensures that the assembly time of the machine in the assembly plant equals the product's assembly time. Actual assembly time.

[0126] Step 3: Solve the distributed assembly line shop scheduling model with no waiting time using a Q-learning-based dual-population co-evolutionary algorithm to generate a distributed assembly line shop scheduling scheme with no waiting time. Specifically, as follows: Figure 2 As shown;

[0127] Step 3.1: Define the current iteration number as... and initialize The maximum number of iterations is defined as ;

[0128] Step 3.2, Set the first Generation population Each individual in the algorithm consists of a three-layer coding scheme, where the first layer of coding is composed of... The first layer of coding consists of a sequence of workpiece numbers to characterize the processing order of the workpieces; the second layer of coding consists of... A factory sequence consisting of the serial numbers of each processing plant is used to characterize the allocation of workpieces. The third layer of coding consists of... A product sequence composed of the serial numbers of each product is used to characterize the assembly order of the products; specifically, such as... Figure 3 As shown;

[0129] Some of the individuals are selected based on total processing time and total carbon emissions. The algorithm initializes the first and second layer codes; and uses the shortest processing time first algorithm to initialize the third layer code. The remaining individuals are initialized using a random generation algorithm; thus obtaining the first individual. Generation population = ,in, express The first in Individual; express Scale.

[0130] Step 3.3, for After normalization, we obtain the first... Generational normalized population ,in, express After normalization, the first individual;

[0131] Step 3.4: Calculate the first step using equation (1). individual Corresponding maximum completion time and total carbon emissions ,get Corresponding solution space coordinates ( );in, Indicates the maximum completion time , Indicates total carbon emissions ;

[0132] like Then Classified as the Left subpopulation ,like Then Classified as the upper right subpopulation ; thereby Divided into the first Subpopulation of generations ,make and Any subpopulation in the middle is denoted as , .

[0133] Step 3.5: Initialize the first equation using equations (7) and (8). The generation Subspecies Reference point ;

[0134] , (7)

[0135] , (8)

[0136] In equations (7) and (8), express The scale; express The first in individual Total carbon emissions, express The first in individual Maximum completion time; express Maximum completion time; express Total carbon emissions.

[0137] Step 3.6, Define the first The generation Subspecies Improved threshold set ,in, express The first in An improved threshold, express The scale;

[0138] Definition of the first The generation Reference points The set of moving steps ,in, express The first in Reference points The step size of movement, express The scale;

[0139] from Randomly select an improved threshold as Improved threshold ;from Randomly select a movement step size as Movement step size .

[0140] Step 3.7, determine the first offspring population The Middle individual Solution space coordinates ( ) and the offspring population Any number in the middle individual Solution space coordinates ( Does it satisfy equation (8) or equation (9)? If it does, then it means... for The non-dominant individual, otherwise, indicates To dominate individuals; thereby All non-dominant individuals constitute the first Substitute non-dominated solution set ,in, express The first in Individual, express The scale;

[0141] (9)

[0142] and (10)

[0143] Step 3.8, based on the reference point Using an improved genetic algorithm to Optimize to obtain the first The generation Optimized subpopulation In specific implementation, the improved genetic algorithm in step 3.8 is carried out according to the following process, specifically, as follows: Figure 4 As shown.

[0144] Step 3.8.1 Based on the reference point Calculate using equation (11) The Middle 1 individual fitness value ;

[0145] (11)

[0146] Step 3.8.2 will After sorting all individuals in ascending order of fitness value, the top few individuals are placed as elite individuals in the next group. The generation Elite Pool Middle; Order The Middle 1 individual The sorting number is ;

[0147] Step 3.8.3 Calculate using equation (12) The Middle individual Probability of being selected and according to The first parent individual is obtained by using a roulette wheel. Second parent individual ;

[0148] (12)

[0149] In equation (12), Indicates the probability of choosing to regulate. express of Power of 1.

[0150] Step 3.8.4 Select and The first layer of encoding randomly generates the intersection interval [ , ], and copy The first layer of coding interval [ , The workpiece sequence is used as the offspring individual. The first layer of coding interval [ , The sequence of workpieces;

[0151] Establish and The first layer of coding interval [ , Mapping relationship, take In the first layer of coding, non-range [ , The workpiece serial number is assigned to the child individual through a mapping relationship. The first layer of coding interval [ , The position of ] is used to obtain the offspring individuals. The first-level encoding of the offspring;

[0152] Take respectively and After the second layer of encoding and two-point crossover, offspring individuals are obtained. The second layer of encoding;

[0153] Use the shortest processing time first algorithm to evaluate offspring individuals. The first and second layer codes are processed to obtain offspring individuals. The third layer of encoding, thus completing the offspring individual The three-layer encoding.

[0154] Step 3.8.5 Mutate the offspring individual C with a certain probability. If mutation occurs, randomly select a workpiece from the first layer of C's encoding and move it to a random position. Also, randomly select a workpiece from the second layer of C's encoding and move it to another processing plant, thus obtaining a new offspring individual C'. Otherwise, directly... Assign it to C';

[0155] Step 3.8.6 Perform a local search on C' to obtain the new individual #1. The new second individual The new third individual The new fourth individual ; and calculate respectively , , The fitness value of the new individual is compared with the fitness value of C'. If the fitness value of the new individual exceeds that of C', then the corresponding new individual replaces C'; otherwise, it is not replaced.

[0156] Randomly select a workpiece serial number from within the factory from C' and exchange it to obtain... ;

[0157] Randomly select a workpiece number from C' within the factory sequence and insert it into the position of other workpiece sequences to obtain... ;

[0158] Randomly select any two factory workpiece serial numbers from C' and swap them to obtain... ;

[0159] Randomly select a workpiece serial number from C' within a factory and insert it into a random position in another factory, to obtain... .

[0160] Step 3.8.7 Add C' to the first... The generation intermediate offspring population Then, and judge Has the required scale been reached? If so, proceed to step 3.8.8; otherwise, return to step 3.8.3 and execute sequentially.

[0161] Step 3.8.8 will and After merging, select the quantity as The most fit individual forms .

[0162] Step 3.9: Calculate the first step using equation (13). Optimized subpopulation The Middle individual and the individual Solution space distance between Thus, the dimension is triangular distance matrix and from The parameter obtained from the solution space distance is less than the population maintenance distance. The Individuals of a generation to a set Then, proceed according to step 3.8. Each individual in the process optimizes against the individuals with poor fitness, thus obtaining the [number]th [unit]. The updated set of individual pairs ; and based on renew , obtained the The generation One maintenance subpopulation ;

[0163] = (13)

[0164] In equation (13), express The first in individual Maximum completion time express The first in individual Total carbon emissions; express The first in individual Maximum completion time express The first in individual Total carbon emissions.

[0165] Step 3.10, place the first Sub-population maintenance Non-dominated solution set After merging and removing duplicates, the optimized top-left subpopulation is obtained. and the optimized lower right subpopulation ,Will and Any optimized subpopulation is denoted as the i-th An optimized subpopulation Then, proceed according to step 3.7. and Process the solution to obtain the optimized set of non-dominated solutions. ,in, express The first in Individual, express Scale.

[0166] Step 3.11: Calculate using equations (14) and (15) Normalized hypervolume value and Normalized hypervolume value ,

[0167] (14)

[0168] (15)

[0169] In equations (14) and (15), Indicates the first An optimized subpopulation Nondominated solution set The first in individual Maximum completion time; Indicates the first An optimized subpopulation Nondominated solution set The first in individual Total carbon emissions; Indicates the first An optimized subpopulation Optimized nondominated solution set The first in individual Maximum completion time; Indicates the first An optimized subpopulation Optimized nondominated solution set The first in individual Total carbon emissions.

[0170] Step 3.12, using equation (16) to obtain and degree of improvement ,like Then, using equations (17) and (18) to apply the first... Reference point Update to obtain the first Reference point , order the An optimized subpopulation Comparison reference points = Even if the comparison reference point Maximum completion time = ,make Total carbon emissions = Otherwise, directly Assign to ;

[0171] (16)

[0172] (17)

[0173] (18)

[0174] In equations (17) and (18), express Maximum completion time; express Total carbon emissions.

[0175] Step 3.13, if =0, then set the first The generation Subspecies Q-learning controller and the Q-learning controller Initialize the parameters;

[0176] Step 3.14, if the first Reference point With the Reference point If they are different, use the Q-learning controller. Get the The generation Subgroup Reference point Movement step size and improved threshold Otherwise, directly Assign to ,Will Assign to .

[0177] In this embodiment, the Q-learning controller is used in the following manner:

[0178] Step a: Obtain the comparison reference point using equation (19) With the current iteration number Deviation between ;

[0179] (19)

[0180] Step b: Define the Gth generation offspring population state set Let the state set Any state is ,in, ;

[0181] when hour, For state ;

[0182] when hour, For state ;

[0183] when )hour, For state .

[0184] Step c: Calculate the first step using equation (20). offspring population State interval value ;

[0185] (20)

[0186] Step d, will and After orthogonalization, the first... The generation Action set ,in, express The first in Each action includes improving the threshold. and moving step size , express Scale.

[0187] Step e: Calculate the first step using equations (21) and (22). The generation Reward Value and according to To update the Q table; thereby utilizing Greedy strategy to obtain the first The generation Subgroup action ,make Improved threshold and Movement step size ;

[0188] (twenty one)

[0189] In equations (21) and (22), Let be the standard deviation in a normal distribution, and we have:

[0190] (twenty two).

[0191] Step 3.15, if G Then output All non-dominated individuals are considered as the final production and processing scheme; otherwise, the first... An optimized subpopulation As the first Subpopulation of generations ,make +1 is assigned to Then, return to step 3.7.

[0192] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the methods described above, and the processor is configured to execute the program stored in the memory.

[0193] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A scheduling method for a distributed assembly line shop with no waiting time based on Q-learning, characterized in that, The distributed assembly line with no waiting time includes a processing stage and an assembly stage, wherein the processing stage is composed of... A processing plant uses processing machines to collect the workpieces to be processed. After processing, the materials are transported to the assembly plant. During the assembly stage, one assembly plant uses one assembly machine to... Individual workpieces are assembled into a product assembly. ;in, Indicates the number to be processed One workpiece, express The total number of workpieces; express The Middle One product Represents a product set The total number of products in the process ensures that each processing plant has A set of processing machines is provided, with each machine having a state including processing, idle, and off. The power output of each machine varies depending on its state, allowing workpieces to be processed on any machine in any processing plant, and ensuring that at any given time, a workpiece is processed on only one machine. The scheduling method is performed as follows: Step 1, constructing the objective function of the distributed assembly no-wait flow shop scheduling model, including: minimizing the maximum completion time and total carbon emissions ; Step 2: Construct the constraints for the distributed assembly line scheduling model with no waiting time; Step 3: Use the Q-learning-based dual-population co-evolutionary algorithm to solve the distributed assembly line workshop scheduling model with no waiting time, and generate a distributed assembly line workshop scheduling scheme with no waiting time.

2. The Q-learning based distributed assembly no-wait job shop scheduling method according to claim 1, wherein, In step 1, the objective function is obtained using equation (1): (1) In equation (1), Represents a product set The Middle Product Assembly completion time; Indicates the total number of moments; Indicates the processing stage The first of the processing plants Is the processing machine in the first... If it is in a processing state at any given moment, then let =1, otherwise, let =0; Indicates the first The first processing plant The power of a processing machine when it is in processing mode; Indicates the processing stage The first processing plant Is the processing machine in the first... If the current state is idle, then let... =1, otherwise, let =0; Indicates the first The first processing plant The power of a processing machine when it is idle; Indicates the first Carbon emission conversion factor of a processing plant.

3. The Q-learning based distributed assembly no-wait job shop scheduling method of claim 2, wherein, Step 2 includes: Step 2.1: Use equation (2) to construct the allocation relationship constraints between the processing plant and the workpiece; (2) In formula (2), represents an index of a workpiece, represents a machining position index of a workpiece; represents the jth workpiece whether or not to be allocated at the ith machining position of the jth machining factory, if yes, then let , otherwise, let ; ; ; represents whether or not the gth workpiece is machined before the jth workpiece in the ith machining factory, if yes, then let , otherwise, let ; ; represents whether or not the jth workpiece is machined before the gth workpiece in the ith machining factory, if yes, then let , otherwise, let ; ; represents whether or not the jth workpiece is machined before the virtual workpiece in the ith machining factory, if yes, then let , otherwise, let ; ; Step 2.2: Use equation (3) to construct the time constraints of the processing plant's no-waiting assembly line workshop; (3) In formula (3), represents an index of a processing machine, represents an index of a time, represents the workpiece starts processing at the processing machine in the processing factory; represents the workpiece finishes processing at the processing machine in the processing factory; represents the workpiece finishes processing at the processing machine in the processing factory; represents the workpiece processing time at the processing machine in the processing factory; represents whether the workpiece is processed at the processing machine at the time in the processing factory, if yes, then let , otherwise, let ; L represents a positive integer; Step 2.3: Use equation (4) to construct the state constraints of the processing machines in the processing plant; (4) In equation (4), Indicates the processing time of the processing machine; Indicates the processing stage The first processing plant Is the processing machine in the first... If a process is in a processing state at any given moment, then let... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If a processing time is idle, then let =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If the processing time is currently in a closed state, then... =1, otherwise, let =0; Indicates the next processing stage The first of the processing plants Is the processing machine in the first... If a process is in a processing state at any given moment, then let... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine idle at the initial processing time? If so, then... =1, otherwise, let =0; Indicates the processing stage The first processing plant Is the processing machine in the first... If a processing time is idle, then let =1, otherwise, let =0; A binary auxiliary variable representing the processing stage; Step 2.4: Use equation (5) to construct the assembly sequence constraints of products in the assembly plant; (5) In equation (5), Represents a product set Index of products in China; Indicates the first Product Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Indicates virtual products Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Indicates the first Product Is it in the first Product If the assembly has already begun, then... Otherwise, let ; Step 2.5: Use equation (6) to construct the assembly time constraints for products in the assembly plant; (6) In formula (6), denotes the time at which the workpiece was completed by the machining machine in the machining factory; denotes the time at which the workpiece belongs to the product ; denotes the time at which the product was started on the assembly machine; denotes the transport time from the machining factory to the assembly factory; denotes the assembly time of the product on the assembly machine; denotes whether the product was assembled on the assembly machine at the time, and if so, then , and otherwise ; denotes the time at which the virtual product was started on the assembly machine.

4. The Q-learning based distributed assembly no-wait job shop scheduling method of claim 3, wherein, Step 3 is performed as follows; Step 3.1, define the current iteration number as and initialize , define the maximum iteration number as ; Step 3.2, set the first generation population Each individual in the population is composed of three layers of encoding, where the first layer of encoding is a workpiece sequence consisting of the serial numbers of the workpieces to represent the processing order of the workpieces; the second layer of encoding is a factory sequence consisting of the serial numbers of the processing factories to represent the allocation of the workpieces; and the third layer of encoding is a product sequence consisting of the serial numbers of the products to represent the assembly order of the products; Some of the individuals are selected based on total processing time and total carbon emissions. The algorithm initializes the first and second layer codes; and uses the shortest processing time first algorithm to initialize the third layer code. The remaining individuals are initialized using a random generation algorithm; thus obtaining the first individual. Generation population = ,in, express The first in Individual; express The scale; Step 3.3, to perform normalization, to obtain the normalized population wherein, denotes the normalized individual; Step 3.4, calculating the first maximum completion time for each individual and the total carbon emission , obtaining the corresponding solution space coordinates ; wherein, represents the maximum completion time , represents the total carbon emission ; If , then is classified into the th left upper subpopulation ; if , then is classified into the th right upper subpopulation ; thereby is divided into the th subpopulation , let any one of and be denoted as , ; Step 3.5: Initialize the first equation using equations (7) and (8). The generation Subspecies Reference point ; ， (7) ， (8) in formula (7) and formula (8), denotes the scale of; denotes the total carbon emissions of the th individual in denotes the maximum completion time of the th individual in denotes the maximum completion time of denotes the total carbon emissions of Step 3.6, define the improved threshold set for the th sub-population wherein th improved threshold value in the th sub-population th improved threshold value in the th sub-population th improved threshold value in the th sub-population size of the th sub-population Definition of the first reference point of the set of movement steps wherein denotes the movement step of the first reference point of the set of movement steps, denotes the size of the from a set of improvement thresholds as the improvement threshold for from a set of move steps as the move step for from a set of move steps as the move step for ​ Step 3.7, determine the first offspring population The Middle individual Solution space coordinates ( ) and the offspring population Any of the following individual Solution space coordinates ( Does it satisfy equation (8) or equation (9)? If it does, then it means... for The non-dominant individual, otherwise, indicates To dominate individuals; thereby All non-dominant individuals constitute the first Substitute non-dominated solution set ,in, express The first in Individual, express The scale; (9) and (10) Step 3.8, based on the reference point Using an improved genetic algorithm to Optimize to obtain the first The generation Optimized subpopulation ; Step 3.9: Calculate the first step using equation (13). Optimized subpopulation The Middle individual and the individual Solution space distance between Thus, the dimension is triangular distance matrix and from The parameter obtained from the solution space distance is less than the population maintenance distance. The Individuals of a generation to a set Then, proceed according to step 3.

9. Each individual in the process optimizes against the individuals with poor fitness, thus obtaining the [number]th [unit]. The updated set of individual pairs ; and based on renew , obtained the The generation One maintenance subpopulation ; = (13) In equation (13), express The first in individual Maximum completion time express The first in individual Total carbon emissions; express The first in individual Maximum completion time express The first in individual Total carbon emissions; Step 3.10, place the first Sub-population maintenance Non-dominated solution set After merging and removing duplicates, the optimized top-left subpopulation is obtained. and the optimized lower right subpopulation ,Will and Any optimized subpopulation is denoted as the i-th An optimized subpopulation Then, proceed according to step 3.

8. and Process the solution to obtain the optimized set of non-dominated solutions. ,in, express The first in Individual, express The scale; Step 3.11: Calculate using equations (14) and (15) Normalized hypervolume value and Normalized hypervolume value , (14) (15) In equations (14) and (15), Indicates the first An optimized subpopulation Nondominated solution set The first in individual Maximum completion time; Indicates the first An optimized subpopulation Nondominated solution set The first in individual Total carbon emissions; Indicates the first An optimized subpopulation Optimized nondominated solution set The first in individual Maximum completion time; Indicates the first An optimized subpopulation Optimized nondominated solution set The first in individual Total carbon emissions; Step 3.12, using equation (16) to obtain and degree of improvement ,like Then, using equations (17) and (18) to apply the first... Reference point Update to obtain the first Reference point , order the An optimized subpopulation Comparison reference points = Even if the comparison reference point Maximum completion time = ,make Total carbon emissions = Otherwise, directly Assign to ; (16) (17) (18) In equations (17) and (18), express Maximum completion time; express Total carbon emissions; Step 3.13, if =0, then set the first The generation Subspecies Q-learning controller and the Q-learning controller Initialize the parameters; Step 3.14, if the first Reference point With the Reference point If they are different, use the Q-learning controller. Get the The generation Subgroup Reference point Movement step size and improved threshold Otherwise, directly Assign to ,Will Assign to ; Step 3.15, if G Then output All non-dominated individuals are considered as the final production and processing scheme; otherwise, the first... An optimized subpopulation As the first Subpopulation of generations ,make +1 is assigned to Then, return to step 3.

7.

5. The distributed assembly shop scheduling method based on Q-learning as described in claim 4, characterized in that, Step 3.8 includes: Step 3.8.1 Based on the reference point Calculate using equation (11) The Middle 1 individual fitness value ; (11) Step 3.8.2 will After sorting all individuals in ascending order of fitness value, the top few individuals are placed as elite individuals in the next group. The generation Elite Pool Middle; Order The Middle 1 individual The sorting number is ; Step 3.8.3 Calculate using equation (12) The Middle individual Probability of being selected and according to The first parent individual is obtained by using a roulette wheel. Second parent individual ; (12) In equation (12), Indicates the probability of choosing to regulate. express of Power; Step 3.8.4 Select and The first layer of encoding randomly generates the intersection interval [ , ], and copy The first layer of coding interval [ , The workpiece sequence is used as the offspring individual. The first layer of coding interval [ , The sequence of workpieces; Establish and The first layer of coding interval [ , Mapping relationship, take In the first layer of coding, non-range [ , The workpiece serial number is assigned to the child individual through a mapping relationship. The first layer of coding interval [ , The position of ] is used to obtain the offspring individuals. The first-level encoding of the offspring; Take respectively and After the second layer of encoding and two-point crossover, offspring individuals are obtained. The second layer of encoding; Use the shortest processing time first algorithm to evaluate offspring individuals. The first and second layer codes are processed to obtain offspring individuals. The third layer of encoding, thus completing the offspring individual Three-layer encoding; Step 3.8.5 Mutate the offspring individual C with a certain probability. If mutation occurs, randomly select a workpiece from the first layer of C's encoding and move it to a random position. Also, randomly select a workpiece from the second layer of C's encoding and move it to another processing plant, thus obtaining a new offspring individual C'. Otherwise, directly... Assign it to C'; Step 3.8.6 Perform a local search on C' to obtain the new individual #1. The new second individual The new third individual The new fourth individual ; and calculate respectively , , The fitness value of the new individual is compared with the fitness value of C'. If the fitness value of the new individual exceeds that of C', then the corresponding new individual replaces C'; otherwise, it is not replaced. Randomly select a workpiece serial number from within the factory from C' and exchange it to obtain... ; Randomly select a workpiece number from C' within the factory sequence and insert it into the position of other workpiece sequences to obtain... ; Randomly select any two factory workpiece serial numbers from C' and swap them to obtain... ; Randomly select a workpiece serial number from C' within a factory and insert it into a random position in another factory, to obtain... ; Step 3.8.7 Add C' to the first... The generation intermediate offspring population Then, and judge Has the required scale been reached? If so, proceed to step 3.8.8; otherwise, return to step 3.8.3 and execute sequentially. Step 3.8.8 will and After merging, select the quantity as The most fit individual forms .

6. The distributed assembly shop scheduling method based on Q-learning as described in claim 5, characterized in that, In step 3.14, the Q-learning controller is used according to the following process: Step a: Obtain the comparison reference point using equation (19) With the current iteration number Deviation between ; (19) Step b: Define the Gth generation offspring population state set Let the state set Any state is ,in, ; when hour, For state ; when hour, For state ; when )hour, For state ; Step c: Calculate the first step using equation (20). offspring population State interval value ; (20) Step d, will and After orthogonalization, the first... The generation Action set ,in, express The first in Each action includes improving the threshold. and moving step size , express The scale; Step e: Calculate the first step using equations (21) and (22). The generation Reward Value and according to To update the Q table; thereby utilizing Greedy strategy to obtain the first The generation Subgroup action ,make Improved threshold and Movement step size ; (21) In equations (21) and (22), Let be the standard deviation in a normal distribution, and we have: (22)。 7. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store programs that support the processor in executing the Q-learning-based distributed assembly waiting flow shop scheduling method according to any one of claims 1-6, wherein the processor is configured to execute the programs stored in the memory.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program, when run by the processor, executes the steps of the low Q-based learning distributed assembly waiting flow shop scheduling method according to any one of claims 1-6.

Images