Semiconductor manufacturing scheduling method, system and computer readable storage medium

Abstract

The present invention discloses a semiconductor manufacturing production scheduling method, system and computer-readable storage medium, the method comprising the following steps: determining a mathematical optimization model according to the data scale; generating a first scheduling sequence according to the mathematical optimization model, iteratively calculating the first scheduling sequence under constraints through the mathematical optimization model to obtain a second scheduling sequence; determining an optimal strategy based on the operation results of each training action, determining a target overall strategy according to the optimal strategy corresponding to each state parameter, and performing simulation scheduling according to the target overall strategy to obtain a third scheduling sequence; and performing scheduling operations according to the third scheduling sequence. The present invention realizes rapid target optimization of semiconductor manufacturing production scheduling, effectively and reasonably allocates production capacity, improves resource utilization, and reduces production costs.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor manufacturing technology, and in particular to a semiconductor manufacturing scheduling method, system, and computer-readable storage medium. Background Art

[0002] Semiconductor manufacturing is a complex industrial manufacturing chain that typically includes four production stages: wafer fabrication, wafer sorting, packaging, and product testing. Wafer fabrication is a complex task, requiring hundreds of process steps to manufacture machines and silicon wafers. Small quantities of products are produced across hundreds or even thousands of processes within a factory, necessitating complex capacity allocation and job scheduling issues. Scheduling, in other words, is a matter of great concern in semiconductor manufacturing. Scheduling generally involves the assembly line processing of n workpieces on m machines. Each workpiece takes a different amount of time to run on each machine, and each machine can only process one workpiece at a time. The goal of scheduling is to determine the order in which the workpieces are processed on each machine and the start time of each process, minimizing the maximum completion time or optimizing other metrics.

[0003] Currently, an existing semiconductor production scheduling method uses an intelligent optimization algorithm constructed based on evolutionary laws, such as a genetic algorithm, for query. However, this method requires constructing a sequence population and performing optimization iterations within the population. It requires separate calculations for each individual in the population. The larger the population, the greater the amount of calculation, resulting in a very time-consuming solution. Summary of the Invention

[0004] The purpose of the present invention is to provide a semiconductor manufacturing scheduling method, system and computer-readable storage medium, which can achieve effective and reasonable allocation of production capacity, improve resource utilization and reduce production costs.

[0005] To achieve the above object, the present invention provides a semiconductor manufacturing production scheduling method, which includes the following steps:

[0006] Obtaining a data scale for semiconductor manufacturing production scheduling, and determining a mathematical optimization model based on the data scale;

[0007] generating a first scheduling sequence according to the mathematical optimization model, and performing iterative calculation on the first scheduling sequence under constraints using the mathematical optimization model to obtain a second scheduling sequence, wherein the second scheduling sequence is a scheduling sequence with a maximum fitness value under the constraints, wherein the constraints include a maximum number of iterations;

[0008] determining all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traversing each of the state parameters, and determining all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies;

[0009] Running the training actions in each of the traversal initial strategies, determining an optimal strategy based on the running results of each of the training actions, determining a target overall strategy based on the optimal strategies corresponding to each of the state parameters, and performing simulation scheduling based on the target overall strategy to obtain a third scheduling sequence;

[0010] The scheduling operation is performed according to a third scheduling order sequence.

[0011] Furthermore, in the above-mentioned semiconductor manufacturing scheduling method, the data scale includes the number of products to be processed.

[0012] Furthermore, in the semiconductor manufacturing scheduling method, the step of iteratively calculating the first scheduling sequence under constraints using the mathematical optimization model to obtain the second scheduling sequence specifically includes:

[0013] When the data size does not exceed a preset threshold, two random integers are generated, where the two random integers are different integers between 1 and N, where N is the length of the first scheduling sequence;

[0014] Swapping the values ​​of the sequence positions corresponding to the two random integers in the first scheduling sequence to obtain a first comparison sequence;

[0015] Calculate and compare the fitness value of the first scheduling order sequence with the fitness value of the first comparison order sequence; when the fitness value of the first comparison order sequence is greater than the fitness value of the first scheduling order sequence, repeat the step of exchanging the values ​​of the first comparison order sequence until a maximum number of iterations in the constraint condition is satisfied, thereby obtaining a second scheduling order sequence, where the second scheduling order sequence is the scheduling order sequence with the maximum fitness value under the constraint condition.

[0016] Furthermore, in the semiconductor manufacturing scheduling method, the step of iteratively calculating the first scheduling sequence under constraints using the mathematical optimization model to obtain the second scheduling sequence specifically includes:

[0017] When the data size exceeds a preset threshold, performing roulette wheel screening, crossover operation, and mutation operation on the first scheduling sequence under constraints to obtain multiple optimal sequences, wherein the constraints include the maximum number of iterations, the number of individuals in the population, the crossover probability, and the mutation probability;

[0018] The second scheduling sequence is determined according to a plurality of preferred sequences, and the second scheduling sequence is a preferred sequence having the largest fitness value among the plurality of preferred sequences.

[0019] Furthermore, in the semiconductor manufacturing scheduling method, the step of performing a roulette wheel screening on the first scheduling sequence under a constraint condition specifically includes:

[0020] Calculating fitness values ​​of multiple random order sequences in the first scheduling order sequence respectively, and constructing a normalized interval according to the ratio of the fitness value of each random order sequence to the sum of the fitness values ​​of all random order sequences, wherein the normalized interval includes multiple subintervals corresponding to each fitness value, and the value range of the normalized interval is [0, 1];

[0021] Generate a plurality of different random numbers between 0 and 1, wherein the number of the random numbers is the same as the number of the plurality of random sequence sequences;

[0022] In the normalized interval, the subinterval into which each random number falls is determined, and the corresponding random sequence is selected according to the fitness value of the subinterval into which the random number falls, so as to obtain a plurality of random sequence sequences after screening.

[0023] Furthermore, in the semiconductor manufacturing scheduling method, the step of performing a crossover operation on the first scheduling sequence under a constraint condition specifically includes:

[0024] Determine the random sequence sequences to be crossed according to the crossover probability and the multiple random sequence sequences after screening, and pair the random sequence sequences to be crossed in pairs to obtain multiple pairs of sequence groups, each pair of sequence groups including two random sequence sequences;

[0025] Generate a random integer between 1 and N, where N is the length of the first scheduled sequence, and swap the forward or reverse values ​​of two random sequence sequences in the sequence group starting from the random integer position to obtain multiple random sequence sequences after crossing.

[0026] Furthermore, in the semiconductor manufacturing scheduling method, the step of performing a mutation operation on the first scheduling sequence under a constraint condition specifically includes:

[0027] Determine the random sequence to be mutated according to the mutation probability and multiple random sequence sequences after crossover;

[0028] Generate two random integers between 1 and N, where N is the length of the first scheduled order sequence, and randomly sort the subsequences between the corresponding positions of the two random integers in the random order sequence to be mutated to obtain multiple mutated random order sequences;

[0029] An optimal sequence is calculated based on the multiple mutated random order sequences, and the optimal sequence is a sequence with the largest fitness value among the multiple mutated random order sequences.

[0030] Furthermore, in the above-mentioned semiconductor manufacturing scheduling method, the training action includes delivery sequence, customer level, arrival time of the product from the buffer zone, and initial sequence of the product.

[0031] In addition, the present invention further provides a semiconductor manufacturing scheduling system for implementing the above-mentioned semiconductor manufacturing scheduling method, comprising:

[0032] A first determining unit is used to determine a mathematical optimization model according to the data scale of semiconductor manufacturing scheduling;

[0033] a first calculation unit, configured to generate a first scheduling sequence according to the mathematical optimization model, and to perform iterative calculation on the first scheduling sequence under constraints using the mathematical optimization model to obtain a second scheduling sequence, wherein the second scheduling sequence is a scheduling sequence with a maximum fitness value under the constraints, wherein the constraints include a maximum number of iterations;

[0034] a second determining unit, configured to determine a target attribute priority of the product to be processed;

[0035] a second computing unit configured to determine all state parameters of the semiconductor manufacturing environment and training actions of the second scheduling sequence, traverse each of the state parameters, and determine all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies; execute the training actions in each of the traversal initial strategies, determine an optimal strategy based on the execution results of each of the training actions, determine a target overall strategy based on the optimal strategies corresponding to each of the state parameters, and perform simulation scheduling based on the target overall strategy to obtain a third scheduling sequence;

[0036] A scheduling unit is used to perform scheduling operations according to a third scheduling order sequence.

[0037] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the above-mentioned semiconductor manufacturing scheduling method.

[0038] By combining the global search capability of intelligent algorithms with the local optimization techniques of reinforcement learning algorithms, the present invention achieves rapid target optimization of semiconductor manufacturing scheduling, effectively and reasonably allocates production capacity, improves resource utilization, and reduces production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a semiconductor manufacturing production scheduling method according to an embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of the calculation and estimation of the fitness value of the scheduling sequence in the present invention;

[0041] Figure 3is a schematic diagram of the first mathematical optimization model of the present invention;

[0042] Figure 4 is a schematic diagram of the second mathematical optimization model of the present invention;

[0043] Figure 5 Schematic diagram of a semiconductor manufacturing production scheduling method according to the present invention;

[0044] Figure 6 It is a schematic diagram of the principle of reinforcement learning training in the present invention;

[0045] Figure 7 Schematic diagram of the Q-value table of the strategy in the reinforcement learning training of the present invention;

[0046] Figure 8 This is a flow chart of deep regression model training in reinforcement learning training of the present invention;

[0047] Figure 9 1 is a schematic structural diagram of a semiconductor manufacturing scheduling system according to a semiconductor manufacturing scheduling method according to an embodiment of the present invention;

[0048] Figure 10 yes Figure 9 Schematic diagram of the structure of the first computing unit in . DETAILED DESCRIPTION

[0049] This embodiment takes a semiconductor manufacturing production scheduling method and system as an example, and the present invention will be described in detail below with reference to specific embodiments and drawings.

[0050] An embodiment of the present invention provides a semiconductor manufacturing production scheduling method, comprising the following steps:

[0051] The invention provides a method for obtaining a data scale for semiconductor manufacturing scheduling, determining a mathematical optimization model based on the data scale, generating a first scheduling sequence based on the mathematical optimization model, iterating the first scheduling sequence under constraints using the mathematical optimization model to obtain a second scheduling sequence, wherein the second scheduling sequence is the scheduling sequence with the maximum fitness value under the constraints, wherein the constraints include a maximum number of iterations, determining all state parameters of the semiconductor manufacturing environment and training actions of the second scheduling sequence, traversing each of the state parameters, and determining all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies, running the training actions in each of the traversal initial strategies, determining the optimal strategy based on the running results of each of the training actions, determining a target overall strategy based on the optimal strategy corresponding to each of the state parameters, and performing simulation scheduling based on the target overall strategy to obtain a third scheduling sequence, and performing scheduling operations based on the third scheduling sequence. The invention combines the global search capability of an intelligent algorithm with the local optimization technique of a reinforcement learning algorithm to achieve rapid target optimization of semiconductor manufacturing scheduling, effectively and reasonably allocate production capacity, improve resource utilization, and reduce production costs.

[0052] See also Figures 1 to 6 , an embodiment of the present invention provides a semiconductor manufacturing production scheduling method, the method specifically comprising the following steps:

[0053] Step S11: Initializing the simulation of the semiconductor manufacturing scene;

[0054] In practical implementation, for example, in semiconductor manufacturing, scheduling the entire process from ingot to wafer, the real-world environment can be abstracted into an entry point for releasing products, an exit point for counting completed products, two buffers for storing products, and various processing machines. The entry point is the module where orders are generated, and the exit point is the collection point for completed products. Idle machines in different factory areas retrieve appropriate products from the same buffer. Understandably, the various components and quantities in each scenario may vary.

[0055] Step S12: Acquire the data scale of semiconductor manufacturing scheduling, and determine a mathematical optimization model according to the data scale;

[0056] See also Figure 2In its specific implementation, the present invention achieves an optimal semiconductor scheduling strategy through a two-level optimization process. The first level of optimization uses an intelligent algorithm to calculate the fitness values ​​corresponding to various semiconductor scheduling sequences, attempting to identify a more optimal sequence. The computational complexity of this intelligent algorithm is related to the data size of semiconductor manufacturing scheduling. Therefore, it is necessary to first obtain the data size used to evaluate semiconductor manufacturing scheduling and then determine a mathematical optimization model based on this data size. In this embodiment, the data size is the number of products to be processed, but it can also be other conditional data such as the total processing time. In the semiconductor production process, the number of products to be processed in a batch is determined in advance according to the production plan, mainly based on the short-term production plan or the medium-term production plan. For example, production is carried out in batches and according to different plans according to customer demand. Before a batch of 500 products is scheduled for production as scheduled, the number of products, 500, is determined in advance. Depending on the number of products, the amount of calculation of the scheduling calculation adaptation value F1 is different, and the mathematical optimization model applied is also different. Specifically, a threshold can be set based on experience or big data. If the data scale does not exceed the threshold, the first mathematical optimization model is selected for application. If the data scale exceeds the threshold, the second mathematical optimization model is selected for application.

[0057] That is, the step of determining the mathematical optimization model according to the data scale specifically includes:

[0058] When the data scale does not exceed the preset threshold, the first mathematical optimization model is selected for application; when the data scale exceeds the preset threshold, the second mathematical optimization model is selected for application.

[0059] In this embodiment, the threshold is 500, the first mathematical optimization model is a hill climbing algorithm, and the second mathematical optimization model is a genetic algorithm. Specifically, in this embodiment, when the number of products to be processed does not exceed 500, the hill climbing algorithm is applied; when the number of products to be processed exceeds 500, the genetic algorithm is applied. This allows the appropriate mathematical optimization model to be selected based on different data sizes and computational workloads, resulting in the fastest and most efficient model optimization and production scheduling.

[0060] Step S13: generating a first scheduling sequence according to the mathematical optimization model;

[0061] In a specific implementation, when the data size does not exceed a preset threshold, a first mathematical optimization model is selected for application. In this embodiment, the first mathematical optimization model is a hill climbing algorithm model, the data size is the number of products to be processed, and a first scheduling sequence is generated based on the number of products to be processed. The length of the first scheduling sequence is the number of products to be processed. Specifically, when the hill climbing algorithm model is selected for application, a first scheduling sequence P0 of length N is generated based on the number of products to be processed N. The first scheduling sequence P0 is a random sequence.

[0062] When the data scale exceeds a preset threshold, the second mathematical optimization model is selected for application. In this embodiment, the second mathematical optimization model is a genetic algorithm model, the data scale is the number of products to be processed, and a first scheduling sequence is generated based on the number of products to be processed and the number of individuals in the population. The first scheduling sequence includes multiple random sequence sequences, the number of random sequence sequences is the number of individuals in the population, and the length of the random sequence sequence is the number of products to be processed. Specifically, when the genetic algorithm model is selected for application, m random sequence sequences L with a length of N are generated based on the number of products to be processed N and the number of individuals in the population m. 11 , L 12 ,…,L 1m ;Remember i=1.

[0063] Step S14: performing iterative calculation on the first scheduling sequence under constraints using the mathematical optimization model to obtain a second scheduling sequence, wherein the second scheduling sequence is a scheduling sequence with a maximum fitness value under the constraints, wherein the constraints include a maximum number of iterations;

[0064] For specific implementation, please refer to Figure 3 When the data scale does not exceed a preset threshold, the first mathematical optimization model is a hill climbing algorithm. In this case, the constraint is the maximum number of iterations n, which is estimated based on experience and target accuracy. After generating a first scheduling sequence P0 of length N, for example, the first scheduling sequence P0 is 2-3-1-7-6-4-8-5, two random integers between 1 and N are generated, and the values ​​corresponding to the two random integer positions in P0 are swapped, denoted as P1. For example, two random integers 3 and 4 are generated, and the values ​​at positions 3 and 4 of the first scheduling sequence P0 are swapped, resulting in a first comparison sequence P1 of 2-3-7-1-6-4-8-5. Next, the fitness value F0 of the first scheduling sequence P0 and the fitness value F1 of the first comparison sequence P1 are calculated and compared.

[0065] If F0F1, it means that the first comparison order sequence P1 is not better than the first scheduling order sequence P0. In this case, it is necessary to repeat the step of exchanging the values ​​of the two random integer positions until a first comparison order sequence P1 is found that is better than the first scheduling order sequence P0.

[0066] If F0 < F1, it indicates that the first comparison sequence P1 is better. At this time, continue to generate two random integers between 1 and N, and then swap the values at the positions corresponding to the two random integer serial numbers in P1, denoted as P2. For example, generate two random integers 6 and 7. At this time, swap the values at the 6th and 7th positions in the first scheduling sequence P1, and the second comparison sequence P2 obtained is 2 - 3 - 7 - 1 - 6 - 8 - 4 - 5. Then, calculate and compare the fitness value F1 of the first comparison sequence P1 and the fitness value F2 of the second comparison sequence P2,... and so on. Generate two random integers between 1 and N, and then swap the values at the positions corresponding to the two random integer serial numbers in P i to obtain P i+1 , until the constraint condition of the maximum number of iterations n (i > n) is satisfied, stop the iteration, that is, calculate and obtain the second scheduling sequence P n , and the second scheduling sequence P n is the scheduling sequence with the maximum fitness value under the maximum number of iterations n.

[0067] For the first scheduling sequence L = [l0, l1, l2,..., l N-1 , its fitness function F[L] is:

[0068]

[0069] where P(li) is the total processing time of this product batch, D(li) is the due date of this product batch, Begin is the initial time, N is the length of the first scheduling sequence, and l0, l1, l2,..., l N-1 are the products in the first scheduling sequence.

[0070] Under the same other conditions, the larger the fitness value of a product batch sequence, the higher its scheduling efficiency and the more optimized and reasonable it is.

[0071] That is, the specific steps of step S14 include:

[0072] When the data scale does not exceed the preset threshold, generate two random integers, and the two random integers are different integers between 1 and N, and N is the length of the first scheduling sequence;

[0073] Swap the values at the positions corresponding to the two random integers in the first scheduling sequence to obtain the first comparison sequence;

[0074] Calculate and compare the fitness value of the first scheduling order sequence with the fitness value of the first comparison order sequence; when the fitness value of the first comparison order sequence is greater than the fitness value of the first scheduling order sequence, repeat the step of exchanging the values ​​of the first comparison order sequence until a maximum number of iterations in the constraint condition is satisfied, thereby obtaining a second scheduling order sequence, where the second scheduling order sequence is the scheduling order sequence with the maximum fitness value under the constraint condition.

[0075] The step S14 further includes:

[0076] When the fitness value of the first comparison order sequence is not greater than the fitness value of the first scheduling order sequence, the above steps of generating the first comparison order sequence are repeated until the fitness value of the obtained first comparison order sequence is greater than the fitness value of the first scheduling order sequence.

[0077] See also Figure 4 When the data size does not exceed the preset threshold, the second mathematical optimization model is a genetic algorithm model. In this case, the constraints are the maximum number of iterations n, the number of individuals in the population m, the crossover probability pc, and the mutation probability pm. The constraints are the number of times estimated based on experience and target accuracy. When m random order sequences of length N are generated, namely the first scheduling order sequence L 11 , L 12 ,…,L 1m After that, the individuals of the population are screened by the roulette model, specifically: the random order sequence L of the m individuals is calculated respectively. 11 , L 12 ,…,L 1m The fitness value F 11 , F 12 ,…,F 1m , with the m fitness values ​​F 11 , F 12 ,…,F 1m , construct a normalized roulette wheel, that is, first calculate m fitness values ​​F 11 , F 12 ,…,F 1m sum , and then calculate m fitness values ​​F respectively 11 , F 12 ,…,F 1m ,and The ratio of , thus constructing the interval [0, 1], with m fitness values ​​F 11 , F 12 ,…,F 1m ,and The interval [0, 1] is divided into m subintervals according to the ratio of [0, 1]; then, m random numbers between 0 and 1 are generated, and the individuals in the m subintervals corresponding to these m random numbers are selected to form a screened population. For example, a random number is 0.05, which falls into F 11 In the subinterval of 11 The corresponding random order sequence L 11 ; Similarly, m random order sequences can be selected, and the m random order sequences may have the same sequence (falling into the same subinterval).

[0078] Next, a crossover operation is performed on the m random sequence sequences of the screened population, that is, individuals that need to be crossed are selected according to the crossover probability pc, and they are paired in pairs. For example, if the screened population has 100 random sequence sequences and the crossover probability pc is 0.8, 0.8*100=80 random sequence sequences are selected and paired into 40 pairs of sequence groups; then for each pair of sequence groups, a random integer between 1 and N is generated, and the values ​​of the two random sequence sequences in the sequence group starting from the position of the random integer are swapped. When there is a repetition or conflict in the number, the numbers are swapped at the end of the conflict to form a crossover. m random order sequences of the rear population; for example, a sequence group for crossover includes two random order sequences P1=2, 3, 1, 7, 5, 6, 4, 8; P2=3, 2, 4, 8, 7, 6, 1, 5; the generated random integer is 3, that is, the values ​​of the two random order sequences P1 and P2 are interchanged from the 3rd position to the left (or right), and after the interchange, the 3rd-digit value 4 of P1 and the 7th-digit value 4 are repeatedly conflicted. At this time, the 7th-digit values ​​4 and 1 of P1 and P2 need to be interchanged to avoid the problem of repeated value conflicts in the same sequence.

[0079] Finally, the mutation operation is performed on the m random sequence sequences of the population after the crossover: the individuals to be mutated are selected according to the mutation probability pm, and for each selected random sequence sequence, two random integers between 1 and N are generated. The subsequences between the corresponding positions of the two random numbers of this sequence are randomly sorted, and finally the latest population m random sequence sequences L are formed. i1 , L i2 ,…,L imFor example, after crossover, a population has 100 random sequence sequences, and the mutation probability pc is 0.05. Then, 0.05*100=5 random sequence sequences are selected for mutation operation. For each pair of sequences in these 5 random sequence sequences, two random integers between 1 and N are generated respectively. The subsequence values ​​between the corresponding positions of the two random integers are randomly sorted. For example, if a sequence is 2, 3, 1, 7, 5, 6, 4, 8, and the two random integers generated are 3 and 5, the subsequence values ​​(1, 7, 5) of the 3rd to 5th positions in the sequence are randomly sorted. For example, the mutated sequence is 2, 3, 5, 1, 7, 6, 4, 8. In this way, after m random sequence sequences L are generated, the subsequence values ​​(1, 7, 5) of the 3rd to 5th positions in the sequence are randomly sorted. 11 , L 12 ,…,L 1m After roulette screening, crossover, and mutation operations, m sequences L' are obtained 11 , L' 12 ,…,L' 1m , calculate the m sequences L' 11 , L' 12 ,…,L' 1m The fitness value of the optimal sequence F1 with the largest fitness value is obtained, thus completing one iterative calculation through the genetic algorithm model.

[0080] Next, m new random order sequences are regenerated, and the roulette wheel screening, crossover, and mutation operations are continued on the m new random order sequences to obtain another m sequences, and the fitness values ​​of the other m sequences are calculated, and the optimal sequence F2 with the largest fitness value is obtained. In this way, multiple iterative calculations are completed through the genetic algorithm model until the number of iterations reaches the maximum number of iterations n, and then the iteration is stopped to obtain the fitness values ​​F1, F2, ..., F of the n optimal sequences. n , select the sequence with the largest fitness value as the optimal sequence, that is, the second scheduling sequence. The second scheduling sequence is the scheduling sequence with the largest fitness value under the constraints. The constraints include the maximum number of iterations, the number of individuals in the population, the crossover probability, and the mutation probability.

[0081] The step S14 specifically includes:

[0082] When the data size exceeds a preset threshold, performing roulette wheel screening, crossover operation, and mutation operation on the first scheduling sequence under constraints to obtain multiple optimal sequences, wherein the constraints include the maximum number of iterations, the number of individuals in the population, the crossover probability, and the mutation probability;

[0083] The second scheduling sequence is determined according to a plurality of preferred sequences, and the second scheduling sequence is a preferred sequence having the largest fitness value among the plurality of preferred sequences.

[0084] The step of performing roulette screening on the first scheduling sequence under the constraint condition specifically includes:

[0085] Calculating fitness values ​​of multiple random order sequences in the first scheduling order sequence respectively, and constructing a normalized interval according to the ratio of the fitness value of each random order sequence to the sum of the fitness values ​​of all random order sequences, wherein the normalized interval includes multiple subintervals corresponding to each fitness value, and the value range of the normalized interval is [0, 1];

[0086] Generate a plurality of different random numbers between 0 and 1, wherein the number of the random numbers is the same as the number of the plurality of random sequence sequences;

[0087] In the normalized interval, the subinterval into which each random number falls is determined, and the corresponding random sequence is selected according to the fitness value of the subinterval into which the random number falls, so as to obtain a plurality of random sequence sequences after screening.

[0088] The step of performing a crossover operation on the first scheduling sequence under the constraint condition specifically includes:

[0089] Determine the random sequence sequences to be crossed according to the crossover probability and the multiple random sequence sequences after screening, and pair the random sequence sequences to be crossed in pairs to obtain multiple pairs of sequence groups, each pair of sequence groups including two random sequence sequences;

[0090] Generate a random integer between 1 and N, where N is the length of the first scheduled sequence, and swap the forward or reverse values ​​of two random sequence sequences in the sequence group starting from the random integer position to obtain multiple random sequence sequences after crossing.

[0091] The step of performing a mutation operation on the first scheduling sequence under the constraint condition specifically includes:

[0092] Determine the random sequence to be mutated according to the mutation probability and multiple random sequence sequences after crossover;

[0093] Generate two random integers between 1 and N, where N is the length of the first scheduled order sequence, and randomly sort the subsequences between the corresponding positions of the two random integers in the random order sequence to be mutated to obtain multiple mutated random order sequences;

[0094] An optimal sequence is calculated based on the multiple mutated random order sequences, and the optimal sequence is a sequence with the largest fitness value among the multiple mutated random order sequences.

[0095] The step of performing roulette screening, crossover operation and mutation operation on the first scheduling sequence under the constraint condition to obtain multiple optimal sequences further includes:

[0096] Repeat the steps of generating the first scheduling sequence and the roulette wheel screening, crossover operation and mutation operation to obtain multiple optimal sequences until the maximum number of iterations in the constraint condition is met.

[0097] Step S15: determining all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traversing each of the state parameters, and determining all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies;

[0098] For specific implementation, please refer to Figures 5 to 8 A scheduling system based on reinforcement learning is proposed, targeting operating speed, production scale, and scheduling methods. This system uses decisions made in any state as the basis for scheduling, ensuring feasible solutions and optimization results while forming a rapid production scheduling solution that can be scheduled in real time. In this embodiment, a simulation scheduling model (i.e., a preset simulation scheduling model) is established in advance, and environmental parameters of the environment in which the simulation scheduling model resides are collected. These collected environmental parameters include equipment information, product information, process flow, and processing time. All state parameters, all action parameters, and standard responses are defined based on these collected environmental parameters.

[0099] In this embodiment, when determining the target overall strategy generated during the reinforcement learning process, it is necessary to first traverse each state parameter and determine all traversal initial strategies corresponding to the state parameter traversed at the current moment from among the pre-set initial strategies. All traversal initial strategies contain the traversal state parameters, and the training actions in each traversal initial strategy are different.

[0100] It should be noted that in this embodiment, during the production scheduling process, the product needs to move between various buffers and machines, and the production scheduling is completed after completing its process flow. For the machines in the production scheduling process, when there are products in the buffer corresponding to the machine, one of the products should be selected for processing. Among them, the buffer is a place where products are temporarily placed, located before a certain machine or a machine of the same type. After the machine processing is completed, the processing of a product needs to be obtained from the buffer. For example, Figure 4 As shown, during the production scheduling process, an idle machine can be used to select a product from the buffer for processing, and each one or more machines corresponds to one or more buffers, such as buffer 1, buffer 2, buffer 3, etc. The buffer selection method can be shown in Table 1:

[0101]

[0102] Table 1

[0103] The method for filtering machine types can be shown in Table 2:

[0104]

[0105] Table 2

[0106] In this embodiment, the setting of the action parameters can be the number of machine types multiplied by the number of buffers. As shown in Table 1, there are 3 actions in the buffer screening mode and 3 actions in the machine type screening mode, so the number of action parameters can be 9, that is, 9 action parameters.

[0107] In this embodiment, the action parameters are based on a one-dimensional action formulated by the machine to select lots and sort the lots in the buffer. Specifically, the parameters are: 1. Due date order; 2. Customer level; 3. Product arrival time from the buffer; 4. Product initial order. The reward is based on the inverse of the stage duration (unit: seconds). When a lot exceeds the due date, the previous positive number (the same number of lots × 100) is subtracted as a penalty.

[0108] Step S16: running the training actions in the traversal initial strategy, determining the optimal strategy based on the running results of each training action, determining the target overall strategy based on the optimal strategy corresponding to each state parameter, and performing simulation scheduling based on the target overall strategy to obtain a third scheduling sequence;

[0109] For specific implementation, please refer to Figure 5-Figure 8 After determining all the initial strategies for traversal, the simulation scheduling model will sequentially execute the training actions in the initial strategies. During each training action, the state of the simulation scheduling model will remain consistent with the traversed state parameters. After training each training action according to the simulation scheduling model, the reward value obtained will determine the training action with the best reward among all the training actions, and this will be used as the optimal training action. The initial strategy corresponding to the optimal training action will be used as the optimal strategy. This optimal strategy includes the traversed state parameters and the optimal training action corresponding to the traversed state parameters. It will then be determined whether the optimal strategies corresponding to all state parameters have been obtained. If the optimal strategies corresponding to all state parameters have been obtained, it can be determined that the reinforcement learning training has been completed. The optimal strategies corresponding to all state parameters will be used as the target overall strategy. Simulation scheduling will be performed based on this target overall strategy to obtain the third scheduling sequence.

[0110] In this embodiment, the training of the optimal strategy requires the establishment of a Q value table to store the state S and all actions A to be taken, that is, Q(S, A). Figure 8As shown, the Q-value table includes actions A1, A2, ..., and An; states S1, S2, ..., and Sn; and qn1, ..., and qnm. For example, if q11 = Q(S1, A1), and qn2 is the largest for state Sn, then the optimal action for state Sn is A2. In this case, the optimal policy for state Sn includes state Sn and action A2.

[0111] In this embodiment, the optimal strategy is determined by traversing each state parameter and running all training actions in the traversal initial strategy corresponding to the traversed state parameter. Then, the target overall strategy is determined based on the optimal strategy corresponding to each state parameter, thereby ensuring the effectiveness of the obtained target overall strategy.

[0112] When it is detected that the simulation scheduling model has completed the target training action, that is, the target training action has been completed at this time, a corresponding reward will be generated. And in this embodiment, each time the simulation scheduling model completes an action, a corresponding reward will be generated. Therefore, after obtaining the reward corresponding to the target training action, it can be determined based on this reward whether the scheduling learning process based on the traversed state parameter has been completed. If not, a new action parameter will be selected from all action parameters as a new training action to continue execution until it is determined that the scheduling learning process based on the traversed state parameter has been completed. And after the target training action is completed, the state of the simulation scheduling model will be converted from the unscheduled state parameter to the traversed state parameter. If it is determined that the scheduling learning process corresponding to the traversed state parameter has been completed, it is necessary to determine whether the scheduling learning process corresponding to all state parameters has been completed. If not, it is necessary to continue to execute the scheduling learning process corresponding to the unfinished state parameter and determine that the reinforcement learning training is not completed. If the scheduling learning process corresponding to all state parameters has been completed, it is determined that the reinforcement learning training has been completed. The method for determining whether the production scheduling learning process corresponding to the traversed state parameters has been completed can be to determine the optimal action parameters corresponding to the traversed state parameters, that is, to determine the rewards corresponding to each action parameter, and select the reward with the best effect among each reward (that is, to compare the obtained reward with the standard reward set in advance to determine the reward with the best effect), the reward with the largest reward value, and then use the action parameter corresponding to this reward as the optimal action parameter. After determining the optimal action parameter corresponding to the traversed state parameters, it can be determined that the production scheduling learning process corresponding to the traversed state parameters has been completed. At this time, the optimal action parameter corresponding to each state parameter in the reinforcement learning training process can be determined, and each group of state parameters and the optimal action parameters corresponding to this group of state parameters can be used as a group of optimal strategies, and then the optimal strategies corresponding to all state parameters can be used as the target overall strategy. Among them, the rewards can include one or more of time, delivery time, equipment utilization, machine equipment switching time, and recipe switching.

[0113] After obtaining the target master strategy, actual production scheduling can be performed according to the target master strategy. After the scheduling is completed, the scheduling results are output. Reinforcement learning training and actual scheduling operations construct a simulation scheduling model and collect environmental parameters to determine state parameters, action parameters, and standard rewards. An initial strategy is then constructed based on each state parameter and action parameter. The simulation scheduling model then undergoes the reinforcement learning training process, initializing the data for the states in the simulation scheduling model and converting them to unscheduled states. Each state parameter is then traversed, and the initial strategy corresponding to the traversed state parameter is determined. Based on the initial strategy, the action corresponding to the state (i.e., the traversed state parameter) is obtained and executed to determine whether scheduling (i.e., scheduling learning) is complete. If not, a new action is obtained while maintaining the current traversed state and execution continues. If yes, i.e., scheduling is complete, then a determination is made to terminate training (i.e., reinforcement learning training). If not, scheduling operations are continued for other state parameters, i.e., updating the strategy. If so, training ends, the target overall strategy is output, and the reinforcement learning training process ends. Then, actual production scheduling begins. During the scheduling process, all data is acquired, and actions corresponding to the state are obtained based on the target overall strategy. These actions are executed, and after the actions are completed, a determination is made as to whether scheduling is complete. If not (i.e., scheduling is not complete), new actions are acquired and executed. If yes (i.e., scheduling is complete), the scheduling results are output, and the process continues until the end.

[0114] For example, in a real-time semiconductor manufacturing scheduling project, if only the processing status of the photolithography area is to be determined, the real environment can be abstracted into an entry, an exit, a buffer, and various processing machines. The state (i.e., state parameter) can be designed as the number of products waiting for processing on each machine + the number of products being processed on each machine + the number of products leaving the buffer + the number of products being shipped to the machine from the buffer + the number of products entering the machine + the number of products completed at the exit. The state is represented as a vector or an array, with each element representing a specific quantity, such as the number of products in buffer 1, the number of products on machine 1, ..., the number of products in buffer n, the number of products on machine n, the number of products entering the machine, and the number of completed products. The action (i.e., action parameter) can be designed as a multi-action consisting of "buffer lot selection" and "machine selection." Lot selection can be based on the highest priority, first-in-first-out, and the fewest available machines. Machine selection can be based on the shortest processing time and the longest idle time. The reward can be based on the inverse of the stage time, with a positive bonus added when machines continuously process the same recipe. When determining the optimal vehicle corresponding to a certain state, the determination can be made through a deep regression model, that is, the steps of running each training action in the traversal initial strategy and determining the optimal strategy based on the running results of each training action include:

[0115] Build a multilayer perceptron:

[0116]

[0117] Where n is the number of layers, is the number of neurons in each layer, D(S, A) is a function with state S and action A as input, and the output is Q value;

[0118] According to the above multi-layer perceptron, the optimal strategy can be obtained, namely:

[0119]

[0120] Among them, D(S, a) is a function of state S and the best action a corresponding to state S.

[0121] It should be noted that, in the reinforcement learning process of this embodiment, Figure 7 As shown, the rewards at each stage change dynamically. The goal is to maximize the final total reward through different combinations, thereby further improving other metrics. Because there are multiple stages in the production scheduling process, it is possible to determine the states and actions corresponding to each stage, as well as the corresponding rewards. Strategies can then be summarized and trained based on these states, actions, and rewards. The reward can be the stage duration (base) plus continuous processing of the same recipe (additional), and the action can be a combination of "buffer lot selection" and "machine selection." Lot selection can be based on the highest priority, first-in-first-out, and the fewest available machines. Machine selection can be based on the shortest processing time and the longest machine idle time. Strategies can be derived by building a multilayer perceptron and performing deep reinforcement learning using the PPO algorithm.

[0122] Step S17: Perform production scheduling according to the third scheduling sequence.

[0123] In specific implementation, after obtaining the optimally sorted third scheduling sequence, scheduling operations can be performed according to the third scheduling sequence. By rapidly optimizing the production schedule of semiconductor manufacturing, reasonable scheduling of manpower, equipment and other resources can be achieved, helping factories to reasonably allocate production capacity, improve resource utilization, reduce production time, balance production lines, reduce enterprise costs, and so on.

[0124] In addition, see 9. The present invention further provides a semiconductor manufacturing scheduling system for implementing the above-mentioned semiconductor manufacturing scheduling method, the system comprising:

[0125] A first determining unit 10 is configured to determine a mathematical optimization model according to the data size of semiconductor manufacturing scheduling;

[0126] a first calculation unit 20 configured to generate a first scheduling sequence according to the mathematical optimization model, and to perform iterative calculations on the first scheduling sequence under constraints using the mathematical optimization model to obtain a second scheduling sequence, wherein the second scheduling sequence is a scheduling sequence with a maximum fitness value under the constraints, wherein the constraints include a maximum number of iterations;

[0127] A second determining unit 30 is used to determine the target attribute priority of the product to be processed;

[0128] The second computing unit 40 is configured to determine all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine all traversal initial strategies corresponding to the traversed state parameters based on each of the initial strategies; execute the training actions in each of the traversal initial strategies, determine the optimal strategy based on the execution results of each of the training actions, determine the target overall strategy based on the optimal strategies corresponding to each of the state parameters, and perform simulation scheduling based on the target overall strategy to obtain a third scheduling sequence;

[0129] The scheduling unit 50 is configured to perform scheduling operations according to a third scheduling sequence.

[0130] Among them, see Figure 10 , the first calculation unit 20 further includes:

[0131] A generating subunit 201 is configured to generate two random integers when the data size does not exceed a preset threshold, wherein the two random integers are different integers between 1 and N, where N is the length of the first scheduling sequence;

[0132] The swap subunit 202 is configured to swap the values ​​of the sequence positions corresponding to the two random integers in the first scheduling sequence to obtain a first comparison sequence;

[0133] The iterator unit 203 is configured to calculate and compare the fitness value of the first scheduling sequence with the fitness value of the first comparison sequence; when the fitness value of the first comparison sequence is greater than the fitness value of the first scheduling sequence, repeat the step of exchanging the values ​​of the first comparison sequence until the maximum number of iterations in the constraint condition is satisfied, thereby obtaining a second scheduling sequence, which is the scheduling sequence with the maximum fitness value under the constraint condition.

[0134] In addition, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the semiconductor manufacturing scheduling method as described above.

[0135] Compared with the prior art, the present invention has the following beneficial effects:

[0136] 1. By changing traditional manual scheduling methods, it can easily cope with complex high-mix production environments. It reduces run times and reduces the cost of handling emergencies such as insertion failures, making enterprise production more flexible and efficient.

[0137] 2. Production planning based on real production capacity synchronizes supplier deliveries with factory scheduling, thereby reducing inventory and transportation costs caused by ordering production raw materials in advance.

[0138] 3. By combining the global search capabilities of intelligent algorithms and the local optimization techniques of reinforcement learning algorithms, production capacity can be effectively and reasonably allocated, resource utilization can be improved, and production costs can be reduced.

[0139] In summary, the present invention achieves rapid target optimization of semiconductor manufacturing scheduling through the combination of the global search capability of the intelligent algorithm and the local optimization skills of the reinforcement learning algorithm, effectively and reasonably allocates production capacity, improves resource utilization, and reduces production costs.

[0140] The description and application of the present invention here are illustrative and are not intended to limit the scope of the present invention to the above-described embodiments. The deformation and changes of the embodiments disclosed here are possible, and the replacement of the embodiments and the various components that are equivalent are well known to those of ordinary skill in the art. It should be clear to those skilled in the art that the present invention can be realized in other forms, structures, arrangements, proportions, and with other components, materials and parts without departing from the spirit or essential characteristics of the present invention. Other deformation and changes can be made to the embodiments disclosed here without departing from the scope and spirit of the present invention.

Claims

1. A semiconductor manufacturing production scheduling method, characterized in that: The method comprises the following steps: Acquire the data scale of semiconductor manufacturing production scheduling, and determine the mathematical optimization model according to the data scale; the data scale includes the number of products to be processed, when the number of products to be processed does not exceed 500, the mathematical optimization model is a hill climbing algorithm model; when the number of products to be processed exceeds 500, the mathematical optimization model is a genetic algorithm model; Generate a first scheduling sequence according to the mathematical optimization model, and obtain a second scheduling sequence by iteratively calculating the first scheduling sequence under constraints through the mathematical optimization model, wherein the second scheduling sequence is a scheduling sequence with the maximum fitness value under the constraints, and the constraints include a maximum number of iterations; Determine all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine all traversal initial strategies corresponding to the traversed state parameters based on each initial strategy; the training actions include delivery order, customer level, product arrival time from the buffer zone, and product initial order; Run the training actions in each of the traversal initial strategies, determine the optimal strategy based on the running results of each of the training actions, determine the target overall strategy according to the optimal strategy corresponding to each of the state parameters, and perform simulation scheduling according to the target overall strategy to obtain a third scheduling sequence; The scheduling operation is performed according to a third scheduling order sequence.

2. The semiconductor manufacturing production scheduling method according to claim 1, characterized in that: The step of iteratively calculating the first scheduling sequence under the constraint condition by the mathematical optimization model to obtain the second scheduling sequence specifically includes: When the data size does not exceed a preset threshold, two random integers are generated, where the two random integers are different integers between 1 and N, where N is the length of the first scheduling sequence; Interchanging the values ​​of the sequence positions corresponding to the two random integers in the first scheduling sequence to obtain a first comparison sequence; Calculate and compare the fitness value of the first scheduling order sequence with the fitness value of the first comparison order sequence; when the fitness value of the first comparison order sequence is greater than the fitness value of the first scheduling order sequence, repeat the step of exchanging the numerical values ​​for the first comparison order sequence until the maximum number of iterations in the constraint condition is met, and obtain the second scheduling order sequence, which is the scheduling order sequence with the largest fitness value under the constraint condition.

3. The semiconductor manufacturing production scheduling method according to claim 1, characterized in that: The step of iteratively calculating the first scheduling sequence under the constraint condition by the mathematical optimization model to obtain the second scheduling sequence specifically includes: When the data scale exceeds a preset threshold, a plurality of preferred sequences are obtained by performing roulette wheel screening, crossover operation and mutation operation on the first scheduling sequence under constraint conditions, wherein the constraint conditions include a maximum number of iterations, the number of individuals in the population, a crossover probability and a mutation probability; The second scheduling order sequence is determined according to a plurality of preferred sequences, and the second scheduling order sequence is a preferred sequence having the largest fitness value among the plurality of preferred sequences.

4. The semiconductor manufacturing production scheduling method according to claim 3, characterized in that: The step of performing roulette selection on the first scheduling sequence under the constraint condition specifically includes: Calculating fitness values ​​of multiple random order sequences in the first scheduling order sequence respectively, constructing a normalized interval according to the ratio of the fitness value of each random order sequence to the sum of the fitness values ​​of all random order sequences, wherein the normalized interval includes multiple sub-intervals corresponding to each fitness value, and the value range of the normalized interval is [0, 1]; Generate a plurality of different random numbers between 0 and 1, wherein the number of the random numbers is the same as the number of the plurality of random order sequences; In the normalized interval, the subinterval into which each random number falls is determined, and the corresponding random order sequence is selected according to the fitness value of the subinterval into which the random number falls, so as to obtain a plurality of random order sequences after screening.

5. The semiconductor manufacturing production scheduling method according to claim 4, characterized in that: The step of performing a crossover operation on the first scheduling sequence under a constraint condition specifically includes: Determine the random sequence sequences to be crossed according to the crossover probability and the multiple random sequence sequences after screening, and pair the random sequence sequences to be crossed in pairs to obtain multiple pairs of sequence groups, each pair of sequence groups includes two random sequence sequences; Generate a random integer between 1 and N, where N is the length of the first scheduling sequence, and swap the forward or reverse values ​​of two random sequence sequences in the sequence group starting from the random integer position to obtain multiple random sequence sequences after crossing.

6. The semiconductor manufacturing production scheduling method according to claim 5, characterized in that: The step of performing a mutation operation on the first scheduling sequence under a constraint condition specifically includes: Determine the random sequence sequence to be mutated according to the mutation probability and multiple random sequence sequences after crossover; Generate two random integers between 1 and N, where N is the length of the first scheduling sequence, and randomly sort the subsequences between the corresponding positions of the two random integers in the random sequence to be mutated to obtain a plurality of mutated random sequence sequences; A preferred sequence is calculated based on the multiple mutated random order sequences, and the preferred sequence is a sequence with the largest fitness value among the multiple mutated random order sequences.

7. A semiconductor manufacturing production scheduling system implementing the semiconductor manufacturing production scheduling method according to any one of claims 1 to 6, characterized in that: The system comprises: A first determining unit, configured to determine a mathematical optimization model according to the data scale of semiconductor manufacturing scheduling; a first calculation unit, configured to generate a first scheduling sequence according to the mathematical optimization model, and to obtain a second scheduling sequence by iteratively calculating the first scheduling sequence under constraints using the mathematical optimization model, wherein the second scheduling sequence is a scheduling sequence with a maximum fitness value under the constraints, wherein the constraints include a maximum number of iterations; A second determination unit, used to determine the target attribute priority of the product to be processed; A second calculation unit is used to determine all state parameters of the semiconductor manufacturing environment and the training actions of the second scheduling sequence, traverse each of the state parameters, and determine all traversal initial strategies corresponding to the traversed state parameters based on each initial strategy; and run the training actions in each of the traversal initial strategies, determine the optimal strategy based on the running results of each of the training actions, determine the target overall strategy according to the optimal strategy corresponding to each of the state parameters, and perform simulation scheduling according to the target overall strategy to obtain a third scheduling sequence; The scheduling unit is used to perform scheduling operations according to a third scheduling order sequence.

8. A computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the semiconductor manufacturing scheduling method according to any one of claims 1 to 6.

Images