Semiconductor production scheduling method considering opportunistic maintenance and medium

Abstract

The invention relates to a semiconductor production scheduling method considering opportunistic maintenance and a medium, and the method comprises the following steps: obtaining the production state of a semiconductor production workshop, building an opportunistic maintenance and scheduling collaborative optimization model, the semiconductor production workshop comprising a plurality of equipment groups, and each equipment group comprising a plurality of equipment; establishing an equipment state model, and carrying out modeling representation on an equipment fault; modeling each device group into an intelligent agent according to the opportunity maintenance and scheduling collaborative optimization model and the device state model, and designing an action space and a structured reward function based on a combined scheduling rule; a QMIX algorithm is utilized to train each agent under a CTDE framework, an independent strategy allocated to each equipment group is utilized in a coordinated manner, and output synchronization is performed according to a real-time workshop state, so that production and maintenance scheduling are optimized. Compared with the prior art, the method can reduce the maintenance cost while improving the production efficiency, and improves the overall performance of a semiconductor manufacturing system.

Description

Technical Field

[0001] The present invention belongs to the technical field of production automation scheduling, and in particular relates to a semiconductor production scheduling method and medium taking opportunistic maintenance into consideration. Background Art

[0002] One of the difficulties in studying semiconductor production scheduling is the problem of collaborative optimization, which means that semiconductor production scheduling needs to be coordinated with other control decisions, such as equipment maintenance. Production scheduling and equipment maintenance are mutually influential and coupled. The equipment continues to age during continuous operation and may frequently fail. Therefore, it is necessary to schedule equipment maintenance activities during processing. However, there is a certain conflict between equipment maintenance and production scheduling: if maintenance activities are delayed until the equipment is idle, the equipment failure rate will increase; if maintenance activities are scheduled during processing, processing time will be occupied, thus affecting the completion time of the order. Traditional scheduling problems and maintenance problems are usually handled independently, but with the increase in complexity and uncertainty of production systems, joint optimization of these two tasks plays an important role in improving resource utilization, production efficiency, and reducing the total cost of manufacturing systems.

[0003] At present, research on maintenance and scheduling collaborative optimization mainly adopts mathematical programming methods, but the production system in the real world is relatively complex and has multiple constraints. Mathematical models often face huge solution space and computational challenges. The Chinese patent application with publication number CN118348928A proposes a method for scheduling semiconductor production line operations, including: obtaining production data of semiconductors to be produced; iteratively calculating multiple scheduling plans randomly obtained based on the production data to obtain a candidate scheduling population that meets the iteration termination condition, wherein the candidate scheduling population includes multiple candidate scheduling plans related to the semiconductors to be produced, wherein each scheduling plan is obtained by feasibility evaluation of a linear programming model; and selecting the candidate scheduling plan with the shortest production cycle from the candidate scheduling population as the target scheduling plan for the semiconductors to be produced. This prior art does not jointly consider the impact of equipment maintenance problems on production scheduling. Its method is based on mathematical programming and has the disadvantages of large solution space and computational difficulties. In addition, traditional maintenance methods often ignore the opportunity cost of maintenance, but in semiconductor manufacturing, the high investment cost of equipment is crucial. In summary, it is necessary to redesign a semiconductor production scheduling method to collaboratively optimize production scheduling and opportunistic maintenance, reduce maintenance costs while improving production efficiency, and improve the overall performance of the semiconductor manufacturing system. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a semiconductor production scheduling method and medium that takes opportunistic maintenance into consideration. The scheduling and maintenance tasks are jointly optimized through multi-agent reinforcement learning, and the opportunity cost is incorporated into the decision-making process. The maintenance cost is reduced while the production efficiency is improved, thereby improving the overall performance of the semiconductor manufacturing system.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides a semiconductor production scheduling method considering opportunistic maintenance, comprising the following steps:

[0007] Obtaining the production status of a semiconductor production workshop and establishing an opportunistic maintenance and scheduling collaborative optimization model, wherein the semiconductor production workshop includes multiple equipment groups, each of which includes multiple equipment groups;

[0008] Establish equipment status model and model equipment failure;

[0009] According to the opportunistic maintenance and scheduling collaborative optimization model and the equipment status model, each equipment group is modeled as an intelligent agent, and an action space and a structured reward function based on a combined scheduling rule are designed;

[0010] The QMIX algorithm is used to train each intelligent agent under the CTDE framework, coordinate the use of independent strategies assigned to each equipment group, and synchronize outputs with real-time workshop status to optimize production and maintenance scheduling.

[0011] Furthermore, the goal of the opportunistic maintenance and scheduling collaborative optimization model is to minimize the total cost of maintenance activities, which includes the opportunity cost and the actual cost of maintenance itself. According to the number of jobs waiting to be processed in the buffer before the equipment and the time to the expected failure, the number of jobs that can be processed by the equipment without maintenance is defined as the opportunity cost;

[0012] The maintenance activities include preventive maintenance and corrective maintenance. The average time of preventive maintenance is the same as the expected average maintenance time. After preventive maintenance occurs, the expected average failure time of the equipment is twice that after corrective maintenance occurs.

[0013] Furthermore, the establishment process of the device state model is specifically as follows:

[0014] Define key equipment status variables, including remaining life time and average preventive maintenance time, and define the estimated value of the remaining life time of the equipment to be less than twice the average preventive maintenance time;

[0015] The equipment failure time is determined by the probability distribution with the expected mean failure time as a parameter, and the simulation verifies whether the predicted failure time is earlier than the current simulation time. If so, the equipment is scheduled for corrective maintenance immediately after the current step is completed; if not, a preventive maintenance decision is made, and the corresponding maintenance time is determined by the probability distribution with the expected mean maintenance time as a parameter;

[0016] The preventive maintenance time is sampled from the probability distribution using the mean time of preventive maintenance as an input parameter. When preventive maintenance is scheduled, the equipment remains shut down until the maintenance task is fully completed. Each device records its last maintenance operation. If the last operation is preventive maintenance, the expected failure time used to calculate the next failure time is drawn from the probability distribution with a higher expected mean failure time input.

[0017] Furthermore, the failure time is recalculated from the corresponding probability distribution before each failure check.

[0018] Furthermore, the combined scheduling rules include an earliest delivery date priority rule, a shortest remaining processing time priority rule, a shortest processing time priority rule, a first-in-first-out rule and a key ratio rule.

[0019] Furthermore, the specific design of the action space based on the combined scheduling rule is as follows:

[0020] The action space is {a 1 ,a 2}, where a 1 Represents the equipment processing action, which is divided into discrete action domains, a 2 Represents a maintenance action; the combined scheduling rule includes multiple simple heuristic scheduling rules, a 1 Multiple simple heuristic scheduling rules are generated by linearly combining the calculated weights, and the weight of each simple heuristic scheduling rule is determined according to the importance of the job in a specific environment.

[0021] Furthermore, the structured reward function is composed of a weighted combination of negative penalties and positive incentives. When the estimated remaining life of the equipment is lower than the average job processing time, the behavior of not scheduling preventive maintenance will be punished; when the buffer utilization rate is lower than the set threshold and the estimated remaining life of the equipment is less than twice the average preventive maintenance time, the maintenance behavior will be rewarded.

[0022] Furthermore, under the CTDE framework, all agents are treated as a whole for centralized training. During the training process, each agent selects actions to be executed from the action space based on the partial information it observes and the limited information of other agents.

[0023] The QMIX algorithm calculates the overall value function by aggregating the local value functions of each strategy. The super network of the QMIX algorithm takes the state of the agent as input and outputs the weight and offset of the super network. The super network of the QMIX algorithm adopts a linear network with an absolute value activation function.

[0024] Furthermore, during centralized training, a small batch of samples are randomly drawn from the experience pool to calculate the scores, and then the global state information is aggregated to randomize the trajectory based on the state selection.

[0025] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. The present invention proposes a semiconductor production scheduling method considering opportunistic maintenance, comprising the following steps: first, obtaining the production status of a semiconductor production workshop and establishing a collaborative optimization model of opportunistic maintenance and scheduling; then, establishing an equipment status model to model and represent equipment failures; based on the above steps, each equipment group in the semiconductor production workshop is modeled as an intelligent agent, and an action space and a structured reward function based on combined scheduling rules are designed; finally, the QMIX algorithm is used to train each intelligent agent under the CTDE framework, and the independent strategies assigned to each equipment group are coordinated to synchronize outputs with real-time workshop status, thereby optimizing production and maintenance scheduling; the above method adopts an opportunistic maintenance strategy designed for a semiconductor manufacturing system and combines it with a production scheduling strategy, so that it can better achieve a balance between reducing maintenance costs and improving production efficiency, and improve the overall performance of the semiconductor manufacturing system. Since the QMIX method is used to collaboratively optimize opportunistic maintenance and scheduling, it can overcome the limitations of traditional methods in complex decision-making scenarios and achieve more intelligent and dynamic optimization decisions.

[0028] 2. In the present invention, the estimated value of the remaining life time RLT of the equipment is defined to be less than twice the mean time of preventive maintenance MTTPM to ensure that opportunistic maintenance will not be too frequent; the failure time is recalculated from the corresponding probability distribution before each fault inspection, which can avoid the excessively long uninterrupted operation time caused by the tail of the failure time distribution and eliminate the extremely short operation time at its lower tail.

[0029] 3. The structured reward function designed by the present invention is based on a weighted combination of negative penalties and positive incentives. When the estimated remaining life of the equipment is lower than the average job processing time, the behavior of not arranging preventive maintenance will be punished; when the buffer utilization rate is lower than the set threshold and the estimated remaining life of the equipment is less than twice the average preventive maintenance time, the maintenance behavior will be rewarded. The above design can balance the urgency of maintenance with the benefits of keeping the equipment running. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the process of the present invention;

[0031] Figure 2 This is how the reward value changes with the number of iterations in the embodiment. DETAILED DESCRIPTION

[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0033] Example:

[0034] This embodiment provides a semiconductor production scheduling method considering opportunistic maintenance, such as Figure 1 As shown, the following steps are included:

[0035] S1. Obtain the production status of the semiconductor production workshop and establish an opportunistic maintenance and scheduling collaborative optimization model.

[0036] There are pending orders in the smart workshop, which are marked as O. 1 , O 2 , ..., O 0 , each order contains a specific type and quantity of jobs, which may vary between different orders. O 、N O and D O Respectively represent order O 0 Since each job is associated with a specific order, the variable J 0，n To represent the order O 0 By tracking each job, its type and deadline can be determined.

[0037] The semiconductor production workshop is divided into M equipment groups, denoted by G 1 , G 2 , ..., G M , each equipment group contains equipment with common processing capabilities and attributes, and follows the same scheduling strategy during the production process. Equipment group G iThe number of devices in is recorded as n i , each device can be marked as m i，j , indicating device group G i The collaborative optimization strategy of maintenance and scheduling is as follows:

[0038]

[0039] The first M×k elements represent the scheduling strategy of each device group. For example, the device group G M The scheduling policy parameters are given by Given. The number of scheduling rules k is equal to the number of elements in the vector. The remaining elements represent the maintenance strategies of each device in each device group.

[0040] Opportunistic maintenance and scheduling collaborative optimization model I 0 The goal is to minimize the total cost of maintenance activities C t , to obtain a set of optimal collaborative maintenance and scheduling strategies, denoted as a * :

[0041]

[0042] Total cost of maintenance activities C t Including opportunity cost and actual cost of maintenance itself, the opportunity cost is defined as the number of jobs that could have been processed without maintenance, based on the number of jobs waiting to be processed in the equipment's front buffer and the time until the expected failure occurs. Maintenance activities include preventive maintenance and corrective maintenance. The mean time to preventive maintenance (MTTPM) is the same as the expected mean time to repair (E-MTTR). After preventive maintenance occurs, the expected mean time to failure (E-MTTF) of the equipment is twice that after corrective maintenance occurs.

[0043] It is assumed that all jobs arrive at time zero, and the processing of each type of job cannot be interrupted. The experimental simulation parameter settings are shown in Table 1. UDR refers to unplanned downtime maintenance, and SDM refers to planned downtime maintenance.

[0044] Table 1 Simulation parameters

[0045]

[0046]

[0047] In terms of scheduling strategy, multiple simple heuristic scheduling rules are integrated into a combined scheduling rule, which includes the earliest delivery date priority rule (EDD), the shortest remaining processing time priority rule (SRPT), the shortest processing time priority rule (SPT), the first-in-first-out rule (FIFO) and the critical ratio rule (CR). The combined scheduling rule calculates the processing priority of the job by combining these five heuristic scheduling rules. The decoding process involves calculating the priority of each job in the equipment group buffer according to a single scheduling rule. The final priority of the job is determined by weighting the decisions made by the strategy.

[0048] S2. Establish a device status model to model and represent device failures. The specific process is as follows:

[0049] Define key equipment status variables, including remaining life time (RLT) and mean time to preventive maintenance (MTTPM), and define the estimated value of the remaining life time (RLT) of the equipment to be less than twice the mean time to preventive maintenance (MTTPM) to ensure that opportunistic maintenance is not too frequent.

[0050] By integrating maintenance strategies with production scheduling decisions, maintenance activities are ensured to be carried out in coordination with production tasks. Then, the equipment failure time is determined by the probability distribution with the expected mean time to failure (E-MTTF) as a parameter. To prevent equipment failure, the simulation system is used to verify whether the predicted failure time is earlier than the current simulation time. If so, the equipment is scheduled to perform corrective maintenance immediately after completing the current step, so as to avoid interference with the ongoing processing steps; if not, the equipment makes a preventive maintenance decision, and the corresponding maintenance time is determined by the probability distribution with the expected mean time to repair (E-MTTR) as a parameter. In order to prevent the equipment from running too long or too short, the failure time is recalculated from the corresponding probability distribution before each fault inspection, thereby avoiding the excessively long uninterrupted running time caused by the tail of the failure time distribution and eliminating the extremely short running time at its lower tail.

[0051] The mean time to preventive maintenance (MTTPM) is used as an input parameter to sample the time to preventive maintenance (TTPM) from the probability distribution. When preventive maintenance is scheduled, the equipment remains shut down until the maintenance task is fully completed. Each device records its last maintenance operation. If the last operation is preventive maintenance, the expected failure time ETTF used to calculate the next failure time is drawn from the probability distribution with a higher expected mean time to failure E-MTTF input, indicating that preventive maintenance can extend the failure interval and improve equipment reliability.

[0052] S3. Model each device group as an intelligent agent and design an action space and structured reward function based on the combined scheduling rules.

[0053] The agent can be responsible for the maintenance and scheduling of the equipment group at the same time, and synchronize the actions of each agent according to the real-time status of the workshop. The agent is based on the Markov decision process and is described by the tuple G = (S, U, P, R, O, γ). At time step t, s t ∈S represents the global environment state. Each agent a∈A chooses an action u a ∈U to form a joint action u. Apply the joint action u to the environment, based on the transition probability P(s t+1 |s t ,u t ) to the next state, each agent receives a reward r(s t ,u t ). During the observation process, the agent can only perceive its own state information. Different agents have different observation spaces, represented by O(s, a). τ a Represents the action observation history of agent a, and constructs the agent function π based on this history a (u a |τ a ). The specific design is as follows:

[0054] (1) Observation space O

[0055] Contains equipment-related, order-related, and job-related information, specifically:

[0056] ①Data related to the device include: the device ID, the type of the last maintenance operation, the current status, the estimated failure time, and the queue length in the buffer.

[0057] ② Order-related data includes: order number, order expiration date, number of jobs published, and number of jobs completed.

[0058] ③Job-related data include: online time, current step, waiting time of current step, due date, remaining time and accumulated waiting time.

[0059] (2) The action space is {a 1 , a 2}

[0060] Among them, a 1 In order to reduce the difficulty of convergence and improve the convergence efficiency of the algorithm, a 1 is divided into discrete action domains, a 2 Indicates the maintenance action, that is, whether the equipment in the equipment group is scheduled for maintenance. Based on the aforementioned combined scheduling rules, a 1Multiple simple heuristic scheduling rules are generated by linearly combining the calculated weights, and the weights of the simple heuristic scheduling rules are determined according to the importance of the job in the specific environment. In this embodiment, the linear weights of the scheduling strategies are adjusted by 0.05 increments to generate a combination of multiple combined scheduling rules.

[0061] (3) Reward function R

[0062] S×A×S→R, rewards are given to desired behaviors and penalties are imposed on undesirable behaviors, depending on the type of action selected. The structured reward function is based on a weighted combination of negative penalties and positive incentives, and the weights are chosen based on experience to effectively promote the realization of desired behaviors. When the estimated remaining life of the equipment is lower than the average job processing time, the behavior of not scheduling preventive maintenance will be punished; when the buffer utilization rate is lower than the set threshold and the estimated remaining life (RLT) of the equipment is less than twice the mean time to preventive maintenance (MTTPM), the selected maintenance behavior will be rewarded. This mechanism aims to balance the urgency of maintenance with the benefits of keeping the equipment running. The total reward r total Is an instant reward t A composite measure that reflects the urgency of maintenance actions based on equipment conditions such as RLT and buffer utilization. In addition, an adjustment factor is included that is designed to encourage maintenance decisions while taking into account waiting conditions in the operation.

[0063] The above mechanism is specifically described by the following pseudo code, where the reward depends on the action selected by the policy:

[0064] If you select Maintenance Action:

[0065] If the estimated remaining life is less than the average processing time of the equipment:

[0066] r t =12

[0067] otherwise:

[0068] If the estimated remaining lifetime is < 2×MTTPM and the buffer utilization is < 10%:

[0069] r t = max{20-8×estimated remaining life / (2×MTTPM), 12}

[0070] otherwise:

[0071] r t =-0.8

[0072] If you do not select a maintenance action:

[0073] If the estimated remaining life is less than the average processing time of the equipment:

[0074] rt =-0.5

[0075] otherwise:

[0076] If buffer utilization = 0:

[0077] r t =-5

[0078] otherwise:

[0079] r t =1

[0080] r total =r t +8×(1-normalized waiting time)

[0081] (4) State transfer function P

[0082] S×A×S→[0, 1], where the reward function is assumed to be bounded. The state transition function represents the transition probability when the agent executes a certain strategy under a given Markov decision process environment.

[0083] S4. Use the QMIX algorithm to train each intelligent agent under the CTDE framework, coordinate the use of independent strategies assigned to each equipment group, and synchronize outputs with real-time workshop status to optimize production and maintenance scheduling.

[0084] The QMIX algorithm calculates the overall value function Q by aggregating the local value functions Q of each strategy. out ,The hypernetwork (HN) of the QMIX algorithm takes the state of the agent as input,and outputs the weights and offsets of the hypernetwork. In order to ensure the non-negativity of the weights, a linear network with an absolute value activation function is used. In the absence of the non-negativity constraint, the offset of the last layer of the hybrid network is obtained through a two-layer network and the rectified linear unit (ReLU) activation function, thus forming a non-linear mapping network.

[0085] The QMIX algorithm runs under the CTDE framework. All agents are treated as a whole for centralized training. During the training process, each agent selects actions to be executed from the action space based on the partial information observed by itself and the limited information of other agents.

[0086] Random sampling can reduce sample correlation and enhance algorithm stability. During centralized training, a small batch of samples (s t ,o t ,u t ,r t ,s t+1 ,o t+1), calculate the score, then aggregate the global state information and randomize the trajectory selected based on the state s. The pseudo code of the QMIX agent training algorithm is as follows:

[0087] Build an evaluation RNN (recurrent neural network) network for each agent, denoted as Net eval , and the target RNN network, denoted as Net target .

[0088] When the stop condition is not met, the loop executes:

[0089] For each agent l:

[0090] In each RNN network,

[0091] In a hybrid network:

[0092]

[0093] Calculating the loss function

[0094] Backpropagation using loss function

[0095] Output the Net of all agents eval .

[0096] During the training process, the training effect is evaluated by observing the changes in the total reward value of the system. Figure 2 The figure shows the change of reward value with the number of iterations, where the vertical axis represents the total reward value of the system and the horizontal axis represents the number of iterations. Figure 2 It can be seen that the reward value curve flattens out around 300,000 iterations, fluctuates slightly around 1 million rounds, but a more stable strategy is obtained after 1.3 million rounds.

[0097] Combining corrective maintenance (CM) and preventive maintenance (PM) with FIFO, CR, EDD and SRPT scheduling rules, and using the single-agent DQN algorithm as a comparison group, this embodiment conducts simulation experiments to verify the effectiveness of the QMIX method used in this embodiment. All experiments were conducted in the same environment, and the training was repeated 10 times to reduce accidental errors, and the performance indicators of all models were averaged to enhance the reliability of the conclusions. The experimental results are shown in Table 2 (UDR and SDM are presented in the form of average values).

[0098] Table 2 Key performance indicators of the baseline heuristic algorithm, DQN and QMIX in MINIFAB

[0099]

[0100]

[0101] As can be seen from Table 2, even under the same simple heuristic scheduling rule, the mean cycle time (CT Mean) varies significantly due to different maintenance scheduling strategies. Evaluating various maintenance strategies involves evaluating the mean of UDR and SDM. Although QMIX increases the proportion of SDM, SDM is generally superior to UDR because it allows proactive planning to minimize overall downtime or take advantage of maintenance opportunities to reduce queue time, thereby mitigating its impact on CT. Under the corrective maintenance strategy, the mean UDR is significantly higher than that of the preventive maintenance strategy. In turn, the SDM mean of the corrective maintenance strategy is zero by definition, while the SDM mean of the preventive maintenance strategy is significantly lower than that of QMIX. The total maintenance time is significantly longer for both corrective and preventive maintenance. On the other hand, although QMIX increases the proportion of SDM, the mean CT is still superior because it effectively mitigates the impact of equipment downtime on CT. The changes in CT performance under preventive maintenance strategies are mainly related to the differences in UDR. QMIX pre-schedules maintenance when the opportunity arises, and it reduces the likelihood of unplanned equipment failures if the estimated RLT exceeds the actual RLT of the equipment. Therefore, SDM replaces UDR more frequently, showing advantages over other maintenance heuristics and DQN.

[0102] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0103] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A semiconductor production scheduling method considering opportunistic maintenance, characterized in that: The following steps are involved: Obtaining the production status of a semiconductor production workshop and establishing an opportunistic maintenance and scheduling collaborative optimization model, wherein the semiconductor production workshop includes multiple equipment groups, each of which includes multiple equipment groups; Establish equipment status model and model equipment failure; According to the opportunistic maintenance and scheduling collaborative optimization model and the equipment status model, each equipment group is modeled as an intelligent agent, and an action space and a structured reward function based on a combined scheduling rule are designed; The QMIX algorithm is used to train each intelligent agent under the CTDE framework, coordinate the use of independent strategies assigned to each equipment group, and synchronize outputs with real-time workshop status to optimize production and maintenance scheduling.

2. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: The goal of the opportunistic maintenance and scheduling collaborative optimization model is to minimize the total cost of maintenance activities, which includes opportunity costs and actual costs of maintenance itself. According to the number of jobs waiting to be processed in the buffer before the equipment and the time until the expected failure occurs, the number of jobs that could be processed by the equipment without maintenance is defined as the opportunity cost; The maintenance activities include preventive maintenance and corrective maintenance. The average time of preventive maintenance is the same as the expected average maintenance time. After preventive maintenance occurs, the expected average failure time of the equipment is twice that after corrective maintenance occurs.

3. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: The process of establishing the device state model is as follows: Define key equipment status variables, including remaining life time and average preventive maintenance time, and define the estimated value of the remaining life time of the equipment to be less than twice the average preventive maintenance time; The equipment failure time is determined by the probability distribution with the expected mean failure time as a parameter, and the simulation verifies whether the predicted failure time is earlier than the current simulation time. If so, the equipment is scheduled for corrective maintenance immediately after the current step is completed; if not, a preventive maintenance decision is made, and the corresponding maintenance time is determined by the probability distribution with the expected mean maintenance time as a parameter; The preventive maintenance time is sampled from the probability distribution using the mean time of preventive maintenance as an input parameter. When preventive maintenance is scheduled, the equipment remains shut down until the maintenance task is fully completed. Each device records its last maintenance operation. If the last operation is preventive maintenance, the expected failure time used to calculate the next failure time is drawn from the probability distribution with a higher expected mean failure time input.

4. A semiconductor production scheduling method considering opportunistic maintenance according to claim 3, characterized in that: The failure time is recalculated from the corresponding probability distribution before each failure check.

5. The semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: The combined scheduling rules include the earliest delivery date priority rule, the shortest remaining processing time priority rule, the shortest processing time priority rule, the first-in-first-out rule and the key ratio rule.

6. The semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: The specific design of the action space based on the combined scheduling rules is as follows: The action space is {a1, a2}, where a1 represents the equipment processing action, which is divided into discrete action domains, and a2 represents the maintenance action; the combined scheduling rule includes multiple simple heuristic scheduling rules, a1 is generated by linearly combining multiple simple heuristic scheduling rules through calculated weights, and the weight of each simple heuristic scheduling rule is determined according to the importance of the job in a specific environment.

7. The semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: The structured reward function is based on a weighted combination of negative penalties and positive incentives. When the estimated remaining life of the equipment is lower than the average job processing time, the behavior of not scheduling preventive maintenance will be punished; when the buffer utilization rate is lower than the set threshold and the estimated remaining life of the equipment is less than twice the average preventive maintenance time, the maintenance behavior will be rewarded.

8. The semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: Under the CTDE framework, all agents are treated as a whole for centralized training. During the training process, each agent selects actions to be executed from the action space based on the partial information it observes and the limited information of other agents. The QMIX algorithm calculates the overall value function by aggregating the local value functions of each strategy. The super network of the QMIX algorithm takes the state of the agent as input and outputs the weight and offset of the super network. The super network of the QMIX algorithm adopts a linear network with an absolute value activation function.

9. The semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that: During centralized training, a small batch of samples are randomly drawn from the experience pool to calculate the scores, and then the global state information is aggregated to randomize the trajectory based on the state selection.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images