A semiconductor production scheduling method and medium that considers opportunistic maintenance

Abstract

This invention relates to a semiconductor production scheduling method and medium considering opportunistic maintenance, comprising the following steps: acquiring the production status of a semiconductor production workshop; establishing an opportunistic maintenance and scheduling collaborative optimization model, wherein the semiconductor production workshop includes multiple equipment groups, each equipment group containing multiple pieces of equipment; establishing an equipment state model to model and represent equipment faults; modeling each equipment group as an agent based on the opportunistic maintenance and scheduling collaborative optimization model and the equipment state model, and designing an action space and structured reward function based on combined scheduling rules; training each agent using the QMIX algorithm within the CTDE framework, coordinating the use of independent policies allocated to each equipment group, and synchronizing outputs based on real-time workshop status, thereby optimizing production and maintenance scheduling. Compared with existing technologies, this invention can improve production efficiency while reducing maintenance costs and improving the overall performance of the semiconductor manufacturing system.

Description

Technical Field

[0001] This invention belongs to the field of production automation scheduling technology, and in particular relates to a semiconductor production scheduling method and medium that takes into account opportunistic maintenance. Background Technology

[0002] One of the challenges in semiconductor production scheduling is the co-optimization problem, which involves coordinating semiconductor production scheduling with other control decisions, such as equipment maintenance. Production scheduling and equipment maintenance are interdependent and coupled. Equipment ages continuously during operation and may experience frequent failures; therefore, scheduling equipment maintenance activities during processing is necessary. However, there is a conflict between equipment maintenance and production scheduling: delaying maintenance activities until equipment is idle increases the equipment failure rate; scheduling maintenance activities during processing consumes processing time, thus affecting order completion times. Traditionally, scheduling and maintenance problems are usually handled independently, but with the increasing complexity and uncertainty of production systems, jointly optimizing these two tasks plays a crucial role in improving resource utilization, production efficiency, and reducing the total cost of manufacturing systems.

[0003] Currently, research on the coordinated optimization of maintenance and scheduling mainly employs mathematical programming methods. However, real-world production systems are inherently complex and subject to multiple constraints, often posing significant challenges to mathematical models in terms of solution space and computational complexity. Chinese patent application CN118348928A proposes a semiconductor production line scheduling method, comprising: acquiring production data of the semiconductor to be produced; iteratively calculating multiple scheduling plans randomly obtained based on the production data to obtain a candidate scheduling population that meets the iteration termination condition, wherein the candidate scheduling population includes multiple candidate scheduling plans related to the semiconductor to be produced, each of which is obtained through feasibility evaluation using a linear programming model; and selecting the candidate scheduling plan with the shortest production cycle from the candidate scheduling population as the target scheduling plan for the semiconductor to be produced. This prior art does not jointly consider the impact of equipment maintenance on production scheduling, and its method, based on mathematical programming, suffers from drawbacks such as a large solution space and computational difficulties. Furthermore, traditional maintenance methods often neglect the opportunity cost of maintenance, while in semiconductor manufacturing, the high investment cost of equipment is crucial. In summary, a new semiconductor production scheduling method needs to be designed to synergistically optimize production scheduling and opportunistic maintenance, thereby improving production efficiency while reducing maintenance costs and enhancing the overall performance of the semiconductor manufacturing system. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art by providing a semiconductor production scheduling method and medium that considers opportunistic maintenance. By using multi-agent reinforcement learning to jointly optimize scheduling and maintenance tasks and incorporating opportunity costs into the decision-making process, this invention improves production efficiency while reducing maintenance costs and enhances the overall performance of the semiconductor manufacturing system.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] This invention provides a semiconductor production scheduling method that considers opportunistic maintenance, comprising the following steps:

[0007] The production status of a semiconductor manufacturing workshop is obtained, and an opportunistic maintenance and scheduling collaborative optimization model is established. The semiconductor manufacturing workshop includes multiple equipment groups, and each equipment group contains multiple pieces of equipment.

[0008] Establish a device state model to represent device faults;

[0009] Based on the opportunistic maintenance and scheduling collaborative optimization model and the equipment state model, each equipment group is modeled as an intelligent agent, and an action space and structured reward function based on combined scheduling rules are designed.

[0010] The QMIX algorithm is used to train each agent within the CTDE framework. The agents coordinate and utilize the independent policies assigned to each equipment group to synchronize outputs based on real-time shop floor status, thereby optimizing production and maintenance scheduling.

[0011] Furthermore, the objective of the opportunistic maintenance and scheduling collaborative optimization model is to minimize the total cost of maintenance activities, which includes opportunity cost and the actual cost of maintenance itself. The opportunity cost is defined as the number of jobs that could have been processed without maintenance, based on the number of jobs waiting to be processed in the equipment's front buffer and the time until the expected failure occurs.

[0012] The maintenance activities include preventive maintenance and corrective maintenance. The mean time for preventive maintenance is the same as the mean time to repair. After preventive maintenance, the mean time to failure of the equipment is twice that after corrective maintenance.

[0013] Furthermore, the process of establishing the device state model is as follows:

[0014] Define key equipment status variables, including remaining useful life and mean time for preventive maintenance, and define that the estimated remaining useful life of the equipment is less than twice the mean time for preventive maintenance;

[0015] The equipment failure time is determined by the probability distribution with the expected mean time of failure as the parameter. The simulation verifies whether the predicted failure time is earlier than the current simulation time. If so, corrective maintenance is scheduled immediately after the current step is completed. If not, a preventive maintenance decision is made, and the corresponding maintenance time is determined by the probability distribution with the expected mean time of maintenance as the parameter.

[0016] The preventive maintenance time is sampled from a probability distribution using the average preventive maintenance time as an input parameter. When preventive maintenance is scheduled, the equipment remains shut down until the maintenance task is fully completed. Each piece of equipment records its last maintenance operation. If the last operation was preventive maintenance, the expected failure time used to calculate the next failure time is drawn from a probability distribution with a higher expected average failure time input.

[0017] Furthermore, the fault time is recalculated from the corresponding probability distribution before each fault check.

[0018] Furthermore, the combined scheduling rules include the earliest delivery date priority rule, the shortest remaining processing time priority rule, the shortest processing time priority rule, the first-in-first-out rule, and the critical ratio rule.

[0019] Furthermore, the specific design of the action space based on the combined scheduling rules is as follows:

[0020] The action space is {a1, a2}, where a1 represents the equipment processing action, which is divided into discrete action domains, and a2 represents the maintenance action. The combined scheduling rule includes multiple simple heuristic scheduling rules. a1 is generated by linearly combining multiple simple heuristic scheduling rules with calculated weights. The weight of each simple heuristic scheduling rule is determined according to the importance of the operation in a specific environment.

[0021] Furthermore, the structured reward function is based on a weighted combination of negative penalties and positive incentives. When the expected remaining lifespan of the equipment is lower than the average job processing time, the failure to schedule preventive maintenance will be penalized; when the buffer utilization rate is lower than a set threshold and the expected remaining lifespan of the equipment is less than twice the average preventive maintenance time, the selection of maintenance behavior will be rewarded.

[0022] Furthermore, under the CTDE framework, all agents are treated as a whole for centralized training. During the training process, each agent selects actions to perform from the action space based on its own partial observations and the limited information from other agents.

[0023] The QMIX algorithm calculates the overall value function by aggregating the local value functions of each policy. The QMIX algorithm's super network takes the agent's state as input and outputs the super network's weights and biases. The QMIX algorithm's super network uses a linear network with an absolute value activation function.

[0024] Furthermore, during centralized training, a small batch of samples is randomly drawn from the experience pool to calculate scores, and then global state information is aggregated to randomize trajectories based on state selection.

[0025] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. This invention proposes a semiconductor production scheduling method considering opportunistic maintenance, comprising the following steps: First, acquiring the production status of the semiconductor production workshop and establishing a collaborative optimization model for opportunistic maintenance and scheduling; then, establishing an equipment status model to represent equipment faults; based on the above steps, modeling each equipment group in the semiconductor production workshop as an agent, and designing an action space and structured reward function based on combined scheduling rules; finally, using the QMIX algorithm to train each agent under the CTDE framework, coordinating the use of independent strategies assigned to each equipment group, and synchronizing outputs based on real-time workshop status, thereby optimizing production and maintenance scheduling; the above method adopts an opportunistic maintenance strategy designed for semiconductor manufacturing systems and combines it with production scheduling strategies, thus achieving a better balance between reducing maintenance costs and improving production efficiency, improving the overall performance of the semiconductor manufacturing system. Because the QMIX method is used for collaborative optimization of opportunistic maintenance and scheduling, it overcomes the limitations of traditional methods in complex decision-making scenarios, achieving more intelligent and dynamic optimization decisions.

[0028] 2. In this invention, the estimated remaining life time (RLT) of the equipment is defined to be less than twice the mean time to preventive maintenance (MTTPM) to ensure that opportunistic maintenance is not too frequent; the failure time is recalculated from the corresponding probability distribution before each failure inspection, which can avoid excessively long uninterrupted running time caused by the tail of the failure time distribution and eliminate the extremely short running time at its lower tail.

[0029] 3. The structured reward function designed in this invention is based on a weighted combination of negative penalties and positive incentives. When the expected remaining lifespan of the equipment is lower than the average operation time, the failure to schedule preventive maintenance will be penalized. When the buffer utilization rate is lower than a set threshold and the expected remaining lifespan of the equipment is less than twice the average preventive maintenance time, the selection of maintenance behavior will be rewarded. The above design can balance the urgency of maintenance with the benefits of keeping the equipment running. Attached Figure Description

[0030] Figure 1 This is a schematic flowchart of the method of the present invention;

[0031] Figure 2 This example shows how the reward value changes with the number of iterations. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0033] Example:

[0034] This embodiment provides a semiconductor production scheduling method that considers opportunistic maintenance, such as Figure 1 As shown, it includes the following steps:

[0035] S1. Obtain the production status of the semiconductor manufacturing workshop and establish an opportunistic maintenance and scheduling collaborative optimization model.

[0036] In the smart workshop, there are pending orders denoted as O1, O2, ..., O0. Each order contains a specific type and quantity of jobs, and there may be differences between different orders. Variable T O N O and D O These represent the type, quantity, and delivery deadline of order O0, respectively. Since each job is associated with a specific order, variable J can be used. 0，n Let represent the nth job in order O0. By tracking each job, its type and delivery deadline can be determined.

[0037] The semiconductor manufacturing workshop is divided into M equipment groups, denoted as G1, G2, ..., G... M Each equipment group contains equipment with shared processing capabilities and attributes, and follows the same scheduling strategy during production. Equipment group G i The number of devices in the system is denoted as n. i Each device can be labeled as m i，j , indicating device group G iLet j be the j-th device in the system. The collaborative optimization strategy for maintenance and scheduling is expressed as follows:

[0038]

[0039] The first M×k elements represent the scheduling strategy for each device group. For example, device group G M The scheduling strategy parameters are determined by Given: The number of scheduling rules, k, is equal to the number of elements in the vector. The remaining elements represent the maintenance strategies for each device within each device group.

[0040] Opportunistic Maintenance and Scheduling Collaborative Optimization Model I 0 The goal is to minimize the total cost of maintenance activities, C. t To obtain a set of optimal collaborative maintenance and scheduling strategies, denoted as a * :

[0041]

[0042] Total maintenance cost C t Including opportunity cost and the actual cost of maintenance itself, opportunity cost is defined as the number of jobs that could have been processed without maintenance, based on the number of jobs waiting to be processed in the equipment's buffer zone and the time remaining until the expected failure occurs. Maintenance activities include preventive maintenance and corrective maintenance. The mean time to repair (MTTPM) for preventive maintenance is the same as the expected mean time to repair (E-MTTR). After preventive maintenance, the expected mean time to failure (E-MTTF) of the equipment is twice that after corrective maintenance.

[0043] All tasks are scheduled to arrive at time zero, and the processing of each type of task cannot be interrupted. The experimental simulation parameters are set as shown in Table 1. UDR refers to unplanned downtime maintenance, and SDM refers to planned downtime maintenance.

[0044] Table 1 Simulation Parameters

[0045]

[0046]

[0047] In terms of scheduling strategy, several simple heuristic scheduling rules are integrated into a single composite scheduling rule. This composite scheduling rule includes the Earliest Delivery Date (EDD) rule, the Shortest Remaining Processing Time (SRPT) rule, the Shortest Processing Time (SPT) rule, the First-In-First-Out (FIFO) rule, and the Critical Ratio (CR) rule. The composite scheduling rule calculates the processing priority of jobs by combining these five heuristic scheduling rules. The decoding process involves calculating the priority of each job within the equipment group buffer according to a single scheduling rule. The final priority of the jobs is determined by weighting the decisions made by the strategy.

[0048] S2. Establish a device state model to represent device faults. The specific process is as follows:

[0049] Define key equipment status variables, including remaining useful life (RLT) and mean time to preventive maintenance (MTTPM). Define the estimated remaining useful life (RLT) of the equipment to be less than twice the mean time to preventive maintenance (MTTPM) to ensure that opportunistic maintenance is not too frequent.

[0050] By integrating maintenance strategies with production scheduling decisions, the synergy between maintenance activities and production tasks is ensured. Equipment failure times are then determined using a probability distribution with the expected mean time between failures (E-MTTF) as a parameter. To prevent equipment failures, a simulation system verifies whether the predicted failure time is earlier than the current simulation time. If so, corrective maintenance is scheduled immediately after the current step is completed, thus avoiding interference with ongoing processing steps. If not, preventative maintenance decisions are made, and the corresponding maintenance time is determined by a probability distribution with the expected mean time between failures (E-MTTF) as a parameter. To prevent excessively long or short equipment uptime, failure times are recalculated from the corresponding probability distribution before each fault check, thereby avoiding excessively long uninterrupted uptime caused by the tail of the failure time distribution and eliminating extremely short uptime at its lower tail.

[0051] The preventive maintenance time (TTPM) is sampled from a probability distribution using the mean time to preventive maintenance (MTTPM) as an input parameter. When preventive maintenance is scheduled, the equipment remains offline until the maintenance task is fully completed. Each piece of equipment records its last maintenance operation. If the last operation was preventive maintenance, the expected time to failure (ETTF) used to calculate the next failure time is sampled from a probability distribution with a higher expected mean time to failure (E-MTTF) input. This shows that preventive maintenance can extend the failure interval and improve equipment reliability.

[0052] S3. Model each device group as an intelligent agent and design an action space and structured reward function based on combined scheduling rules.

[0053] An intelligent agent can simultaneously be responsible for the maintenance and scheduling of the equipment group, making collaborative decisions and synchronizing the actions of each agent based on the real-time status of the workshop. The agent is based on a Markov decision process and is described by a tuple G = (S, U, P, R, O, γ). At time step t, s t ∈S represents the global environment state. Each agent a∈A selects an action u. a ∈U to form a joint action u. The joint action u is applied to the environment based on the transition probability P(s). t+1 |s t u t When transitioning to the next state, each agent receives a reward r(s). t u t During observation, the agent can only perceive its own state information. Different agents have different observation spaces, denoted by O(s, a). a This represents the action observation history of agent a, and a proxy function π is constructed based on this history. a (u a |τ a The specific design is as follows:

[0054] (1) Observation space O

[0055] It includes information related to equipment, orders, and operations, specifically:

[0056] ① Data related to the device includes: device ID, type of the last maintenance operation, current status, estimated downtime, and queue length in the buffer.

[0057] ② Data related to orders includes: order number, order due date, number of jobs published, and number of jobs completed.

[0058] ③ Data related to the task includes: online time, current step, waiting time for the current step, due date, remaining time, and cumulative waiting time.

[0059] (2) The action space is {a1, a2}

[0060] Here, a1 represents the equipment processing action. To reduce convergence difficulty and improve the algorithm's convergence efficiency, a1 is divided into discrete action domains. a2 represents the maintenance action, i.e., whether the equipment in the equipment group is scheduled for maintenance. Based on the aforementioned combined scheduling rules, a1 is generated by linearly combining multiple simple heuristic scheduling rules with calculated weights. The weights of the simple heuristic scheduling rules are determined according to the importance of the task in a specific environment. In this embodiment, the linear weights of the scheduling strategy are adjusted by incrementing by 0.05 to generate a combination of multiple combined scheduling rules.

[0061] (3) Reward function R

[0062] S×A×S→R, based on the selected action type, rewards desired behaviors and penalizes undesirable behaviors. The structured reward function is based on a weighted combination of negative penalties and positive incentives, with weights chosen empirically to effectively drive the achievement of desired behaviors. When the estimated remaining life of the equipment is less than the average job processing time, the failure to schedule preventative maintenance is penalized; when buffer utilization is below a set threshold and the estimated remaining life of the equipment (RLT) is less than twice the mean preventative maintenance time (MTTPM), the selection of maintenance is rewarded. This mechanism aims to balance the urgency of maintenance with the benefits of keeping the equipment running. Total reward r total It is an instant reward. t A comprehensive metric that reflects the urgency of maintenance actions based on equipment condition (such as RLT and buffer utilization). Additionally, an adjustment factor is included to encourage maintenance decisions while taking into account operational waiting conditions.

[0063] The above mechanism is specifically described in the following pseudocode, where the reward depends on the action chosen by the policy:

[0064] If you choose the maintenance action:

[0065] If the estimated remaining lifespan is less than the average processing time of the equipment:

[0066] r t =12

[0067] otherwise:

[0068] If the estimated remaining lifetime is <2×MTTPM and the buffer utilization rate is <10%:

[0069] r t =max{20 - 8 × estimated remaining lifetime / (2 × MTTPM), 12}

[0070] otherwise:

[0071] r t =-0.8

[0072] If no maintenance action is selected:

[0073] If the estimated remaining lifespan is less than the average processing time of the equipment:

[0074] r t =-0.5

[0075] otherwise:

[0076] If buffer utilization = 0:

[0077] rt =-5

[0078] otherwise:

[0079] r t =1

[0080] r total =r t +8 × (1 - standardized waiting time)

[0081] (4) State transition function P

[0082] S×A×S→[0, 1], where it is assumed that the reward function is bounded. The state transition function represents the transition probability of an agent executing a given policy in a Markov decision process environment.

[0083] S4. Train each agent under the CTDE framework using the QMIX algorithm, coordinate the use of independent policies assigned to each equipment group, and synchronize outputs based on real-time shop floor status to optimize production and maintenance scheduling.

[0084] The QMIX algorithm calculates the global value function Q by aggregating the local value functions Q of each strategy. out The QMIX algorithm's supernetwork (HN) takes the agent's state as input and outputs the supernetwork's weights and biases. To ensure the non-negativity of the weights, a linear network with an absolute value activation function is used. Without non-negativity constraints, the biases of the last layer of the hybrid network are obtained through two layers of networks and the Rectified Linear Unit (ReLU) activation function, thus forming a nonlinear mapping network.

[0085] The QMIX algorithm operates within the CTDE framework, where all agents are treated as a whole and trained centrally. During training, each agent selects actions from the action space based on its own observations and limited information from other agents.

[0086] Random sampling can reduce sample correlation, thereby enhancing algorithm stability. During centralized training, a small batch of samples (s) is typically drawn from the experience pool. t ,o t ,u t ,r t ,s t+1 ,o t+1 The algorithm calculates a score, then aggregates global state information and randomizes the trajectory selected based on state s. The pseudocode for the QMIX agent training algorithm is as follows:

[0087] For each agent, construct an evaluation RNN (Recurrent Neural Network), denoted as Net. eval And the target RNN network, denoted as Net. target .

[0088] The process continues in a loop until the stopping condition is met.

[0089] For each agent l:

[0090] In each RNN network,

[0091] In hybrid networks:

[0092]

[0093] Calculate the loss function

[0094] Backpropagation using the loss function

[0095] Output the Net of all agents eval .

[0096] During training, the training effect is evaluated by observing changes in the system's total reward value. Figure 2 This shows how the reward value changes with the number of iterations, where the vertical axis represents the total system reward value and the horizontal axis represents the number of iterations. From Figure 2 As can be seen, the reward curve tends to flatten out around 300,000 iterations, fluctuates slightly around 1 million rounds, but achieves a more stable strategy after 1.3 million rounds.

[0097] Combining corrective maintenance (CM) and preventive maintenance (PM) with FIFO, CR, EDD, and SRPT scheduling rules, and using the single-agent DQN algorithm as a control group, this embodiment conducted simulation experiments to verify the effectiveness of the QMIX method used in this embodiment. All experiments were conducted in the same environment, and the training was repeated 10 times to reduce random errors. The reliability of the conclusions was enhanced by averaging the performance metrics of all models. The experimental results are shown in Table 2 (UDR and SDM are presented as average values).

[0098] Table 2 Key Performance Indicators for Benchmark Heuristics, DQN, and QMIX in MINIFAB

[0099]

[0100]

[0101] As shown in Table 2, even under the same simple heuristic scheduling rules, the mean cycle time (CT Mean) varies significantly due to different maintenance scheduling strategies. Evaluating various maintenance strategies involves assessing the mean of UDR and SDM. Although QMIX increases the proportion of SDM, SDM generally outperforms UDR because it allows for proactive planning to minimize overall downtime or utilize maintenance opportunities to reduce queuing time, thus mitigating its impact on CT. Under corrective maintenance strategies, the mean UDR is significantly higher than under preventive maintenance strategies. Conversely, the mean SDM for corrective maintenance strategies is defined as zero, while the mean SDM for preventive maintenance strategies is significantly lower than that for QMIX. Both corrective and preventive maintenance result in significantly longer total maintenance times. On the other hand, although QMIX increases the proportion of SDM, the mean CT remains superior due to its effective mitigation of the impact of equipment downtime on CT. The changes in CT performance under preventive maintenance strategies are mainly related to differences in UDR. QMIX schedules maintenance in advance when opportunities arise. If the estimated RLT exceeds the actual RLT of the equipment, it reduces the likelihood of unplanned equipment failures. Therefore, SDM replaces UDR more frequently, demonstrating its advantages over other maintenance heuristics and DQN.

[0102] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.

Claims

1. A semiconductor production scheduling method considering opportunistic maintenance, characterized in that, Includes the following steps: The production status of a semiconductor manufacturing workshop is obtained, and an opportunistic maintenance and scheduling collaborative optimization model is established. The semiconductor manufacturing workshop includes multiple equipment groups, and each equipment group contains multiple pieces of equipment. Establish a device state model to represent device faults; Based on the opportunistic maintenance and scheduling collaborative optimization model and the equipment state model, each equipment group is modeled as an intelligent agent, and an action space and structured reward function based on combined scheduling rules are designed. The QMIX algorithm is used to train each agent under the CTDE framework, and the independent policies assigned to each equipment group are coordinated to synchronize outputs with real-time shop floor status, thereby optimizing production and maintenance scheduling. The objective of the opportunistic maintenance and scheduling collaborative optimization model is to minimize the total cost of maintenance activities. The total cost of maintenance activities includes opportunity cost and the actual cost of maintenance itself. The opportunity cost is defined as the number of jobs that could have been processed without maintenance, based on the number of jobs waiting to be processed in the equipment's front buffer and the time until the expected failure occurs. The maintenance activities include preventive maintenance and corrective maintenance. The mean time for preventive maintenance is the same as the expected mean time to repair. After preventive maintenance, the expected mean time to failure of the equipment is twice that after corrective maintenance. The process of establishing the device state model is as follows: Define key equipment status variables, including remaining useful life and mean time for preventive maintenance, and define that the estimated remaining useful life of the equipment is less than twice the mean time for preventive maintenance; The equipment failure time is determined by the probability distribution with the expected mean time of failure as the parameter. The simulation verifies whether the predicted failure time is earlier than the current simulation time. If so, corrective maintenance is scheduled immediately after the current step is completed. If not, a preventive maintenance decision is made, and the corresponding maintenance time is determined by the probability distribution with the expected mean time of maintenance as the parameter. The preventive maintenance time is sampled from a probability distribution using the average preventive maintenance time as an input parameter. When preventive maintenance is scheduled, the equipment remains shut down until the maintenance task is fully completed. Each piece of equipment records its last maintenance operation. If the last operation was preventive maintenance, the expected failure time used to calculate the next failure time is drawn from a probability distribution with a higher expected average failure time input.

2. The semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that, Before each fault check, the fault time is recalculated from the corresponding probability distribution.

3. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that, The combined scheduling rules include the earliest delivery date priority rule, the shortest remaining processing time priority rule, the shortest processing time priority rule, the first-in-first-out rule, and the critical ratio rule.

4. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that, The specific design of the action space based on the combined scheduling rules is as follows: The action space is { , },in The processing actions of the equipment are represented and divided into discrete action domains. This indicates a maintenance action; the combined scheduling rule includes multiple simple heuristic scheduling rules. The scheduling rules are generated by linearly combining multiple simple heuristics based on calculated weights. The weights of each simple heuristic are determined based on the importance of the job in a specific environment.

5. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that, The structured reward function is based on a weighted combination of negative penalties and positive incentives. When the estimated remaining lifespan of the equipment is less than the average operation time, the failure to schedule preventive maintenance will be penalized; when the buffer utilization rate is lower than a set threshold and the estimated remaining lifespan of the equipment is less than twice the average preventive maintenance time, the selection of maintenance behavior will be rewarded.

6. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that, In the CTDE framework, all agents are treated as a whole and trained centrally. During the training process, each agent selects actions to perform from the action space based on the partial information it observes and the limited information from other agents. The QMIX algorithm calculates the overall value function by aggregating the local value functions of each policy. The QMIX algorithm's super network takes the agent's state as input and outputs the super network's weights and biases. The QMIX algorithm's super network uses a linear network with an absolute value activation function.

7. A semiconductor production scheduling method considering opportunistic maintenance according to claim 1, characterized in that, During centralized training, a small batch of samples is randomly drawn from the experience pool to calculate scores, and then global state information is aggregated to randomize trajectories based on state selection.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images