Monocrystalline silicon rod hybrid flow shop scheduling method with sequence correlation setting time

Abstract

The invention discloses a single crystal silicon rod hybrid flow shop scheduling method with sequence correlation setting time, and belongs to the technical field of intelligent manufacturing and hybrid flow shop scheduling. The method comprises the following steps: firstly, establishing a mathematical model of the scheduling problem of the silicon single crystal rod hybrid flow shop with sequence correlation setting time; secondly, designing states, actions and reward functions in a Q learning algorithm in combination with the characteristics of the problem, further combining the five designed low-level heuristic operators, dynamically scheduling the use of the operators by Q learning, enabling the operators to play roles in a targeted manner in different search stages, and under the synergistic effect, solving the problem in the prior art. The method is superior to a traditional method depending on random exploration in the aspects of convergence rate optimization, solution quality and search stability, and the scheduling problem of the single crystal silicon rod hybrid flow shop with sequence correlation setting time can be more efficiently solved.

Description

Technical Field

[0001] This invention relates to a method for scheduling a hybrid production line for monocrystalline silicon rods with sequential correlation setting time, belonging to the field of intelligent manufacturing and hybrid production line scheduling technology. Background Technology

[0002] With the rapid development of technologies such as the Internet of Things, artificial intelligence, and cloud computing, the strategic importance of semiconductors in the information industry is becoming increasingly prominent. As a core material for semiconductor manufacturing, the market demand for monocrystalline silicon rods continues to grow, and they are widely used in high-tech fields such as integrated circuits, new energy photovoltaics, and microelectronic devices. The photovoltaic industry has become an important pillar of green energy development, leading to a significant increase in demand for high-efficiency, high-purity monocrystalline silicon materials. However, this high demand is not matched by relatively lagging production capacity.

[0003] Therefore, promoting the large-scale and industrialized production of monocrystalline silicon rods has become a trend, and related literature is gradually increasing. However, most current research focuses on improving the production quality of monocrystalline silicon rods, while research on production scheduling issues is relatively limited. Faced with the demand for diverse and large-volume orders, improving production efficiency and reducing manufacturing costs have become crucial for semiconductor companies to enhance their competitiveness and achieve large-scale development.

[0004] Currently, most monocrystalline silicon rod manufacturers still rely on manual scheduling and planning, which struggles to meet the market demand for large-volume, multi-variety production. Therefore, exploring efficient and practical scheduling algorithms to optimize production processes has become a crucial research direction in this field. Existing research has primarily modeled monocrystalline silicon rod production scheduling as a hybrid flow shop scheduling problem, which has been proven to be a strongly NP-hard problem. This invention aims to explore the scheduling optimization of the monocrystalline silicon rod hybrid flow shop scheduling problem (SDST_MSR_HFSP) with sequence-dependent set times in actual production processes, thereby improving its production efficiency.

[0005] Research reveals significant room for improvement in existing technologies: Most current studies on hybrid flow shop scheduling (HFSP) do not focus on the specific case of monocrystalline silicon rods. While both belong to HFSP, monocrystalline silicon rods face more stringent production requirements due to their unique characteristics. Monocrystalline silicon rods are diverse, and to ensure product quality, equipment cleaning and calibration are necessary to prevent cross-contamination. Therefore, setup time is crucial and closely related to the production sequence, especially during the crystal pulling stage, where even similar products require setup time—a significant difference from other problems. However, most current research neglects setup time or merges it with processing time, which contradicts real-world production. Therefore, considering setup time is essential for promoting the large-scale production and industrialization of monocrystalline silicon rods. Furthermore, the high demand for monocrystalline silicon rods necessitates large-scale production, but many methods are still being simulated and validated on small to medium scales, lacking large-scale production testing. Without suitable methods for large-scale production, production plans will be disrupted, leading to adverse effects. Therefore, large-scale, multi-variety data testing is of great significance in addressing this production deficiency. Therefore, it is evident that optimizing the scheduling of monocrystalline silicon rods in a hybrid production line based on actual production conditions has significant value.

[0006] In view of this, the present invention is hereby proposed. Summary of the Invention

[0007] This invention provides a method for scheduling a hybrid production line of monocrystalline silicon rods with sequence-dependent setting times. First, a mathematical model of the scheduling problem of a hybrid production line of monocrystalline silicon rods with sequence-dependent setting times is established. Second, the state, action, and reward functions in the Q-learning algorithm are designed according to the characteristics of the problem, so that the Q-learning algorithm has strong optimization capabilities when solving the scheduling problem of a hybrid production line of monocrystalline silicon rods with setting times.

[0008] The technical solution of this invention is:

[0009] According to a first aspect of the present invention, a method for scheduling a single-crystal silicon rod hybrid production line with sequential correlation setting time is provided, comprising:

[0010] Step 1: Based on the requirement of minimizing the maximum completion time in the production scheduling of the monocrystalline silicon rod hybrid production line, determine the objective function and constraints to establish a mathematical model of the monocrystalline silicon rod hybrid production line scheduling problem with sequence-dependent set times.

[0011] Step 2: Based on Q-learning as the core reinforcement learning algorithm, find the global optimal solution to the single-crystal silicon rod hybrid flow shop scheduling problem with sequence-dependent time settings, specifically:

[0012] Step 2.1: Design an encoding and decoding scheme for the scheduling problem of a hybrid production line of single-crystal silicon rods with sequence-related time settings;

[0013] Step 2.2: Construct a low-level heuristic pool based on the preset heuristics;

[0014] Step 2.3: Design the states and actions in Q-learning: Design the states based on the improvement ratio and construct the action set based on the low-level heuristic pool;

[0015] Step 2.4: Initialize parameter settings;

[0016] Step 2.5: Generate an initial solution using a metaheuristic method;

[0017] Step 2.6: Calculate the target value of the initial solution based on the objective function in the mathematical model;

[0018] Step 2.7: Take the initial solution as the current optimal solution;

[0019] Step 2.8: Randomly select a state from the state space as the initial state;

[0020] Step 2.9: Based on the state, action set, and initial parameters, execute the Q-learning iterative optimization process to obtain the optimal solution corresponding to the optimal target value under the current iteration number;

[0021] Step 2.10: Determine if the termination condition is met. If it is, output the global optimal solution; otherwise, jump to Step 2.9.

[0022] Furthermore, the objective function of the mathematical model is to minimize the maximum completion time; the constraints include:

[0023] The first constraint is that the maximum completion time is greater than or equal to the completion time of each job.

[0024] The second constraint stipulates that each job can only be assigned to one machine at each processing stage;

[0025] The third constraint states that each machine can only process one job at a time.

[0026] The fourth constraint refers to the set time required to process different jobs sequentially on the same machine in the same processing stage;

[0027] The fifth constraint refers to the constraints on start and finish times for each machine when processing different workpieces;

[0028] The sixth constraint indicates that a subsequent processing stage can only begin after the previous processing stage has been completed;

[0029] The seventh constraint represents the completion time constraint that takes into account the sequence correlation setting time;

[0030] The eighth constraint states that the completion time of a task is equal to its start time plus the processing time.

[0031] The ninth constraint indicates that each job is processed in the order of processing stages;

[0032] Time constraints are used to ensure that the start and finish times of a task are not less than 0.

[0033] Among them, "job" refers to a single-crystal silicon rod.

[0034] Furthermore, the encoding / decoding scheme includes:

[0035] The coding scheme uses a two-dimensional list representation, where each two-dimensional list consists of a two-element sublist; the first element in the sublist represents the job number, and the second element represents the product type to which the job belongs;

[0036] The decoding scheme considers the processing order of the job and machine allocation. For the processing order of the job, the processing order in the first stage is directly reflected by the encoding scheme. In the remaining stages, the processing order of the job is determined by the first-come-first-served rule. In terms of machine allocation, an improved first-idle heuristic rule is adopted, which checks whether there is an available machine that can be directly processed and is of the same type as the job to be processed.

[0037] Furthermore, the low-level heuristic pool includes a variety of heuristics, including: reverse difference operation, swap difference operation, permutation difference operation, and insertion difference operation.

[0038] Furthermore, the design of states and actions in Q-learning includes:

[0039] State design: Divide the state space into non-overlapping blocks; let π represent the solution before the action is executed, and π' represent the solution after the action is executed; calculate the improvement ratio to represent the reward based on the solution before the action is executed and the next state to which the solution belongs based on the improvement ratio;

[0040] Action design: A collection of all heuristic building actions from the low-level heuristic pool.

[0041] Furthermore, Step 2.9 includes:

[0042] Step 2.9.1: Obtain actions using an action selection strategy; and execute actions on the current optimal solution to obtain a new solution π. * And obtain the new objective value f(π) * When a new solution is better than an old solution, the new solution is taken as the current optimal solution.

[0043] Step 2.9.2: Set the reward function r, update the Q table according to the obtained reward, and update the state to the next state;

[0044] Step 2.9.3: Record the optimal solution corresponding to the optimal objective value at the current iteration number.

[0045] Furthermore, the reward function expression is: In the formula, f(π) represents the target value of the solution π before the action is performed; f(π) * ) represents the target value of the solution after the action is performed, i.e., the target value of the new solution.

[0046] According to a second aspect of the present invention, a single-crystal silicon rod hybrid production line scheduling system with sequence-dependent setting time is provided, comprising a module based on the single-crystal silicon rod hybrid production line scheduling method with sequence-dependent setting time as described above.

[0047] According to a third aspect of the present invention, a processor is provided for running a program, wherein the program, when running, executes the single-crystal silicon rod mixing flow shop scheduling method with sequence-related setting time as described in any one of the preceding claims.

[0048] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein, when the program is executed, the device on which the computer-readable storage medium is located executes the single-crystal silicon rod mixing production line scheduling method with sequence-correlated setting time as described in any one of the preceding claims.

[0049] The beneficial effects of this invention are as follows: For the scheduling problem of a mixed-flow workshop for single-crystal silicon rods with sequential correlation setting time, this invention proposes a reward function and state partitioning mechanism based on an improvement ratio. Specifically, the improvement ratio of the objective function value of the solution before and after executing an action is used as the reward signal. The reward signal is the feedback reward obtained after executing the action, and the level of the reward can guide Q-learning to select the next state. The design of the improvement ratio being consistent with the reward function allows Q-learning to adaptively adjust the update of action value according to different improvement magnitudes. This design not only overcomes the shortcomings of traditional reward functions in characterizing local improvement magnitude differences, but also enables the learning process to more accurately reflect changes in search quality, thereby effectively improving the linkage and learning efficiency among state, action, and reward. Based on this, this invention further combines five designed low-level heuristic operators, which are dynamically scheduled by Q-learning to play a targeted role in different search stages. On the one hand, the reward function and state partitioning mechanism ensure the sensitivity and guidance of Q-learning in improving solution quality, guaranteeing rapid convergence of the algorithm to the high-quality solution region. On the other hand, diverse low-level heuristics provide extensive perturbations to the solution space, significantly enhancing the diversity of the search. Through the synergistic effect of these two aspects, this invention outperforms traditional methods relying on random exploration in terms of optimizing convergence speed, solution quality, and search stability, and can more efficiently solve the single-crystal silicon rod hybrid flow shop scheduling problem with sequence-correlated setting times. Attached Figure Description

[0050] Figure 1 This is a flowchart of the reinforcement learning algorithm based on Q-learning used in this invention to solve problems;

[0051] Figure 2 This is the SDST_MSR_HFSP layout diagram;

[0052] Figure 3 This is a schematic diagram of heuristics in a low-level heuristic pool;

[0053] Figure 4 This is a performance comparison chart of the algorithms across 20 instances;

[0054] Figure 5 These are box plots for each algorithm;

[0055] Figure 6 This is a box plot of the present invention and two variant experiments A1 and A2. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0057] In a hybrid production line workshop, there are n single-crystal silicon rods of w different types. Each single-crystal silicon rod must pass through k sequentially connected processing stages according to a predetermined order. Each processing stage i has m... i Taiwan homogeneous parallel machines, of which m i ≥1, and at least one stage is equipped with two or more parallel machines. When the single-crystal silicon rod reaches a certain stage i, it must be transferred from stage m. i One machine can be selected from the available machines for processing. Before each processing stage begins, the selected machine must consider a key factor—the set-up time dependent on the production sequence. This set-up time depends not only on the current product type of the monocrystalline silicon rod but also on the product type of the monocrystalline silicon rod previously processed on that machine, exhibiting a clear sequence correlation. Therefore, the goal of this invention is to rationally allocate processing machines to n monocrystalline silicon rods at each processing stage and determine the processing sequence on each machine. Due to the diverse product types and large production scale of monocrystalline silicon rods, without a suitable scheduling scheme, a large accumulation of set-up time will occur, significantly reducing production efficiency and extending delivery cycles. To address this, this invention proposes a monocrystalline silicon rod hybrid production line scheduling method with sequence-dependent set-up time, aiming to minimize time losses caused by set-up time while ensuring product changeover efficiency, thereby improving overall production line operating efficiency.

[0058] Example 1:

[0059] like Figures 1-6 As shown, according to a first aspect of the present invention, a method for scheduling a single-crystal silicon rod hybrid production line with sequence-related setting time is provided, comprising:

[0060] Step 1: Based on the requirement of minimizing the maximum completion time in the production scheduling of a hybrid monocrystalline silicon rod production line, a mathematical model for the scheduling problem of a hybrid monocrystalline silicon rod production line with sequence-dependent time settings is established. The mathematical model includes an objective function and constraints. The objective function primarily minimizes the maximum completion time of the operation. Table 1 contains the parameters of the mathematical model. Figure 2 The SDST_MSR_HFSP layout diagram illustrates the scheduling problem of a hybrid production line for monocrystalline silicon rods with time-dependent sequential settings. Figure 2The set of pending production operations refers to the w types of single-crystal silicon rods to be processed. i This represents the i-th processing stage. Each job can be processed on any machine in the i-th processing stage. After all processing stages are completed, a set of w products with all processing completed is obtained.

[0061] For ease of description, the SDST_MSR_HFSP model studied in this invention uses "job" to refer to single-crystal silicon rods during modeling and is developed based on the following assumptions:

[0062] Each machine in the same phase can only process one job at any given time;

[0063] Each job can only be processed by one machine at any given time;

[0064] All jobs and machines are available at time 0.

[0065] Preemption is not allowed; once a task begins, its current process cannot be interrupted.

[0066] Processing stages cannot overlap; that is, the next processing stage of a certain operation can only begin after the previous processing stage has been completed.

[0067] An infinitely large intermediate buffer is provided between each adjacent stage to prevent congestion.

[0068] Table 1

[0069]

[0070] The mathematical model formula for SDST_MSR_HFSP is as follows.

[0071] Objective: Minimize z = C max (1)

[0072] subject to:

[0073]

[0074]

[0075] Formula (1) indicates that the optimization objective z of the mathematical model is to minimize the maximum completion time C. max .

[0076] Formulas (2) to (12) represent the constraints, specifically:

[0077] Formula (2) is the first constraint, representing the maximum completion time C. max The completion time is greater than or equal to the completion time of each task.

[0078] Formula (3) is the second constraint, which stipulates that each job can only be assigned to one machine in each processing stage.

[0079] Formula (4) is the third constraint, which means that each machine can only process one job at a time.

[0080] Formula (5) is the fourth constraint, which represents the setup time required to process different jobs sequentially on the same machine in the same processing stage.

[0081] Formula (6) is the fifth constraint, which represents the constraint on the start time and completion time when each machine processes a workpiece.

[0082] Formula (7) is the sixth constraint, which means that the subsequent processing stage can only begin after the previous processing stage is completed.

[0083] Formula (8) is the seventh constraint, which represents the completion time constraint considering the sequence correlation setting time.

[0084] Formula (9) is the eighth constraint, which means that the completion time of a task is equal to its start time plus the processing time.

[0085] Formula (10) is the ninth constraint, indicating that each job is processed in the order of processing stages.

[0086] Formulas (11) and (12) are time constraints that ensure that the start time and completion time of the operation are not less than 0.

[0087] Step 2: Flowchart of the reinforcement learning algorithm based on Q-learning is as follows. Figure 1 As shown, the specific process is as follows:

[0088] Step 2.1: Design an encoding and decoding scheme for the scheduling problem of a single-crystal silicon rod hybrid production line with sequence-related time settings; the encoding scheme adopts a two-dimensional list representation, and each two-dimensional list consists of a two-element sublist; where the first element in the sublist represents the job number and the second element represents the product type to which the job belongs;

[0089] The encoding scheme is closely related to the problem to be solved. This invention aims to solve the scheduling problem SDST_MSR_HFSP of a mixed production line workshop for monocrystalline silicon rods with sequence-related setting times. It determines the order of production operations (monocrystalline silicon rods) and the setting time required for switching between operations. Therefore, for SDST_MSR_HFSP, each feasible solution π is represented by a two-dimensional list. Each two-dimensional list consists of two-element sublists. The first element in each sublist represents the operation number, and the second element represents the product type to which the operation belongs. For example, π = [[1,2],[4,1],[2,1],[3,5]] indicates that the processing order of operations is operation 1, operation 4, operation 2, operation 3, and the types of the preceding and following operations can be known (for example, [1,2] indicates that operation 1 belongs to product type 2, and so on). During the decoding process, the processing order of operations and machine allocation must be considered simultaneously. For the processing order of operations, the processing order in the first stage can be directly reflected by the encoding scheme; while in the remaining stages, the processing order of operations is determined by the "First-Come, First-Served" (FCFS) rule. Regarding machine allocation, since multiple machines operate in parallel at each stage and the setup time related to the processing sequence needs to be considered, this invention improves the "earliest idle" heuristic rule. Specifically, the improved "earliest idle" heuristic rule checks whether there is an available machine that can be directly processed and is of the same type as the task to be processed. If so, it comprehensively compares two preset options and ultimately selects the machine that can achieve the earliest production start time. Otherwise, if it does not exist, it directly selects the earliest idle machine without comparing the two preset options. The two preset options are: selecting the earliest idle machine that requires a setup time, and selecting a machine that does not require a setup time but is not the earliest idle machine (the machine that is not the earliest idle machine is the second earliest in the sorting). This improved heuristic rule fully considers the production characteristics of a mixed assembly line workshop, meaning that it does not necessarily select the earliest idle machine, but rather comprehensively considers the impact of setup time. This method not only improves production efficiency but also optimizes resource utilization and ensures the smooth operation of the production process.

[0090] Step 2.2: Construct a low-level heuristic pool;

[0091] Five simple yet effective heuristics were designed to form low-level heuristic pools. Low-level heuristic pools (LLHs) can perform neighborhood searches on job sequences. The specific process of LLHs is as follows: Figure 3 As shown in the diagram (the same color indicates that the work belongs to the same product type):

[0092] First heuristic LLH1: Reverse differential operation: Randomly select two non-adjacent single-crystal silicon rods with a distance greater than 3 from the single-crystal silicon rod sequence, and reverse the sequence between these two silicon rods.

[0093] The second heuristic LLH2: Swap difference operation: Randomly select two adjacent single crystal silicon rods from the sequence of single crystal silicon rods, and then swap their positions.

[0094] The third heuristic LLH3: Permutation difference operation: Randomly select two non - adjacent single crystal silicon rods from the sequence of single crystal silicon rods and swap their positions.

[0095] The fourth heuristic LLH4: Insertion difference operation 1: Randomly select two single crystal silicon rods i and j at different positions from the sequence of single crystal silicon rods (position relationship: i < j), and insert single crystal silicon rod i in front of or behind j.

[0096] The fifth heuristic LLH5: Insertion difference operation 2: Randomly select two single crystal silicon rods i and j at different positions from the sequence of single crystal silicon rods (position relationship: i < j), and insert single crystal silicon rod j in front of or behind i.

[0097] Step2.3, State and action design in Q - learning;

[0098] State design:

[0099] Divide the state space S into non - overlapping blocks, specifically: S = {S1, S2, S3, S4} = {(0, 1), [1, 1.5), [1.5, 2), [2, ∞)}; where S1, S2, S3, S4 represent state 1, state 2, state 3, and state 4.

[0100] Let π represent the solution before performing an action, and π' represent the solution after performing an action; calculate the improvement ratio r used to represent the reward based on the solution before performing the action and the solution after performing the action, and determine the next state to which the solution belongs according to the improvement ratio r.

[0101] Specifically: The core of Q - learning is to learn the Q - function, and the state is one of the inputs of the Q - function, directly affecting the evaluation of the Q - value. Since the solution space of the SDST_MSR_HFSP problem is huge, it is not realistic to directly use the solution to represent the state. Therefore, in this invention, the state space S is first divided into non - overlapping blocks S = {S1, S2, S3, S4} = {(0, 1), (1, 1.5), [1.5, 2), [2, ∞)}. Then let π represent the solution before performing an action, and π' represent the solution after performing an action. According to the actual situation of the objective value of the solution, determine the improvement ratio r, and determine the next state to which the solution belongs according to the improvement ratio r; the expression of the improvement ratio r is:

[0102]

[0103] In the formula, f(π) represents the target value of the solution π before the action is performed; f(π') represents the target value of the solution π after the action is performed. The target value is the value obtained based on the objective function in the mathematical model.

[0104] Action Design: In each state, the agent needs to select an action to perform, thereby interacting with the environment. This invention uses five heuristics as search operators, which are also optional actions in Q-learning. Therefore, the action set is defined as Action = {LLH1, LLH2, LLH3, LLH4, LLH5};

[0105] Step 2.4 Initialize parameter settings, specifically including initializing the Q-table, state, action, learning rate λ, discount rate γ, and greedy rate ε;

[0106] Step 2.5: Generate the initial solution π0 using a metaheuristic method;

[0107] Step 2.6 Calculate the target value f(π0) of the initial solution: Call the objective function in the mathematical model to calculate the minimum maximum completion time corresponding to the initial solution, which is the target value of the initial solution;

[0108] Step 2.7: Take the initial solution as the current optimal solution;

[0109] Step 2.8: Randomly select a state from the state space as the initial state s0;

[0110] Step 2.9, Q-learning iterative optimization process:

[0111] Step 2.9.1: Obtain action a using an action selection strategy. t And perform action a on the current optimal solution. t A new solution π is obtained. * And obtain the new objective value f(π) * When a new solution is better than an old solution, the new solution is taken as the current optimal solution (the smaller the target value, the better). The action selection strategy used in this invention is a greedy strategy, as shown in Table 2 below.

[0112] Table 2

[0113]

[0114] Step 2.9.2: Set up the reward function, update the Q table according to the obtained reward r, and change the state s. t Update to the next state s t+1 The reward function and Q-table update formula are as follows:

[0115]

[0116] Among them, s t Represents the state at time t; r(s) t ,a t Let t be the time when s is in state s. t Next, execute action a t The reward r obtained later;

[0117] Step 2.9.3: Record the optimal solution corresponding to the optimal objective value at the current iteration number;

[0118] Step 2.10: Determine if the termination condition (maximum number of iterations = 1000) is met. If it is met, output the global optimal solution; otherwise, jump to Step 2.9.

[0119] Furthermore, to test the effectiveness of the Q-learning algorithm in solving SDST_MAR_HFSP, Q-learning was compared with Genetic Algorithm (GA), Variable Neighborhood Search (VNS), and Simulated Annealing (SA). All algorithms were implemented in Python 3.11, using PyCharm as the development tool, and ran on a computer equipped with an Intel(R) Core i5(TM)-12400U@2.50GHz CPU, 16GB of memory, and Windows 10.

[0120] To verify the experimental results, this invention designed 40 instances, each explicitly representing the number of monocrystalline silicon rods produced, the number of production stages, the number of product types, and the machine configuration. The scale of each instance can be represented by b_k_w_p-h. Experimental data were randomly generated, i.e., n (the number of monocrystalline silicon rods produced) was selected from {20, 40, 60, 80, 100}, k (the number of production stages) from {3, 5}, w (the number of product types) from {5, 10}, and ph represented the machine configuration scheme. Regarding machine configuration, considering the hybrid flow shop model and the actual production background of monocrystalline silicon rods, this invention defines two different machine configurations, represented by the letters a and b, as shown below:

[0121] a: The first phase will be equipped with 10 machines, and the remaining phases will each be equipped with 5 machines.

[0122] b: The first phase will be equipped with 20 machines, and the remaining phases will each be equipped with 5 machines.

[0123] By combining the number of monocrystalline silicon rods, production stages, product types, and machine configuration schemes, a total of 5×2×2×2=40 instances were obtained. Furthermore, other production data were generated based on the actual production scenario of the studied monocrystalline silicon rods and adjusted within a reasonable range to conform to the actual process. Specifically, considering the actual situation, the setup time of the first stage follows a uniform distribution of U[1,3], while the setup time of the remaining stages follows a uniform distribution of U[0.8,1.5]. Similarly, the unit processing time of the first stage follows a uniform distribution of U[5,10], while the other stages follow a uniform distribution of U[1,2]. This invention uses three controllable parameters: learning rate λ, discount rate γ, and greed rate ε. Finally, based on the orthogonal experimental results, the optimal parameter combination was determined: λ=0.1, γ=0.5, ε=0.6.

[0124] To evaluate the algorithm's performance, this invention selects relative error (RE) as the performance evaluation metric for the experimental results. Its calculation method is shown in the following formula:

[0125]

[0126] Among them, f finish_time_q f represents the total minimum maximum completion time obtained by algorithm q on a given instance. Finish_time_best This represents the optimal minimum maximum completion time obtained by all compared algorithms on the same instance. Therefore, a smaller RE value indicates better algorithm performance. To ensure the robustness of the results, all algorithms were run independently 20 times in each test instance, and their performance was evaluated using the Mean Relative Error (ARE). The formula for calculating ARE is as follows: where Y represents the total number of independent transports by the algorithm, and in this invention, Y is set to 20.

[0127]

[0128] To demonstrate the superior performance and robustness of the proposed Q-learning algorithm in the production of small, medium, and large-scale single-crystal silicon rods, 20 examples covering small, medium, and large production scales were selected. Table 3 lists the 20 selected examples covering small, medium, and large-scale production. Figure 4 Performance comparisons of the algorithms are given for 20 examples. Figure 5 These are the box plots for each algorithm.

[0129] Table 3

[0130]

[0131]

[0132] Depend on Figure 4It can be seen that the lowest mean relative error (ARE) was achieved in 20 examples covering small, medium and large production scales. The Q-learning algorithm proposed in this invention has outstanding performance in solving the production problem of single crystal silicon rod mixing flow workshop and has more advantages than other algorithms.

[0133] Figure 5 Box plots of the ARE values ​​for each algorithm are presented, showing that Q-learning not only boasts superior performance but also higher stability. Even in the worst-case scenario, it outperforms other algorithms, demonstrating strong optimization capabilities in solving the SDST_MSR_HFSP problem.

[0134] As can be seen from the above technical solution, in terms of completion time, compared with other methods, this invention can significantly reduce the maximum completion time of monocrystalline silicon rods in a hybrid production line, effectively improving production efficiency. In terms of production scale, this invention provides a practical and effective scheduling method for large-scale monocrystalline silicon rod production in actual industry. Verification shows that this invention can obtain a satisfactory solution in a short time even when facing large-scale production demands, and it is superior to other inventions.

[0135] To verify the effectiveness of the key design proposed in this invention, two variant experiments were further constructed as control groups:

[0136] Variant A1: In this variant, the selection of the action with the maximum Q value in the greedy strategy of this invention is replaced by a random selection mechanism, that is, a low-level heuristic (LLH) is randomly selected for application in each iteration, while the rest remains unchanged. This variant aims to examine the contribution of the greedy strategy in Q-learning introduced in this invention to the optimization of the heuristic scheduling strategy.

[0137] Variant A2: In this variant, the SDST_MSR_HFSP encoding and decoding scheme, consistent with the present invention, is still used. However, during the decoding stage, the improved "first idle machine selection rule considering setting time" of the present invention is replaced with the traditional "earliest machine principle" (i.e., always selecting the first idle machine regardless of the difference in setting time). This variant aims to verify whether the improvements in machine allocation rules of the present invention can effectively improve production efficiency and resource utilization.

[0138] To enhance the reliability of the results, experiments were conducted on all instances, and box plots were generated using the mean relative error (ARE) to compare the complete Q-learning algorithm with two variant experiments, A1 and A2, as shown below. Figure 6 This facilitates a direct and intuitive demonstration of the effectiveness of the key components of the algorithm of this invention. From Figure 6It can be seen that Q-learning has the lowest median, the smallest interquartile range (IQR), and relatively short upper and lower whiskers. These results demonstrate that the learning mechanism and the improved selection of the earliest idle machine rule in the algorithm proposed in this invention are indeed effective components, fully validating its effectiveness.

[0139] Example 2:

[0140] A scheduling system for a hybrid production line of monocrystalline silicon rods with sequence-dependent set times includes: a first module for executing Step 1: determining the objective function and constraints based on the requirement of minimizing the maximum completion time in the production scheduling of the hybrid production line of monocrystalline silicon rods, to establish a mathematical model for the scheduling problem of the hybrid production line of monocrystalline silicon rods with sequence-dependent set times; and a second module for executing Step 2: solving the global optimal solution of the scheduling problem of the hybrid production line of monocrystalline silicon rods with sequence-dependent set times using a reinforcement learning algorithm based on Q-learning. For details regarding the modules not described above, please refer to the relevant descriptions in other embodiments.

[0141] Example 3:

[0142] A processor for running a program, wherein the program executes the single-crystal silicon rod hybrid production line scheduling method with sequence correlation setting time as described in Example 1.

[0143] Example 4:

[0144] A computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the single-crystal silicon rod mixing production line scheduling method with sequence correlation setting time as described in Embodiment 1.

[0145] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0146] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A single crystal silicon rod hybrid flow shop scheduling method with sequence dependent setup times, characterized by, Comprise: Step1, according to the requirement of minimizing the maximum completion time in single crystal silicon rod mixed flow shop scheduling, the objective function and constraint condition are determined to establish the mathematical model of single crystal silicon rod mixed flow shop scheduling problem with sequence related setup time; Step2, based on the Q learning as the core of reinforcement learning algorithm, the global optimal solution of single crystal silicon rod mixed flow shop scheduling problem with sequence related setup time is solved, specifically: Step2.1, the encoding and decoding scheme of single crystal silicon rod mixed flow shop scheduling problem with sequence related setup time is designed; Step2.2, according to the preset heuristic, the low-level heuristic pool is constructed; Step2.3, the state and action in Q learning are designed: the state is designed according to the improvement rate, and the action set is constructed according to the low-level heuristic pool; Step2.4, the parameter setting is initialized; Step2.5, the initial solution is generated by meta-heuristic method; Step2.6, the objective value of the initial solution is calculated according to the objective function in the mathematical model; Step2.7, the initial solution is taken as the current optimal solution; Step2.8, a state is randomly selected from the state space as the initial state; Step2.9, according to the state, action set and initial setting parameters, the Q learning iterative optimization process is executed to obtain the optimal solution corresponding to the optimal objective value under the current iteration times; Step2.10, whether the termination condition is met is judged, if yes, the global optimal solution is output; Otherwise, jump to Step2.

9.

2. The single crystal silicon rod hybrid flow-shop scheduling method with sequence-dependent setup times according to claim 1, wherein, The objective function of the mathematical model takes minimizing the maximum completion time as the optimization goal;The constraint conditions include: First constraint, the first constraint is that the maximum completion time is greater than or equal to the completion time of each job; Second constraint, the second constraint specifies that each job can only be assigned to one machine at each processing stage; Third constraint, the third constraint indicates that each machine can only process one job at the same time; Fourth constraint, the fourth constraint indicates that the setup time is required for processing different jobs in the same machine at the same processing stage; Fifth constraint, the fifth constraint indicates the constraints on the start time and completion time of each machine when processing different workpieces; Sixth constraint, the sixth constraint indicates that the subsequent processing stage must be started after the previous processing stage is completed; Seventh constraint, the seventh constraint indicates the completion time constraint considering sequence related setup time; Eighth constraint, the eighth constraint indicates that the completion time of the job is equal to its start time plus processing time; Ninth constraint, the ninth constraint indicates that each job is processed in the order of processing stage; Time constraint, used to ensure that the start time and completion time of the job are not less than 0; Wherein, the job represents a single crystal silicon rod.

3. The single crystal silicon rod hybrid flow-shop scheduling method with sequence dependent setup times of claim 1, wherein, The encoding and decoding scheme comprises: Encoding scheme, the encoding scheme adopts two-dimensional list representation, each two-dimensional list is composed of two-element sub-list;Wherein, the first element in the sub-list represents the job number, and the second element represents the product type to which the job belongs; a decoding scheme which takes into account the processing order of jobs and the machine assignment; for the processing order of jobs, the processing order of the first stage is directly reflected by the encoding scheme; while for the remaining stages, the processing order of jobs is determined by the first-come-first-served rule; for the machine assignment, an improved first idle heuristic rule is used, which consists in checking whether there is an available machine which can directly process and is of the same type as the job to be processed; wherein the job represents a single crystal silicon rod.

4. The single crystal silicon rod hybrid flow-shop scheduling method with sequence-dependent setup times of claim 1, wherein, The low-level heuristic pool includes a plurality of heuristics, including: reverse order difference operation, exchange difference operation, permutation difference operation, insertion difference operation.

5. The single crystal silicon rod hybrid flow-shop scheduling method with sequence-dependent setup times of claim 1, wherein, The state and action in the Q learning are designed, including: State design: Divide the state space into non-overlapping blocks; Let π represent the solution before performing the action, and π' represent the solution after performing the action; calculate the improvement ratio for representing the reward according to the solution before performing the action and the solution after performing the action, and determine the next state to which the solution belongs according to the improvement ratio; Action design: all heuristics in the low-level heuristic pool are constructed into an action set.

6. The single crystal silicon rod hybrid flow-shop scheduling method with sequence-dependent setup times of claim 1, wherein, The Step2.9 includes: Step2.9.1, acquire action by using action selection strategy; and execute action on current optimal solution to obtain new solution π * , and obtain new solution target value f(π * ), when new solution is better than old solution, take new solution as current optimal solution; Step2.9.2, setting a reward function r, updating the Q table according to the obtained reward, and updating the state to the next state; Step2.9.3, record the optimal solution corresponding to the optimal target value in the current iteration number.

7. The single crystal silicon rod hybrid flow-shop scheduling method with sequence-dependent setup times according to claim 6, wherein, The reward function expression is: In the formula, f(π) represents the objective value of the solution π before the action is performed; f(π * ) represents the objective value of the solution after the action is performed, that is, the new solution objective value.

8. A single crystal silicon rod hybrid flow shop scheduling system with sequence dependent setup times, characterized by, A module of the single crystal silicon rod hybrid flow shop scheduling method with sequence related setup time based on any one of claims 1-7.

9. A processor, comprising: The processor is used to run a program, wherein the program performs the single crystal silicon rod hybrid flow shop scheduling method with sequence related setup time according to any one of claims 1-7 when running.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a stored program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to perform the single crystal silicon rod hybrid flow shop scheduling method with sequence related setup time according to any one of claims 1-7 when the program runs.

Images