Packaging test workshop intelligent scheduling algorithm based on multi-scale information fusion driving

Abstract

The invention provides a packaging test workshop intelligent scheduling algorithm based on multi-scale information fusion driving, and belongs to the field of intelligent optimization scheduling, and the method comprises the steps: firstly analyzing a workshop production process and constraint conditions, and constructing a multi-target optimization model with the target of minimizing the maximum completion time, the total mold change time and the total tardiness time; a scheduling framework fusing an LSTM-Informer prediction model and an ITD3 algorithm is provided, prediction features are generated through multi-source data collection and fusion, multi-scale prediction of a buffer area state is realized by using the LSTM-Informer, and the ITD3 algorithm is driven based on prediction information to carry out dynamic decision making; and meanwhile, a composite scheduling rule and a layered reward mechanism are designed to improve the solving efficiency. According to the method, the dynamic scheduling problem in actual production can be effectively solved, and the maximum completion time, the total die change time and the total tardiness time are remarkably shortened.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing for semiconductor packaging and testing, and more specifically to an intelligent scheduling optimization method driven by multi-source data fusion for packaging and testing workshops. Background Technology

[0002] As the core of the information technology industry, the semiconductor industry is a strategic, fundamental, and pioneering industry that supports economic and social development and safeguards national security. The packaging and testing process, as a crucial downstream step in the semiconductor manufacturing process, directly impacts chip performance, reliability, and cost control. With the market's increasing demand for diversified and highly integrated chips, the production model in packaging and testing workshops is gradually shifting towards multi-variety, small-batch, and mixed automated production lines, significantly increasing the complexity of production scheduling issues.

[0003] Packaging and testing workshops typically contain multiple processing centers, each equipped with multiple parallel machines (homogeneous or heterogeneous). The products to be processed are diverse and must flow sequentially according to fixed processes, forming a typical hybrid assembly line environment. In this environment, production scheduling needs to address three core issues: first, workpiece selection, i.e., how to allocate the most suitable workpieces to idle machines; second, buffer state prediction, since workpieces need to enter the buffer zone for the next process after completing one process, and the buffer state changes dynamically over time, accurate prediction of its future state is crucial for scheduling decisions; and third, machine mold changing, different products require mold changing operations during processing on the same machine. The mold changing time depends on the types of products being processed before and after, and the mold changing process cannot be interrupted; frequent mold changing significantly reduces production efficiency.

[0004] Currently, packaging and testing shop scheduling faces multiple challenges: On the one hand, it requires optimizing multiple mutually constraining objectives simultaneously, including minimizing maximum completion time, total mold changeover time, and total delay time; on the other hand, the dynamic and complex nature of multi-source data within the shop (such as the status of machining center buffers, equipment processing time, mold changeover time, etc.) makes it difficult for traditional scheduling methods to accurately capture real-time changes in the production process, resulting in insufficient adaptability of scheduling solutions. For example, if mold changeover decisions are made solely based on the current buffer status, the dynamic nature of future workpiece arrivals may be ignored, leading to frequent mold changes, production congestion, and ultimately, decreased production efficiency and increased risk of delays.

[0005] Furthermore, traditional scheduling algorithms have limitations when dealing with multi-objective optimization, dynamic environment adaptability, and large-scale complex scenarios: they either struggle to balance multiple optimization objectives or lack sufficient ability to fuse and predict multi-source time-series data, failing to provide accurate forward-looking information for scheduling decisions. Therefore, considering the characteristics of hybrid production lines in semiconductor packaging and testing, researching how to fuse multi-source production data, construct accurate state prediction models, and combine them with intelligent optimization algorithms to achieve dynamic and efficient production scheduling is of great significance for improving workshop production efficiency, reducing operating costs, and enhancing market responsiveness. Summary of the Invention

[0006] To address current technical problems, the main objective of this invention is to provide an intelligent scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion. This method first analyzes the workshop production process and constraints, constructing a multi-objective optimization model aimed at minimizing the maximum completion time, total mold change time, and total delay time. A scheduling framework integrating the LSTM-Informer prediction model and the ITD3 algorithm is proposed: predictive features are generated through multi-source data acquisition and fusion; LSTM-Informer is used to achieve multi-scale prediction of buffer states; and the ITD3 algorithm is driven by this predictive information for dynamic decision-making. Simultaneously, composite scheduling rules and a hierarchical reward mechanism are designed to improve solution efficiency. This method effectively solves the dynamic scheduling problem in actual production, significantly reducing the maximum completion time, total mold change time, and total delay time.

[0007] To achieve the above-mentioned technical features, the objective of this invention is as follows: a smart scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion, the method comprising the following steps: Step 1: Collect real-time information on the processing status and buffer status of each processing center in the packaging and testing workshop, including equipment utilization rate, equipment mold change rate, equipment idle rate, and the number of work-in-process products for various products; Step 2: Perform time alignment and structuring on the above multi-source data, integrate the status information of the current process and adjacent processes, and construct a unified high-dimensional temporal feature vector as the input of the LSTM-Informer model; Step 3: Extract short-term dynamic features using LSTM and extract long-term trend information using Informer to achieve multi-scale prediction of the status of buffer products within the future time window; Step 4: Based on the predicted information, construct the mold change cost-effectiveness index, and combine it with the delay status of each workpiece to establish composite scheduling rules to guide the dynamic matching and priority ranking of equipment and workpieces. Step 5: Train the scheduling strategy based on the improved ITD3 algorithm, and introduce a state-action similarity screening mechanism and hierarchical reward design to improve the stability and convergence speed of the strategy.

[0008] Preferably, the specific process of Step 1 is as follows: Collect status data from each processing center in the workshop, including: 1) The average processing utilization rate and its standard deviation for each machine; 2) The average mold change rate and its standard deviation for each machine; 3) The average idle rate and its standard deviation for each machine; 4) The number of work-in-process items of each type in each buffer zone; The original data was aligned according to a unified timestamp to form an original dataset containing multidimensional time series features, which provided high-quality time series data for subsequent prediction models.

[0009] Preferably, the specific process of Step 2 is as follows: Structured processing of multi-source state data includes the following operations: 1) Integrate the buffer states of the current process and the previous process to establish the relationship between the front and back processing centers; 2) Concatenate the machine processing status characteristics with the buffer state; 3) Transform the original features into a unified high-dimensional vector representation through value embedding, temporal embedding, and positional embedding; 4) Finally, a time-series input sequence with a unified structure is generated as the input to the LSTM-Informer model.

[0010] Preferably, the specific process of Step 3 is as follows: Constructing an LSTM-Informer prediction model to perform multi-step predictions of future states for the buffer state, specifically including: 1) The LSTM module is used to capture short-term changes and local fluctuations in features; 2) The Informer module introduces a sparse attention mechanism to extract long-term dependencies and compress redundant information; 3) The encoder module extracts core temporal features through a multi-head attention layer and feature distillation; 4) The decoder module enables accurate prediction of the quantity of buffer product inventory within a future time window.

[0011] Preferably, the specific process of Step 4 is as follows: After obtaining the predicted information of future states, a cost-effectiveness index for model switching is constructed based on the prediction results, and a composite scheduling rule is designed, as follows: 1) Calculate the cost-effectiveness of mold changeover based on the mold changeover time, workpiece processing time, and predicted work-in-process inventory required for each product changeover. : (1) in, For workpiece index, For machining center or process index, For machine indexing, For product category indexing, This represents the total number of product categories. The total number of workpieces. To predict the window length, For the first Workpieces in this type of product The This process is done on the machine. On the processing time, For machines Processing the first The first product is converted into the second processing. The mold change time required for this product In order to be in At that moment, the Within the buffer zone of the first processing center Forecast work-in-process inventory for this product Indicates if workpiece Belongs to the For this product, ;otherwise ; 2) Based on the urgency of the workpiece E 1. Determine the scheduling priority, and calculate it using the following formula: (2) in, For workpiece index, Here, n is the index for the machining center or process, and n is the total number of workpieces. This represents the total number of processing stages. For process Average processing time across all available machines For workpiece Delivery time, For workpiece The expected start time for processing is calculated using the following formula: (3) in, For workpiece index, For machine indexing, For product category indexing, For workpiece The total number of completed processes, For machining center The total number of machines included. For the first Workpieces in this type of product The This process is done on the machine. End time on For machines Processing the first Workpieces in this type of product The The end time of each process step; 3) Construct a composite scheduling rule system, including the following three cases: ① If there are overdue items, select the item with the highest urgency. ②If there are no pending workpieces, select the workpiece that matches the current mold; ③ If neither of the first two conditions is met, then the workpiece with the smallest mold change cost-effectiveness index RH1 is selected for processing. 4) The above scheduling rules serve as one of the action space selections for ITD3 reinforcement learning.

[0012] Preferably, the specific process of Step 4 is as follows: After obtaining the predicted information of future states, a cost-effectiveness index for model switching is constructed based on the prediction results, and a composite scheduling rule two is designed, as follows: 1) Calculate the cost-effectiveness of mold changeover based on the mold changeover time, workpiece processing time, and predicted work-in-process inventory required for each product changeover. :

[0013] in, For workpiece index, For machining center or process index, For machine indexing, For product category indexing, The total number of product categories, The total number of workpieces. To predict the window length, For the first Workpieces in this type of product The This process is done on the machine. On the processing time, For machines Processing the first The first product is converted into the second processing. The mold change time required for this product In order to be in At that moment, the Within the buffer zone of the first processing center Forecast work-in-process inventory for this product Indicates if workpiece Belongs to the For this product, ;otherwise ; 2) Based on the urgency of the workpiece Determine the scheduling priority using the following formula: (5) in, For workpiece index, Here, n is the index for the machining center or process, and n is the total number of workpieces. This represents the total number of processing stages. For process Average processing time across all available machines For workpiece Delivery time, For workpiece The expected start time of processing is calculated as shown in equation (3); 3) Construct a composite scheduling rule system, including the following three cases: ① If there are overdue items, select the item with the highest urgency. ②If there are no pending workpieces, select the workpiece that matches the current mold; ③ If neither of the first two conditions is met, then select the mold change cost-effectiveness index. Process the smallest workpiece; 4) The above scheduling rules serve as one of the action space selections for ITD3 reinforcement learning.

[0014] Preferably, the specific process of Step 4 is as follows: After obtaining the prediction information of future states, a cost-effectiveness index for model switching is constructed based on the prediction results, and a composite scheduling rule three is designed, as follows: 1) Calculate the cost-effectiveness of mold changeover based on the mold changeover time, workpiece processing time, and predicted work-in-process inventory required for each product changeover. :

[0015] in, For workpiece index, For machining center or process index, For machine indexing, For product category indexing, This represents the total number of product categories. The total number of workpieces. To predict the window length, For the first Workpieces in this type of product The This process is done on the machine. On the processing time, For machines Processing the first The first product is converted into the second processing. The mold change time required for this product In order to be in At that moment, the Within the buffer zone of the first processing center Forecast work-in-process inventory for this product Indicates if workpiece Belongs to the For this product, ;otherwise ; 2) Based on the urgency of the workpiece The scheduling priority is determined by the formula shown in formula (5); 3) Construct a composite scheduling rule system, including the following three cases: ① If there are seriously delayed workpieces: Then select the workpiece with the highest urgency. ②If there are no pending workpieces, select the workpiece that matches the current mold; ③ If neither of the first two conditions is met, then select the mold change cost-effectiveness index. Process the smallest workpiece; 4) The above scheduling rules serve as one of the action space selections for ITD3 reinforcement learning.

[0016] Preferably, the specific process of Step 5 is as follows: The specific operation process based on the improved TD3 reinforcement learning algorithm is as follows: 1) First, obtain the state features. Input the state features into online In the network; 2) with Greedy strategy selects the optimal action ; 3) Interact with the environment to receive instant rewards and the next state and form an experience group

[0017] 4) Write experience into the playback pool Previously, the algorithm introduced a similarity check mechanism, which means that if the similarity between the current experience and the historical experience exceeds a given threshold, the current experience is determined to be a redundant sample and will not be stored.

[0018] Preferably, the specific process of Step 5 is as follows: ITD3 employs a dimensionless tiered reward mechanism, with the specific calculation method as follows: 1) First, compare and Actual delay rate at any time ,like The actual delay rate at any given time is less than The actual overdue rate at any given time is calculated as a positive reward, if The actual delay rate at any given time is greater than Actual overdue rate at any time The calculation multiplier is a negative reward; the specific calculation formula is as follows: ; 2) If neither of the above two conditions is met, then the assessment of the delay rate will be applied sequentially using a similar method. Calculate the reward, that is Machine idle rate Calculate the reward, that is and machine mold change rate Calculate the reward, that is ; 3) If none of the above conditions are met, then a reward will be given. .

[0019] The present invention has the following beneficial effects: This invention constructs a multi-objective optimization model that minimizes the maximum completion time, total mold change time, and total delay time. It generates high-dimensional time-series features through multi-source data acquisition and fusion, utilizes LSTM-Informer to achieve multi-scale prediction of buffer states, and combines the sample similarity experience replay mechanism, composite scheduling rules, and hierarchical reward mechanism of the ITD3 algorithm to improve the foresight and optimization efficiency of dynamic scheduling. Extensive numerical simulation experiments demonstrate that it can effectively reduce the maximum completion time, total mold change time, and total delay time during the scheduling process, thereby reducing processing waiting time and production costs, and improving enterprise competitiveness. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] Figure 1 This is the layout of the packaging and testing workshop for this invention.

[0022] Figure 2 This is the algorithm framework of the LSTM–Informer-ITD3 of this invention.

[0023] Figure 3 This is a schematic diagram of multi-source data acquisition according to the present invention.

[0024] Figure 4 This invention relates to the LSTM-Informer framework.

[0025] Figure 5 This is the LSTM-Informer model encoder of the present invention.

[0026] Figure 6 This is the dual-branch coding structure of the present invention.

[0027] Figure 7 This invention relates to the LSTM-Informer model decoder.

[0028] Figure 8 This is a diagram illustrating the overall framework of the ITD3 algorithm of this invention.

[0029] Figure 9 The graph shows the computation results of the six reinforcement learning algorithms of this invention on 27 examples.

[0030] Figure 10 The graph shows the average rank results of different reinforcement learning algorithms in this invention. Detailed Implementation

[0031] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0032] Example 1: This embodiment addresses the scheduling problem of a hybrid semiconductor packaging and testing workflow. Specifically, the problem studies scheduling a total of [number missing] [units missing]. in For the number of product types, For the first The number of workpieces for a product, including One machining center (each machining center is equipped with) In a workshop with parallel machines, workpieces are processed sequentially according to fixed procedures, and the processing order of workpieces on each machine is rationally arranged.

[0033] Throughout the entire packaging and testing process, this method primarily considers three optimization objectives: minimizing the maximum completion time. Total mold change time Total delay time The total mold change time is related to the types of products processed before and after the machine.

[0034] The entire packaging and testing process of a workpiece involves several key time factors: mold change time is the preparation time required for a machine to switch from processing one product to processing another, which depends on the types of products processed before and after the machine; processing time is the actual processing time of the workpiece on the machine, which is related to the product type, the workpiece, and the machine used; in addition, the buffer before the workpiece enters the next process after completing one process, although the buffer is infinite, the impact of its state changes on scheduling must be considered, and the flow of the workpiece between processes also implies time costs.

[0035] Based on this, the present invention provides an intelligent scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion, the process of which is as follows: Figure 2 As shown: The algorithm consists of three core modules: multi-source data acquisition and prediction feature generation, LSTM-Informer-based multi-scale prediction, and the ITD3 algorithm driven by multi-scale prediction information fusion. Module 1 involves an agent collecting state information from each processing center at each decision point, generating high-dimensional temporal features for prediction through concatenation and alignment. Module 2 is an LSTM-Informer joint prediction model that inputs the collected multi-source temporal data into the prediction model, using LSTM to capture short-term dynamic features while combining the Informer encoder-decoder architecture to extract long-term trends, achieving accurate prediction of the quantity of work-in-process in the buffer of each processing center within a future time window. Module 3 integrates the trained LSTM-Informer model with the ITD3 algorithm, introducing prediction information into action evaluation to construct a mold-changing cost-effectiveness index, forming a dynamic mold-changing strategy based on multi-scale prediction.

[0036] Example 2: This embodiment provides an intelligent scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion.

[0037] See Figure 1-8 First, for ease of explanation, let's clarify the symbols used in this example: Model assumptions: Considering the characteristics and manageability of the actual problem, the following reasonable assumptions are made about the problem: (1) Each machine can only process one workpiece at a time, and workpieces are prohibited from occupying the machine in advance.

[0038] (2) At time zero, all machines are in an available state, and the initial processing attributes of the machines are the same as the first type of processed product.

[0039] (3) The mold change time for different products on all machines is known, and the equipment is not allowed to be interrupted when changing molds.

[0040] (4) The processing time of different products on different machining center machines is known, and the processing process cannot be interrupted.

[0041] (5) All machining centers have unlimited buffer zones, meaning that the workpiece can immediately enter the buffer zone of the next machining center after machining is completed.

[0042] Symbol definition:

[0043] Scheduling model: This invention takes the packaging and testing workshop as the research object (workshop layout as follows). Figure 1 As shown), its production scheduling optimization problem can be defined as follows: the workshop contains One processing center ( Each machining center is equipped with parallel machine ( ≥2, supports homogeneous or heterogeneous machine types), total Products awaiting processing ( ≥2). Each product According to standard batch size If there are 100 workpieces, then the total number of workpieces is 100. And all workpieces must pass through in sequence. The next process. When the workpiece completes the first... After the first process, proceed directly to the next step. A buffer zone before each process step, and for each workpiece Delivery period is specified for all. If the completion time Exceed If a delay penalty occurs, no penalty will be imposed; otherwise, no penalty will be imposed. Before processing a workpiece, each machine can make a mold-changing decision based on the type of workpiece it has just processed and the type of workpiece to be processed. The mold-changing time is specified in the original text. Depends on the machine Product types before and after and It is worth noting that the type of workpiece to be processed should be determined based on the buffer state after the mold change is completed. Therefore, each time a mold change decision is made, the state of various products should be predicted using the mold change time as a prediction window in order to comprehensively evaluate whether it is worthwhile to change the mold or switch to which product workpiece.

[0044] Objective function: (7) in: (8) (9) (10) Constraints: (11) (12) (13) (14) (15) (16) (17) (18) (19) (20) Equation (11) is used to calculate the workpiece. Complete the first The end time of each process step; Equation (12) is used for calculating the machine. The total mold-changing time; Equation (13) defines the first... Workpieces in this type of product The This process is done on the machine. The relationship between the start and end times of processing on the surface; Equation (14) defines the first Workpieces in this type of product The The start time of the processing step on this machine is related to the machine's... The relationship between the earliest available time and the mold change time; Equation (15) is used to calculate the workpiece The total delay time; Equation (16) indicates that a workpiece can only belong to one type of product; Equation (17) indicates that a process can only be processed by one machine at a time; Equation (18) indicates that a machine can only process one product process at a time; Equation (19) defines the binary decision variables; Equation (20) gives the range of values ​​for the key parameters.

[0045] Scheduling Algorithm: ① Multi-source data acquisition and predictive feature generation The multi-source data acquisition mechanism covers the status of the processing center buffer and the equipment processing status. Status data is collected at the beginning and end of each process, and a sampling strategy with consistent timing is introduced. The multi-source data is then stitched and aligned based on the time axis (e.g., ...). Figure 3 (As shown). Specifically includes: (1) Processing Center Buffer Status Data: This invention establishes an independent buffer log for each processing center, recording the quantity changes of various products within the buffer at the start and end of the process, and then obtaining the real-time inventory level of each product through cumulative summation. Each record is marked with "timestamp, quantity of product type 1, quantity of product type 2, ... The quantity of each product is added to the buffer log of the corresponding processing center.

[0046] (2) Equipment processing status data: To ensure the consistency of the processing center's feature data in the time dimension, this invention performs equally spaced feature extraction based on the time axis. At the sampling time, the processing time and mold change time of each machine within the corresponding time window are statistically analyzed, and the following six features are constructed: ① Machine average utilization rate and standard deviation: ;(twenty one) ;(twenty two) in, For machines exist The utilization rate within a time period is calculated using the following formula: ;(twenty three) ② Machine average mold change utilization rate and standard deviation: ;(twenty four) (25) in, For machines exist The formula for calculating the mold change rate within a time period is as follows: (26) ③ Machine average idle rate and standard deviation: (27) (28) in, For machines exist The idle rate within a time period.

[0047] During feature extraction, all feature information is represented by a timestamp, MU_ave_ t MU_std_ t ,MCV_ave_ t ,MCV_std_ t MIdle_ave_ t MIdle_std_ tAdd it to the feature log of the corresponding processing center in the form of "". Among them, MU_std_ t This indicates the percentage of time each machine spends processing within a time window, reflecting the equipment's processing intensity and load balance. ,MCV_std_ t This indicates the percentage of time the machine spends on mold changing within the time window, reflecting the equipment's mold changing time and frequency. MIdle_std_ t This indicates the percentage of time the machine spends in a non-processing, non-molding state within the time window, reflecting the idle status of resources.

[0048] ② Multi-source data fusion mechanism based on adjacent processing centers To describe the correlation between equipment processing status and buffer status, this invention integrates data based on the processing center. Specifically, after completing multi-source data acquisition, the equipment processing status at each sampling time is aligned column-wise with its own and the buffer status of the previous processing center, and merged to generate structured sample data. This fusion method effectively enhances the correlation between upstream and downstream processes, providing high-quality time-series data for subsequent prediction models.

[0049] ③LSTM-Informer model encoder To enhance the model's ability to model extreme values, mutations, and trends in long-term series, this invention builds upon the original... Based on this, LSTM-Informer variant structures (such as...) were introduced. Figure 4 As shown), an LSTM module is inserted before the traditional Informer encoder module. This module takes the encoded time series as input, and its output is directly used as the input of the subsequent attention module (e.g., Figure 5 (As shown). Through this structure, the prediction model can first capture local temporal dependencies using LSTM, and then further construct global dependencies with the help of the sparse attention mechanism of Informer, thereby achieving effective fusion of local and global information.

[0050] It is worth noting that the attention mechanism employed by Informer is based on a sparse probabilistic sampling strategy. By calculating the dot product similarity between the query vector and the key vector, it retains only the key query points that significantly contribute to the prediction results, thus mitigating the limitations of traditional self-attention. Time complexity reduced to This significantly improves the model's efficiency in long sequence modeling. Building on this, to alleviate the redundancy issue caused by non-critical query points in the self-attention mechanism, the Informer encoder introduces a hierarchical feature distillation mechanism. This mechanism compresses and filters feature representations by introducing one-dimensional convolution and max-pooling operations between adjacent attention modules, guiding subsequent attention modules to focus on more representative high-value features. Simultaneously, to further enhance the encoder's robustness and its ability to perceive short-period patterns, the Informer model introduces a dual-branch encoding structure (such as...). Figure 6 (as shown) 1) In the main branch, the complete embedding sequence is taken as input and the global feature representation is extracted through three attention modules; 2) The auxiliary branch selects the latter half of the embedded sequence as input, uses the same attention mechanism, but extracts local feature representations only through two attention modules; 3) The feature matrices output by the main branch and the auxiliary branch are concatenated along the channel dimension to form a fused encoded representation. This dual-branch encoding introduces a multi-scale feature modeling mechanism while maintaining computational efficiency, effectively improving the encoder's performance in complex time-series prediction tasks.

[0051] ④LSTM-Informer model decoder The decoder architecture adopts a generative design concept, aiming to achieve parallel prediction of future multi-scale information (such as...). Figure 7 As shown), the details are as follows: 1) The input sequence consists of historical true values ​​and all-zero placeholders corresponding to the length of the prediction time window. After adding position encoding, a mask is used to prevent information leakage. 2) Input the input sequence into a sparse self-attention module to model the internal temporal dependencies; 3) The output of the decoder is used as the query vector Q, which is input into the multi-head cross-attention module along with the key vector K and value vector V output by the encoder, thereby fusing the global temporal features extracted by the encoder; 4) The decoder can generate multi-step prediction results for the entire prediction time window in one forward propagation, which significantly improves the efficiency and accuracy of multi-step prediction for long sequences.

[0052] ⑤ algorithm exist In the algorithm (e.g.) Figure 8 As shown), first obtain the state features. Then input the state features into online In the network, and with Greedy strategy selects the optimal action Then, interact with the environment to receive instant rewards. and the next state and form an experience group It is worth noting that experience is written into the replay pool. Previously, the algorithm introduced a similarity check mechanism, which means that if the similarity between the current experience and the historical experience exceeds a given threshold, the current experience is determined to be a redundant sample and will not be stored.

[0053] exist During the training phase of the algorithm, the priority is first calculated based on the temporal difference error of each sample in the empirical replay pool. And based on this, determine the probability of it being sampled. , specifically ,in To adjust the hyperparameters that influence the priority level, This represents the total number of samples in the experience replay pool. Subsequently, a batch of samples is sampled from the experience replay pool according to this probability and input into the online... In a network, calculate the target Value. To avoid The problem of overvaluation Two independent network and And take the minimum of the two as the target. The value is calculated using the following formula: (29) in As a discount factor, For the goal network.

[0054] Immediately afterwards, The network updates using the minimum mean square error as the loss. And whenever Network Update After that, The network will be updated once.

[0055] Finally, to ensure the convergence stability of the target network, a soft update strategy is adopted, the calculation formula of which is as follows: (30) (31) in This is the weighting factor.

[0056] ⑥ State Space To accurately characterize the workshop's operational status, this paper sets the state space as follows: ,in The machine status is represented by the following six characteristics: (i) the average utilization rate and standard deviation of the machine, (ii) the average mold change utilization rate and standard deviation of the machine, and (iii) the average idle rate and standard deviation of the machine. The status of a workpiece is indicated by the following four characteristics: (i) the average completion rate and standard deviation of the workpiece, and (ii) the estimated on-time rate and actual on-time rate of the workpiece.

[0057] ⑦ Action Space Three composite scheduling rules were designed to address the production needs and optimization goals of the packaging and testing workshop. All rules employ a "machine selection first, workpiece selection second" scheduling process, but each has its own emphasis on handling delays, workpiece selection, and mold change evaluation. These are explained in detail below: 1) Scheduling Rule 1: First, select the earliest available machine globally. Then, determine if there are any overdue workpieces among all the workpieces that the machine can currently process. If there are overdue workpieces, then schedule them according to their urgency. Select the workpiece with the most severe delays for processing; otherwise, prioritize processing workpieces with the same type of mold. If no workpieces with the same type of mold are available, choose the mold replacement option based on cost-effectiveness. The most critical workpieces are processed. Among them, the urgency level... Cost-effectiveness of mold changing The calculation formula is as follows: (32) (33) in, Indicates workpiece The expected start time for processing is calculated using the following formula: (34) 2) Scheduling Rule 2: Similar to Scheduling Rule 1, first select the earliest available machine globally, then determine if there are any overdue workpieces among all the workpieces that the machine can currently process. If there are overdue workpieces, then prioritize them according to their urgency. Select the workpiece with the most severe delays for processing; otherwise, prioritize processing workpieces with the same type of mold. If no workpieces with the same type of mold are available, choose the mold replacement option based on cost-effectiveness. The most critical workpieces are processed. Among them, the urgency level... Cost-effectiveness of mold changing The calculation formula is as follows: (35)

[0058] in, Indicates workpiece The expected start time of processing is calculated as shown in equation (34); 3) Scheduling Rule 3: Similar to Scheduling Rule 2, first select the earliest available machine globally, then determine if there are any severely delayed workpieces among all the workpieces that the machine can currently process. If there are severely delayed workpieces (i.e., ... ), then according to the degree of urgency Select the workpiece with the most severe delays for processing; otherwise, prioritize processing workpieces with the same type of mold. If no workpieces with the same type of mold are available, choose the mold replacement option based on cost-effectiveness. The highest-performing workpieces are processed. Among these, the cost-effectiveness of mold changing is considered. The calculation formula is as follows:

[0059] in, .

[0060] ⑧ Tiered reward mechanism To maximize the cumulative reward of the ITD3 algorithm, a method based on the actual workpiece delay rate was designed. ), workpiece evaluation delay rate ( ), machine idle rate ( ) and machine mold change rate ( A dimensionless tiered reward mechanism. Specifically: 1) First, compare and The actual delay rate at any given time, if The actual delay rate at any given time is less than The actual overdue rate at any given time is calculated as a positive reward, if The actual delay rate at any given time is greater than Actual overdue rate at any time The calculation multiplier is a negative reward; the specific calculation formula is as follows: ; 2) If neither of the above two conditions is met, then the bonus will be calculated using the assessed delay rate in a similar manner. The machine idle rate is used to calculate the reward, that is The reward is calculated based on the machine mold change rate, i.e. ; 3) If none of the above conditions are met, then a reward will be given. .

[0061] ⑨ Experience replay mechanism based on sample similarity This invention introduces an experience replay mechanism based on sample similarity, wherein each time a new experience arrives, several historical experiences are randomly selected from the experience replay pool, and their cosine similarity with the current state is calculated: (38) in Indicates a historical state. Indicates the current state. and These represent the corresponding actions. If... and If the current experience is redundant with historical experience, it will not be stored; otherwise, it will be stored.

[0062] To balance the exploratory nature and stability of the ITD3 algorithm, the threshold... The dynamic attenuation strategy is adopted, and the calculation formula is as follows: (39) in and These are the initial threshold and the final threshold, respectively. This represents the current number of training steps. The total number of workpieces. The number of processes, This is the scaling factor.

[0063] ⑩ Algorithm Example Comparison To verify the effectiveness of the algorithm proposed in this invention, three test case sets were randomly generated, comprising small-scale (cases 1-9), medium-scale (cases 10-18), and large-scale (cases 19-27) scenarios, each containing 9 test cases (abbreviated as "number of workpieces n × total number of processes S"). The test case generation rules are: (i) the workpiece arrival time follows an exponential distribution; (ii) the processing time of each process is within... (iii) Delivery time is determined by multiplying the sum of the average processing times of all processes by an amplification factor greater than 1 and the urgency level (DDT=1.0); (iv) Changeover time between different products is within... The process is randomly generated, and the mold change time is determined by the product type before and after the switch. Each test case contains four products, each with different processing times. The number of completed workpieces is counted by recording the work logs of each buffer, thus clarifying the completion status of each product.

[0064] This section compares five advanced reinforcement learning algorithms: DQN, DDQN, DDQN, D3QN, and AC. The computational results of the six deep reinforcement learning algorithms, obtained by running all the test cases described above, are as follows: Figure 9As shown in the figure. Based on this calculation, it can be concluded that: (i) the LSTM-Informer-ITD3 algorithm significantly outperforms the other five deep reinforcement learning algorithms in both HV and RPD metrics (average RPD value not lower than -35.25%); (ii) the Wilcoxon test p-values ​​of the LSTM-Informer-ITD3 algorithm and the other four advanced deep reinforcement learning algorithms in terms of HV metrics are 7.450E-08, 2.980E-08, 4.470E-08, 1.490E-08, and 9.199E-04, respectively, and their rank test results are as follows: Figure 10 As shown (the larger the value, the better), this indicates that the LSTM-Informer-ITD3 algorithm significantly outperforms the other five deep reinforcement learning algorithms.

Claims

1. A smart scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion, characterized in that, The method includes the following steps: Step 1: Collect real-time information on the processing status and buffer status of each processing center in the packaging and testing workshop, including equipment utilization rate, equipment mold change rate, equipment idle rate, and the number of work-in-process products for various products; Step 2: Perform time alignment and structuring on the above multi-source data, integrate the status information of the current process and adjacent processes, and construct a unified high-dimensional temporal feature vector as the input of the LSTM-Informer model; Step 3: Extract short-term dynamic features using LSTM and extract long-term trend information using Informer to achieve multi-scale prediction of the status of buffer products within the future time window; Step 4: Based on the predicted information, construct the mold change cost-effectiveness index, and combine it with the delay status of each workpiece to establish composite scheduling rules to guide the dynamic matching and priority ranking of equipment and workpieces. Step 5: Train the scheduling strategy based on the improved ITD3 algorithm, and introduce a state-action similarity screening mechanism and hierarchical reward design to improve the stability and convergence speed of the strategy.

2. The intelligent scheduling algorithm for packaging and testing workshops based on multi-scale information fusion as described in claim 1, characterized in that, The specific process of Step 1 is as follows: Collect status data from each processing center in the workshop, including: 1) The average processing utilization rate and its standard deviation for each machine; 2) The average mold change rate and its standard deviation for each machine; 3) The average idle rate and its standard deviation for each machine; 4) The number of work-in-process items of each type in each buffer zone; The original data was aligned according to a unified timestamp to form an original dataset containing multidimensional time series features, which provided high-quality time series data for subsequent prediction models.

3. The intelligent scheduling algorithm for packaging and testing workshops based on multi-scale information fusion as described in claim 1, characterized in that, The specific process of Step 2 is as follows: Structured processing of multi-source state data includes the following operations: 1) Integrate the buffer states of the current process and the previous process to establish the relationship between the front and back processing centers; 2) Concatenate the machine processing status characteristics with the buffer state; 3) Transform the original features into a unified high-dimensional vector representation through value embedding, temporal embedding, and positional embedding; 4) Finally, a time-series input sequence with a unified structure is generated as the input to the LSTM-Informer model.

4. The intelligent scheduling algorithm for packaging and testing workshops based on multi-scale information fusion as described in claim 1, characterized in that, The specific process of Step 3 is as follows: Constructing an LSTM-Informer prediction model to perform multi-step predictions of future states for the buffer state, specifically including: 1) The LSTM module is used to capture short-term changes and local fluctuations in features; 2) The Informer module introduces a sparse attention mechanism to extract long-term dependencies and compress redundant information; 3) The encoder module extracts core temporal features through a multi-head attention layer and feature distillation; 4) The decoder module enables accurate prediction of the quantity of buffer product inventory within a future time window.

5. The intelligent scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion as described in claim 1, characterized in that, The specific process of Step 4 is as follows: After obtaining the predicted information of future states, a cost-effectiveness index for model switching is constructed based on the prediction results, and a composite scheduling rule is designed, as follows: 1) Calculate the cost-effectiveness of mold changeover based on the mold changeover time, workpiece processing time, and predicted work-in-process inventory required for each product changeover. : ； （1） in, For workpiece index, For machining center or process index, For machine indexing, For product category indexing, This represents the total number of product categories. The total number of workpieces. To predict the window length, For the first Workpieces in this type of product The This process is done on the machine. On the processing time, For machines Processing the first The first product is converted into the second processing. The mold change time required for this product In order to be in At that moment, the Within the buffer zone of the first processing center Forecast work-in-process inventory for this product Indicates if workpiece Belongs to the For this product, ;otherwise ; 2) Based on the urgency of the workpiece E 1. Determine the scheduling priority, and calculate it using the following formula: ；（2） in, For workpiece index, Here, n is the index for the machining center or process, and n is the total number of workpieces. This represents the total number of processing stages. For process Average processing time across all available machines For workpiece Delivery time, For workpiece The expected start time for processing is calculated using the following formula: ；（3） in, For workpiece index, For machine indexing, For product category indexing, For workpiece The total number of completed processes, For machining center The total number of machines included. For the first Workpieces in this type of product The This process is done on the machine. End time on For machines Processing the first Workpieces in this type of product The The end time of each process step; 3) Construct a composite scheduling rule system, including the following three cases: ① If there are overdue items, select the item with the highest urgency. ②If there are no pending workpieces, select the workpiece that matches the current mold; ③ If neither of the first two conditions is met, then the workpiece with the smallest mold change cost-effectiveness index RH1 is selected for processing. 4) The above scheduling rules serve as one of the action space selections for ITD3 reinforcement learning.

6. The intelligent scheduling algorithm for packaging and testing workshops based on multi-scale information fusion as described in claim 5, characterized in that, The specific process of Step 4 is as follows: After obtaining the predicted information of future states, a cost-effectiveness index for model switching is constructed based on the prediction results, and a composite scheduling rule two is designed, as follows: 1) Calculate the cost-effectiveness of mold changeover based on the mold changeover time, workpiece processing time, and predicted work-in-process inventory required for each product changeover. : in, For workpiece index, For machining center or process index, For machine indexing, For product category indexing, This represents the total number of product categories. The total number of workpieces. To predict the window length, For the first Workpieces in this type of product The This process is done on the machine. On the processing time, For machines Processing the first The first product is converted into the second processing. The mold change time required for this product In order to be in At that moment, the Within the buffer zone of the first processing center Forecast work-in-process inventory for this product Indicates if workpiece Belongs to the For this product, ;otherwise ; 2) Based on the urgency of the workpiece Determine the scheduling priority using the following formula: ；（5） in, For workpiece index, Here, n is the index for the machining center or process, and n is the total number of workpieces. This represents the total number of processing stages. For process Average processing time across all available machines For workpiece Delivery time, For workpiece The expected start time of processing is calculated as shown in equation (3); 3) Construct a composite scheduling rule system, including the following three cases: ① If there are overdue items, select the item with the highest urgency. ②If there are no pending workpieces, select the workpiece that matches the current mold; ③ If neither of the first two conditions is met, then select the mold change cost-effectiveness index. Process the smallest workpiece; 4) The above scheduling rules serve as one of the action space selections for ITD3 reinforcement learning.

7. The intelligent scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion as described in claim 6, characterized in that, The specific process of Step 4 is as follows: After obtaining the prediction information of future states, a cost-effectiveness index for model switching is constructed based on the prediction results, and a composite scheduling rule three is designed, as follows: 1) Calculate the cost-effectiveness of mold changeover based on the mold changeover time, workpiece processing time, and predicted work-in-process inventory required for each product changeover. : in, For workpiece index, For machining center or process index, For machine indexing, For product category indexing, This represents the total number of product categories. The total number of workpieces. To predict the window length, For the first Workpieces in this type of product The This process is done on the machine. On the processing time, For machines Processing the first The first product is converted into the second processing. The mold change time required for this product In order to be in At that moment, the Within the buffer zone of the first processing center Forecast work-in-process inventory for this product Indicates if workpiece Belongs to the For this product, ;otherwise ; 2) Based on the urgency of the workpiece The scheduling priority is determined by the formula shown in formula (5); 3) Construct a composite scheduling rule system, including the following three cases: ① If there are seriously delayed workpieces: Then select the workpiece with the highest urgency. ②If there are no pending workpieces, select the workpiece that matches the current mold; ③ If neither of the first two conditions is met, then select the mold change cost-effectiveness index. Process the smallest workpiece; 4) The above scheduling rules serve as one of the action space selections for ITD3 reinforcement learning.

8. The intelligent scheduling algorithm for packaging and testing workshops based on multi-scale information fusion as described in claim 1, characterized in that, The specific process of Step 5 is as follows: The specific operation process based on the improved TD3 reinforcement learning algorithm is as follows: 1) First, obtain the state features. Input the state features into online In the network; 2) with Greedy strategy selects the optimal action ; 3) Interact with the environment to receive instant rewards and the next state and form an experience group 4) Write experience into the playback pool Previously, the algorithm introduced a similarity check mechanism, which means that if the similarity between the current experience and the historical experience exceeds a given threshold, the current experience is determined to be a redundant sample and will not be stored.

9. The intelligent scheduling algorithm for packaging and testing workshops driven by multi-scale information fusion according to claim 1, characterized in that, The specific process of Step 5 is as follows: ITD3 employs a dimensionless tiered reward mechanism, with the specific calculation method as follows: 1) First, compare and Actual delay rate at any time ,like The actual delay rate at any given time is less than The actual overdue rate at any given time is calculated as a positive reward, if The actual delay rate at any given time is greater than Actual overdue rate at any time The calculation multiplier is a negative reward; the specific calculation formula is as follows: ; 2) If neither of the above two conditions is met, then the assessment of the delay rate will be applied sequentially using a similar method. Calculate the reward, that is Machine idle rate Calculate the reward, that is and machine mold change rate Calculate the reward, that is ; 3) If none of the above conditions are met, then a reward will be given. .

Images