Chip manufacturing process parameter optimization method and system based on federated learning

Abstract

The present invention provides a chip manufacturing process parameter optimization method and system based on federated learning, which relates to the field of chip technology, including collecting multi-factory process parameters and yield data, using wavelet transform and dual evaluation to screen key parameters, and building a deep Q network model; performing group division based on process trajectory similarity, using differential privacy and homomorphic encryption to protect data security, and realizing group optimization through attention mechanism and knowledge distillation; setting a three-level time window to monitor yield and dynamically adjusting the optimization strategy. The present invention can effectively improve chip manufacturing yield, protect data privacy, and realize intelligent optimization of process parameters.

Description

Technical Field

[0001] The present invention relates to chip technology, and in particular to a chip manufacturing process parameter optimization method and system based on federated learning. Background Art

[0002] With the continuous development of integrated circuit manufacturing technology, the optimization of process parameters in the chip manufacturing process plays a key role in improving yield. Traditional chip manufacturing process parameter optimization mainly relies on engineers' experience and historical data from a single factory for adjustment. In recent years, with the development of artificial intelligence technology, machine learning methods have been introduced into the chip manufacturing field to achieve automatic optimization of process parameters in a data-driven way. At the same time, federated learning, as a distributed machine learning framework that protects data privacy, provides a new technical path for collaborative optimization among multiple factories. At present, many research institutions and companies at home and abroad are exploring the application of federated learning in chip manufacturing process optimization, in order to improve the overall manufacturing level through collaborative learning of multi-factory data.

[0003] Existing chip manufacturing process parameter optimization methods have the problem of insufficient data privacy protection in practical applications. Since chip manufacturing process data involves core corporate secrets, it is difficult for various manufacturing plants to directly share original data, which limits the collaborative optimization effect among multiple plants.

[0004] The current optimization methods lack detailed analysis and classification of the process characteristics of different factories. Due to the differences in equipment conditions and process levels of various chip manufacturing factories, simply optimizing the data of all factories uniformly often fails to achieve ideal results, and a more targeted optimization strategy needs to be established.

[0005] The existing methods lack an effective adaptive adjustment mechanism in the continuous optimization process. When the process parameter optimization effect is not good, it is difficult for the existing methods to find the problem and make adjustments in time, which easily leads to the optimization falling into the local optimum and failing to achieve continuous yield improvement. This reflects that the existing technology has obvious deficiencies in dynamic optimization capabilities. Summary of the invention

[0006] The embodiments of the present invention provide a chip manufacturing process parameter optimization method and system based on federated learning, which can solve the problems in the prior art.

[0007] According to a first aspect of the embodiments of the present invention,

[0008] Provides a chip manufacturing process parameter optimization method based on federated learning, including:

[0009] Collect process parameter data and yield data from multiple chip manufacturing plants, perform multi-scale decomposition of the process parameter data through wavelet transform to obtain parameter fluctuation characteristics; use mutual information entropy and Pearson correlation coefficient to perform dual evaluation on the parameter fluctuation characteristics, and screen out a subset of key process parameters; map the subset of key process parameters to a low-dimensional feature vector through a variational autoencoder; construct a reinforcement learning state space based on the low-dimensional feature vector, set the adjustment strategy of the process parameters as the action space, and establish a deep Q network model with the yield improvement level as the reward function; use a dynamic time warping algorithm to calculate the similarity of the process trajectories of different chip manufacturing plants based on the low-dimensional feature vector, and generate a process trajectory similarity matrix;

[0010] Based on the process trajectory similarity matrix, a hybrid strategy of spectral clustering and density peak clustering is adopted to divide the multiple chip manufacturing plants into process groups, and an independent group management node is set for each process group; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model to obtain a local training gradient; differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node using homomorphic encryption; after receiving the encrypted local training gradient, the group management node uses the attention mechanism to calculate the historical yield contribution of each chip manufacturing plant and uses it as the gradient aggregation weight; weighted gradient aggregation is performed based on the gradient aggregation weight, and group optimization features are extracted through knowledge distillation, and the deep Q network model is updated to obtain a group optimization strategy model;

[0011] Each chip manufacturing factory loads the family optimization strategy model, and adopts the ε-greedy strategy to select the process parameter adjustment plan based on the real-time status of the low-dimensional feature vector; constructs three-level sliding time windows of short-term, medium-term and long-term to monitor the yield change trend after adjustment respectively; when the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with the historical optimization data, the network parameters of the deep Q network model are optimized by the meta-learning method; the latest low-dimensional feature vector is input into the dynamic time warping algorithm to recalculate the process trajectory similarity matrix, triggering the dynamic reconstruction of the process family, and realizing the adaptive adjustment of the optimization strategy, wherein the corresponding preset yield threshold is dynamically updated according to the historical yield fluctuation range of each process family.

[0012] The dynamic time warping algorithm is used to calculate the process trajectory similarity of different chip manufacturing plants based on the low-dimensional feature vector, and the process trajectory similarity matrix is ​​generated, including:

[0013] Based on the low-dimensional feature vector, sampling is performed in a sliding time window to obtain time series feature points, the time series feature points are subjected to z-score normalization processing to obtain a standardized time series sequence, the variance of the standardized time series sequence in the local time window is calculated, and the elastic window size of the dynamic time warping algorithm is determined according to the variance, and the elastic window size is dynamically adjusted with the variance;

[0014] Constructing a cumulative distance matrix of the dynamic time warping algorithm, calculating the Euclidean distance between different standardized time series based on the elastic window size, taking the Euclidean distance as the initial element of the cumulative distance matrix, and using the dynamic time warping algorithm to calculate the minimum cumulative distance of adjacent elements in the cumulative distance matrix, the path corresponding to the minimum cumulative distance is the optimal matching path, and calculating the dynamic time warping distance between two standardized time series based on the optimal matching path;

[0015] The dynamic time warping distance is input into the Gaussian kernel function to calculate the process trajectory similarity between any two chip manufacturing plants. The kernel width parameter of the Gaussian kernel function is adaptively adjusted according to the distribution of the dynamic time warping distance. The calculated process trajectory similarity is filled into the corresponding position of the similarity matrix to generate a process trajectory similarity matrix reflecting the process trajectory similarity between different chip manufacturing plants.

[0016] Based on the process trajectory similarity matrix, a hybrid strategy of spectral clustering and density peak clustering is adopted to divide the multiple chip manufacturing plants into process groups, and each process group is provided with an independent group management node; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model, and obtains a local training gradient including:

[0017] Calculate the degree matrix of the process trajectory similarity matrix, wherein the diagonal elements of the degree matrix are the sum of the corresponding row elements of the process trajectory similarity matrix, construct a normalized Laplace matrix based on the degree matrix, extract the eigenvector corresponding to the minimum non-zero eigenvalue of the normalized Laplace matrix, and normalize the eigenvector to obtain a low-dimensional feature representation of each chip manufacturing plant;

[0018] Calculating density indicators of each chip manufacturing plant based on the low-dimensional feature representation, wherein the density indicators include a local data density value and a relative distance value, and the local data density value calculates the data distribution density around each chip manufacturing plant through an adaptive kernel function;

[0019] The product of the local data density value and the relative distance value is used as a decision value, a cluster center factory is selected based on the decision value, and the remaining chip manufacturing factories are allocated to the nearest cluster center factory according to the density reachability principle to form a process cluster;

[0020] Calculating management indicators of chip manufacturing plants in each process group, the management indicators include connectivity, resource status and network performance, and selecting a group management node for each process group based on the management indicators, the group management node is used to coordinate parameter optimization of the group;

[0021] Construct a deep Q network model including a strategy network and a target network. The strategy network inputs process parameters and outputs adjustment plans. The optimization value of the adjustment plans is evaluated based on the target network. Each chip manufacturing plant uses the deep Q network model to perform local parameter optimization training.

[0022] The network gradient of the deep Q network model during the parameter optimization training process is calculated, and the network gradient is normalized to obtain a local training gradient.

[0023] Construct a deep Q network model including a strategy network and a target network. The strategy network inputs process parameters and outputs an adjustment plan. The optimization value of the adjustment plan is evaluated based on the target network. Each chip manufacturing plant uses the deep Q network model to perform local parameter optimization training, including:

[0024] Constructing a deep Q network model including a strategy network and a target network, wherein the strategy network receives process parameter input and outputs a parameter adjustment scheme through a multi-layer neural network structure, and the target network is used to evaluate the optimization value of the parameter adjustment scheme;

[0025] The process parameters are input into the strategy network, the strategy network generates a parameter adjustment plan based on the current process parameter state, the target network receives the parameter adjustment plan, and evaluates the optimization value of the parameter adjustment plan relative to the current process parameter state;

[0026] Based on the optimization value of the target network evaluation, an optimal parameter adjustment scheme is selected, the optimal parameter adjustment scheme is executed to obtain the adjusted process parameters, and the corresponding yield change value is obtained; the current process parameter state, the optimal parameter adjustment scheme, the yield change value, and the adjusted process parameters are used to construct a training sample, and the deep Q network model is locally optimized and trained using the training sample.

[0027] The density index of each chip manufacturing plant is calculated based on the low-dimensional feature representation, and the density index includes a local data density value and a relative distance value including:

[0028] Calculating the local data density value of each chip manufacturing plant based on the low-dimensional feature representation, wherein the local data density value is calculated by an adaptive kernel function, and a bandwidth parameter of the adaptive kernel function is dynamically adjusted according to local data distribution characteristics;

[0029] The local data density value calculation formula is as follows:

[0030] ;

[0031] Among them, ρ i is the local data density value of factory i, n is the total number of factories, K is the adaptive kernel function, x i is the low-dimensional feature vector of factory i, x j is the low-dimensional feature vector of factory j, is the Euclidean distance, h(x i ) is the adaptive bandwidth function;

[0032] Calculating a bandwidth parameter of the adaptive kernel function according to the local data distribution characteristics, wherein the bandwidth parameter is determined by the product of a reference bandwidth and a local coefficient of variation, wherein the local coefficient of variation represents a degree of dispersion of data around each chip manufacturing plant;

[0033] The adaptive kernel function is used to calculate the influence weight between each chip manufacturing plant and an adjacent plant, and the local data density value of each chip manufacturing plant is determined based on the influence weight;

[0034] Calculating the relative distance value of each chip manufacturing factory according to the local data density value;

[0035] The relative distance value calculation formula is as follows:

[0036] ;

[0037] Among them, δ i is the relative distance value of factory i, is the minimum Euclidean distance between all factories whose local data density values ​​are greater than factory i and factory i, is the relative distance value of the point with the maximum density, d ij is the Euclidean distance from factory i to factory j, ρ j is the local data density value of factory j;

[0038] When the local data density value of a chip manufacturing plant is the largest, the maximum distance from the chip manufacturing plant to other plants is taken as the relative distance value.

[0039] The local training gradient is injected with differential privacy noise through Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node by homomorphic encryption; after receiving the encrypted local training gradient, the group management node uses the attention mechanism to calculate the historical yield contribution of each chip manufacturing plant, and uses it as the gradient aggregation weight including:

[0040] Performing density estimation on the local training gradient to construct a Gaussian mixture model, using an expectation maximization algorithm to calculate the mean vector and covariance matrix of the Gaussian mixture model, determining a mixing weight based on the mean vector and the covariance matrix, and obtaining a distribution characteristic of the local training gradient;

[0041] Calculate the sensitivity of the local training gradient to a single training sample, determine the noise intensity based on a preset privacy budget and the sensitivity, sample a noise vector of corresponding intensity from the Gaussian mixture model, and superimpose the noise vector on the local training gradient to obtain a noisy gradient;

[0042] Generate a homomorphic encryption key pair, the homomorphic encryption key pair includes a public key for encryption and a private key for decryption, use the public key to encrypt the noisy gradient to obtain an encrypted gradient, and transmit the encrypted gradient to the group management node;

[0043] Building a temporal attention module based on the group management node, the temporal attention module receives historical yield data of a chip manufacturing plant, uses position coding to maintain the temporal relationship of the historical yield data, and calculates the correlation strength of the yield state at each moment through an attention mechanism;

[0044] The current yield status is used as the query vector and the historical yield sequence is used as the key vector to calculate the attention score. The contribution of each chip manufacturing plant is evaluated based on the attention score and the degree of yield improvement. The contribution is normalized to obtain the gradient aggregation weight.

[0045] When the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with historical optimization data, the network parameters of the deep Q network model are optimized by a meta-learning method, including:

[0046] Construct a multi-level time window to monitor the yield index, the multi-level time window includes a four-hour real-time window, a twenty-four-hour daily window and a seven-day weekly window, each time window corresponds to a preset yield threshold, and continuously monitors the change trend of the yield index in each time window; when the yield index in any time window is continuously lower than the corresponding preset yield threshold, collect the current process parameters, equipment status and yield data, construct the process parameters, the equipment status and the yield data as a state vector and input it into the deep Q network model;

[0047] The deep Q network model re-evaluates the optional action space under the current process state based on the state vector, and the optional action space includes multiple groups of process parameter adjustment schemes, and each group of process parameter adjustment schemes corresponds to different parameter combinations; extracts empirical data including state information, action information and optimization effects from historical optimization data to construct an optimization task library, and retrieves similar scenarios in the optimization task library based on the state vector to obtain optimization strategy features under the similar scenarios;

[0048] A meta-learning method is used to migrate the optimization strategy features to the current scenario, the network parameters of the deep Q network model are adjusted to adapt the deep Q network model to the current optimization needs, and the optional action space is evaluated based on the adjusted network parameters; the action value of each process parameter adjustment scheme in the optional action space is calculated, the process parameter adjustment scheme with the highest action value is selected as the optimization scheme, the optimization scheme is executed and the optimization effect is recorded.

[0049] According to a second aspect of the embodiments of the present invention,

[0050] Provide a chip manufacturing process parameter optimization system based on federated learning, including:

[0051] The first unit is used to collect process parameter data and yield data from multiple chip manufacturing plants, perform multi-scale decomposition of the process parameter data through wavelet transform to obtain parameter fluctuation characteristics; use mutual information entropy and Pearson correlation coefficient to perform dual evaluation on the parameter fluctuation characteristics, and screen out a subset of key process parameters; map the subset of key process parameters to a low-dimensional feature vector through a variational autoencoder; construct a reinforcement learning state space based on the low-dimensional feature vector, set the adjustment strategy of the process parameters as the action space, and establish a deep Q network model with the yield improvement level as the reward function; use a dynamic time warping algorithm to calculate the similarity of the process trajectories of different chip manufacturing plants based on the low-dimensional feature vector, and generate a process trajectory similarity matrix;

[0052] The second unit is used to divide the multiple chip manufacturing plants into process groups based on the process trajectory similarity matrix by adopting a hybrid strategy of spectral clustering and density peak clustering, and each process group is provided with an independent group management node; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model to obtain a local training gradient; differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node by homomorphic encryption; after receiving the encrypted local training gradient, the group management node calculates the historical yield contribution of each chip manufacturing plant by using the attention mechanism, and uses it as the gradient aggregation weight; weighted gradient aggregation is performed based on the gradient aggregation weight, and group optimization features are extracted by knowledge distillation, and the deep Q network model is updated to obtain a group optimization strategy model;

[0053] The third unit is used for each chip manufacturing factory to load the group optimization strategy model, and adopt the ε-greedy strategy to select the process parameter adjustment plan based on the real-time status of the low-dimensional feature vector; construct short-term, medium-term and long-term three-level sliding time windows to monitor the yield change trend after adjustment respectively; when the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with the historical optimization data, the network parameters of the deep Q network model are optimized by the meta-learning method; the latest low-dimensional feature vector is input into the dynamic time warping algorithm to recalculate the process trajectory similarity matrix, trigger the dynamic reconstruction of the process group, and realize the adaptive adjustment of the optimization strategy, wherein the corresponding preset yield threshold is dynamically updated according to the historical yield fluctuation range of each process group.

[0054] According to a third aspect of the embodiments of the present invention,

[0055] An electronic device is provided, comprising:

[0056] processor;

[0057] a memory for storing processor-executable instructions;

[0058] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0059] According to a fourth aspect of the embodiments of the present invention,

[0060] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0061] The beneficial effects of this application are as follows:

[0062] 1. The present invention uses wavelet transform to perform multi-scale decomposition and combines the dual evaluation mechanism of mutual information entropy and Pearson correlation coefficient to effectively identify key process parameters that affect yield, and achieves parameter dimension reduction through variational autoencoder, significantly reducing the computational complexity of the optimization process. At the same time, the deep Q network modeling parameter optimization strategy is adopted, and the dynamic time warping algorithm is combined to calculate the process trajectory similarity, providing a reliable basis for the division of process groups.

[0063] 2. This invention adopts a federated learning framework based on differential privacy and homomorphic encryption, which realizes the sharing of optimization experience while protecting the data privacy of each factory. By introducing the attention mechanism to calculate the yield contribution as the gradient aggregation weight, and using knowledge distillation to extract the population optimization features, the training efficiency and model performance of federated learning are improved, so that each factory can make full use of the collective wisdom to improve the local optimization strategy.

[0064] 3. The present invention constructs a three-level sliding time window monitoring system, cooperates with the meta-learning method to dynamically optimize the parameters of the deep Q network, and realizes the real-time adjustment of the optimization strategy through the adaptive reconstruction mechanism of the process group. This multi-level adaptive optimization mechanism significantly improves the robustness and generalization ability of the method, can effectively cope with the challenges brought by process parameter fluctuations, and continuously improve the manufacturing yield level. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of a process flow of a chip manufacturing process parameter optimization method based on federated learning according to an embodiment of the present invention;

[0066] Figure 2 Schematic diagram of the structure of a chip manufacturing process parameter optimization system based on federated learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0069] Figure 1 FIG. 1 is a flow chart of a method for optimizing chip manufacturing process parameters based on federated learning according to an embodiment of the present invention. Figure 1As shown, the method includes:

[0070] S11. Collect process parameter data and yield data from multiple chip manufacturing plants, perform multi-scale decomposition of the process parameter data through wavelet transform to obtain parameter fluctuation characteristics; use mutual information entropy and Pearson correlation coefficient to perform dual evaluation on the parameter fluctuation characteristics, and screen out a subset of key process parameters; map the subset of key process parameters to a low-dimensional feature vector through a variational autoencoder; construct a reinforcement learning state space based on the low-dimensional feature vector, set the adjustment strategy of the process parameters as the action space, and establish a deep Q network model with the yield improvement level as the reward function; use a dynamic time warping algorithm to calculate the similarity of the process trajectories of different chip manufacturing plants based on the low-dimensional feature vector, and generate a process trajectory similarity matrix;

[0071] S12. Based on the process trajectory similarity matrix, a hybrid strategy of spectral clustering and density peak clustering is adopted to divide the multiple chip manufacturing plants into process groups, and an independent group management node is set for each process group; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model to obtain a local training gradient; differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node by homomorphic encryption; after receiving the encrypted local training gradient, the group management node uses the attention mechanism to calculate the historical yield contribution of each chip manufacturing plant and uses it as the gradient aggregation weight; weighted gradient aggregation is performed based on the gradient aggregation weight, and group optimization features are extracted through knowledge distillation, and the deep Q network model is updated to obtain a group optimization strategy model;

[0072] S13. Each chip manufacturing factory loads the cluster optimization strategy model, and adopts the ε-greedy strategy to select the process parameter adjustment plan based on the real-time status of the low-dimensional feature vector; constructs three-level sliding time windows of short-term, medium-term and long-term to monitor the yield change trend after adjustment respectively; when the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with the historical optimization data, the network parameters of the deep Q network model are optimized by the meta-learning method; the latest low-dimensional feature vector is input into the dynamic time warping algorithm to recalculate the process trajectory similarity matrix, triggering the dynamic reconstruction of the process cluster, and realizing the adaptive adjustment of the optimization strategy, wherein the corresponding preset yield threshold is dynamically updated according to the historical yield fluctuation range of each process cluster.

[0073] In an optional implementation, using a dynamic time warping algorithm to calculate the process trajectory similarity of different chip manufacturing plants based on the low-dimensional feature vector, generating a process trajectory similarity matrix includes:

[0074] Based on the low-dimensional feature vector, sampling is performed in a sliding time window to obtain time series feature points, the time series feature points are subjected to z-score normalization processing to obtain a standardized time series sequence, the variance of the standardized time series sequence in the local time window is calculated, and the elastic window size of the dynamic time warping algorithm is determined according to the variance, and the elastic window size is dynamically adjusted with the variance;

[0075] Constructing a cumulative distance matrix of the dynamic time warping algorithm, calculating the Euclidean distance between different standardized time series based on the elastic window size, taking the Euclidean distance as the initial element of the cumulative distance matrix, and using the dynamic time warping algorithm to calculate the minimum cumulative distance of adjacent elements in the cumulative distance matrix, the path corresponding to the minimum cumulative distance is the optimal matching path, and calculating the dynamic time warping distance between two standardized time series based on the optimal matching path;

[0076] The dynamic time warping distance is input into the Gaussian kernel function to calculate the process trajectory similarity between any two chip manufacturing plants. The kernel width parameter of the Gaussian kernel function is adaptively adjusted according to the distribution of the dynamic time warping distance. The calculated process trajectory similarity is filled into the corresponding position of the similarity matrix to generate a process trajectory similarity matrix reflecting the process trajectory similarity between different chip manufacturing plants.

[0077] First, the low-dimensional feature vectors of different chip manufacturing plants are processed by time series sampling. By setting a sliding time window, the feature vectors are uniformly sampled. The sampling interval can be set to 1 hour, and the length of each time window is 24 hours. The corresponding feature value is obtained at each sampling time point to form a sequence of time series feature points. For example, the feature values ​​collected by a chip manufacturing plant within 24 consecutive hours may be [0.35, 0.42, 0.38, 0.45...] and other values.

[0078] Then, the sampled time series feature points are preprocessed by standardization. The z-score standardization method is used to convert the original feature values ​​into a standard normal distribution with a mean of 0 and a standard deviation of 1. The standardized time series is more suitable for subsequent similarity calculations. For example, the original feature value of 0.35 may become -0.8 after standardization.

[0079] Then calculate the variance of the standardized time series in the local time window. Set the local time window size to 6 hours, and calculate the degree of dispersion of the sequence values ​​in this window. When the variance is large, it means that the process fluctuations are large during this time period, and a larger elastic window is required; when the variance is small, the process is more stable, and a smaller elastic window can be used. For example, if the variance in a 6-hour window is 0.25, the elastic window size can be set to 4 accordingly; if the variance is 0.5, set the elastic window size to 8.

[0080] Based on the above elastic window size, the cumulative distance matrix of the dynamic time warping algorithm is constructed. First, the Euclidean distance of the corresponding time points of any two standardized time series is calculated as the initial element of the matrix. For example, if the value of sequence A at a certain moment is 0.5 and the value of sequence B at the corresponding moment is 0.8, then the distance at this position is 0.3. Then, under the elastic window constraint, the minimum cumulative distance of adjacent elements in the matrix is ​​calculated to obtain the optimal matching path. This path represents the best alignment of the two sequences.

[0081] Finally, the Gaussian kernel function is used to convert the dynamic time warping distance into a similarity value. The width parameter of the kernel function is adaptively adjusted according to the distribution characteristics of the distances between all factories. For example, when the overall distance distribution is more dispersed, the kernel width is increased; when the distance distribution is concentrated, the kernel width is reduced. The final similarity value is between 0 and 1, and the larger the value, the more similar the process trajectories are. The similarity values ​​between each factory are filled into the corresponding positions of the similarity matrix to generate a complete similarity matrix.

[0082] The solution of this application can:

[0083] By dynamically adjusting the elastic window size, it can adaptively handle different situations such as severe and stable process fluctuations, improving the accuracy and robustness of trajectory matching. The use of cumulative distance matrix and optimal matching path effectively solves the problem of misalignment of process trajectories in different factories on a time scale, and achieves a more accurate similarity measurement. The adaptive conversion mechanism based on the Gaussian kernel function makes the similarity calculation results more consistent with the actual process characteristics, providing a reliable quantitative basis for subsequent process optimization and knowledge transfer.

[0084] In an optional implementation, based on the process trajectory similarity matrix, a hybrid strategy of spectral clustering and density peak clustering is adopted to divide the multiple chip manufacturing plants into process groups, and each process group is provided with an independent group management node; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model, and obtains a local training gradient including:

[0085] Calculate the degree matrix of the process trajectory similarity matrix, wherein the diagonal elements of the degree matrix are the sum of the corresponding row elements of the process trajectory similarity matrix, construct a normalized Laplace matrix based on the degree matrix, extract the eigenvector corresponding to the minimum non-zero eigenvalue of the normalized Laplace matrix, and normalize the eigenvector to obtain a low-dimensional feature representation of each chip manufacturing plant;

[0086] Calculating density indicators of each chip manufacturing plant based on the low-dimensional feature representation, wherein the density indicators include a local data density value and a relative distance value, and the local data density value calculates the data distribution density around each chip manufacturing plant through an adaptive kernel function;

[0087] The product of the local data density value and the relative distance value is used as a decision value, a cluster center factory is selected based on the decision value, and the remaining chip manufacturing factories are allocated to the nearest cluster center factory according to the density reachability principle to form a process cluster;

[0088] Calculating management indicators of chip manufacturing plants in each process group, the management indicators include connectivity, resource status and network performance, and selecting a group management node for each process group based on the management indicators, the group management node is used to coordinate parameter optimization of the group;

[0089] Construct a deep Q network model including a strategy network and a target network. The strategy network inputs process parameters and outputs adjustment plans. The optimization value of the adjustment plans is evaluated based on the target network. Each chip manufacturing plant uses the deep Q network model to perform local parameter optimization training.

[0090] The network gradient of the deep Q network model during the parameter optimization training process is calculated, and the network gradient is normalized to obtain a local training gradient.

[0091] The present invention proposes a chip manufacturing plant clustering management method based on process trajectory similarity. The method divides multiple chip manufacturing plants into process groups through a hybrid clustering strategy, and sets an independent management node for each group, thereby realizing collaborative training of parameter optimization. The specific implementation steps are as follows:

[0092] First, construct a process trajectory similarity matrix. By analyzing the historical production data of each chip manufacturing plant, extracting process trajectory features, and calculating the similarity between factories, a similarity matrix is ​​formed. For example, for 10 chip manufacturing plants, a 10x10 similarity matrix can be obtained, and each element in the matrix represents the degree of similarity of the process trajectories between two factories.

[0093] Next, based on the process trajectory similarity matrix, a hybrid strategy of spectral clustering and density peak clustering is used to cluster factories. First, the degree matrix of the similarity matrix is ​​calculated, and the diagonal elements of the degree matrix are the sum of the corresponding row elements of the similarity matrix. For example, for the first factory, the diagonal elements of its degree matrix are the sum of the similarities with the other 9 factories. Then, a normalized Laplace matrix is ​​constructed based on the degree matrix, and the eigenvector corresponding to the minimum non-zero eigenvalue of the matrix is ​​extracted. The eigenvector is normalized to obtain a low-dimensional feature representation of each chip manufacturing factory.

[0094] Based on the low-dimensional feature representation, the density index of each chip manufacturing plant is calculated. The density index includes the local data density value and the relative distance value. The local data density value calculates the density of data distribution around each chip manufacturing plant through an adaptive kernel function. For example, a distance threshold can be set, and the number of neighboring plants within the threshold range can be counted as the local data density value. The relative distance value indicates the distance from each plant to the nearest plant with higher density.

[0095] The product of the local data density value and the relative distance value is used as the decision value, and the cluster center factory is selected based on the decision value. Factories with higher decision values ​​are selected as cluster centers because they have both higher local density and are farther away from other high-density points. For example, the three factories with the highest decision values ​​can be selected as the initial cluster centers. Then, the remaining chip manufacturing factories are allocated to the nearest cluster center factories according to the density reachability principle to form process clusters.

[0096] For each formed process cluster, the management indicators of the chip manufacturing factories in the cluster are calculated. The management indicators include connectivity, resource status and network performance. Connectivity can be measured by calculating the average similarity between the factory and other factories in the cluster; resource status can consider the computing power, storage capacity, etc. of the factory; network performance can be evaluated by testing the communication delay and bandwidth between factories. Based on these management indicators, a most suitable factory is selected for each process cluster as the cluster management node, which will be responsible for coordinating the parameter optimization process of this cluster.

[0097] After determining the process groups and management nodes, a deep Q network model consisting of a policy network and a target network is constructed. The policy network receives process parameters as input and outputs adjustment plans; the target network is used to evaluate the optimization value of the adjustment plan. For example, the policy network can be a multi-layer perceptron, where the input layer receives process parameters such as temperature, pressure, and time, and the output layer gives adjustment suggestions for these parameters; the target network has a similar structure, but is used to predict the expected return after adjustment.

[0098] Each chip manufacturing plant uses a deep Q network model for local parameter optimization training. During the training process, the plant selects an adjustment plan based on the current process status, performs the adjustment and observes the results, and then updates the network parameters. For example, if the product yield is improved after the adjustment, the probability of selecting the adjustment plan is strengthened.

[0099] Finally, the network gradient of the deep Q network model during the parameter optimization training process is calculated, and the network gradient is normalized to obtain the local training gradient. The normalization process can use the maximum and minimum normalization method to map the gradient value to the range of 0-1. These local training gradients will be used in the subsequent collaborative optimization process.

[0100] The solution of this application can:

[0101] This method realizes the intelligent clustering of chip manufacturing plants through a hybrid clustering strategy, fully considers the similarity of process trajectories, and enables plants with similar process characteristics to be divided into the same group, thus improving the pertinence and efficiency of parameter optimization. The deep Q network model is used for local parameter optimization, combined with the design of the strategy network and the target network, so that each plant can adaptively adjust the process parameters according to its own characteristics, improving the stability and reliability of the optimization effect. By setting up a group management node to coordinate the parameter optimization process, the organic combination of distributed learning and centralized management is realized, which not only ensures the autonomy of each plant, but also promotes knowledge sharing and improves the overall optimization efficiency and resource utilization.

[0102] In an optional implementation, a deep Q network model including a strategy network and a target network is constructed, the strategy network inputs process parameters and outputs an adjustment plan, and the optimization value of the adjustment plan is evaluated based on the target network. Each chip manufacturing plant uses the deep Q network model to perform local parameter optimization training, including:

[0103] Constructing a deep Q network model including a strategy network and a target network, wherein the strategy network receives process parameter input and outputs a parameter adjustment scheme through a multi-layer neural network structure, and the target network is used to evaluate the optimization value of the parameter adjustment scheme;

[0104] The process parameters are input into the strategy network, the strategy network generates a parameter adjustment plan based on the current process parameter state, the target network receives the parameter adjustment plan, and evaluates the optimization value of the parameter adjustment plan relative to the current process parameter state;

[0105] Based on the optimization value of the target network evaluation, an optimal parameter adjustment scheme is selected, the optimal parameter adjustment scheme is executed to obtain the adjusted process parameters, and the corresponding yield change value is obtained; the current process parameter state, the optimal parameter adjustment scheme, the yield change value, and the adjusted process parameters are used to construct a training sample, and the deep Q network model is locally optimized and trained using the training sample.

[0106] This embodiment provides a chip manufacturing process parameter optimization method based on a deep Q network model. The method realizes intelligent optimization of chip manufacturing process parameters by constructing a deep Q network model including a strategy network and a target network, thereby improving chip manufacturing yield.

[0107] First, a deep Q network model is constructed. The model consists of two main components: the strategy network and the target network. The strategy network is responsible for receiving process parameter inputs and generating parameter adjustment plans through a multi-layer neural network structure. The target network is used to evaluate the optimization value of the parameter adjustment plan.

[0108] The structure of the policy network includes an input layer, multiple hidden layers, and an output layer. The input layer receives the current process parameters, such as temperature, pressure, time, etc. The hidden layer uses a multi-layer fully connected neural network, and each layer uses the ReLU activation function for nonlinear transformation. The output layer generates a parameter adjustment plan, including the adjustment range of each process parameter.

[0109] The structure of the target network is similar to that of the policy network, but its input is the parameter adjustment plan generated by the policy network, and its output is the optimization value evaluation of the plan. The target network evaluates the long-term value of the adjustment plan by predicting the cumulative rewards that may be obtained in the future.

[0110] Next, the model is trained and optimized. First, the current process parameters are input into the policy network. For example, the temperature of a chip manufacturing process is 300°C, the pressure is 5 Pa, and the time is 60 minutes. Based on these inputs, the policy network generates a parameter adjustment plan, such as adjusting the temperature to 305°C, the pressure to 5.2 Pa, and the time to 58 minutes.

[0111] Then, the target network receives this adjustment plan and evaluates its optimization value. Assume that the optimization value evaluated by the target network is 0.85 (normalized to between 0 and 1). Based on this evaluation result, the optimal parameter adjustment plan is selected. In actual applications, multiple adjustment plans may be generated, and the plan with the highest evaluation value is selected.

[0112] Execute the selected optimal parameter adjustment scheme to obtain the adjusted process parameters. In this example, the new process parameters are: temperature 305°C, pressure 5.2 Pa, time 58 minutes. At the same time, record the yield change value after executing the scheme. Assume that the yield rate is increased from the original 95% to 96.5%, and the yield change value is 1.5%.

[0113] The current process parameter state (300°C, 5 Pa, 60 minutes), the optimal parameter adjustment solution (+5°C, +0.2 Pa, -2 minutes), the yield change value (1.5%), and the adjusted process parameters (305°C, 5.2 Pa, 58 minutes) are constructed into a training sample. This sample is used to perform local optimization training on the deep Q network model.

[0114] The training process uses experience replay technology to store multiple similar training samples in an experience pool. During each training, a batch of samples are randomly selected from the experience pool for batch training to break the correlation between samples and improve learning efficiency.

[0115] During training, the network parameters are updated using the temporal difference learning algorithm. Specifically, the difference between the actual reward (yield change value) and the value predicted by the target network is calculated, and this difference is used as the loss function to update the parameters of the policy network through the back-propagation algorithm.

[0116] In order to improve the training stability, the target network fixed technology is adopted. That is, the target network parameters are kept unchanged within a certain number of steps, and only the policy network parameters are updated. After a certain number of steps, the policy network parameters are copied to the target network to achieve soft update of the target network.

[0117] In addition, in order to balance exploration and utilization, the ε-greedy strategy is used when selecting parameter adjustment schemes. That is, an adjustment scheme is randomly selected with a probability of ε, and the scheme with the highest evaluation value of the target network is selected with a probability of 1-ε. As the training progresses, the ε value is gradually reduced to increase the utilization of the optimal scheme.

[0118] In actual applications, each chip manufacturing plant can use the above method to perform localized parameter optimization training based on its own historical data and specific process flow. By continuously accumulating training samples and optimizing models, the deep Q network can gradually learn better parameter adjustment strategies, thereby continuously improving chip manufacturing yield.

[0119] The solution of this application can:

[0120] This method can adaptively learn the optimal parameter adjustment strategy without manually setting complex rules, greatly reducing the need for manual intervention and improving the efficiency and accuracy of process parameter optimization. By combining the strategy network and the target network, this method not only considers the immediate yield improvement, but also evaluates the long-term optimization value, which helps to find a more stable and reliable parameter adjustment solution and avoid falling into the local optimum. This method supports localized training in each chip manufacturing plant, and can make full use of the plant's own historical data and specific process flow to achieve personalized parameter optimization and adapt to the actual needs of different plants, thereby comprehensively improving the yield and production efficiency of chip manufacturing.

[0121] In an optional implementation, the density index of each chip manufacturing plant is calculated based on the low-dimensional feature representation, and the density index includes a local data density value and a relative distance value including:

[0122] Calculating the local data density value of each chip manufacturing plant based on the low-dimensional feature representation, wherein the local data density value is calculated by an adaptive kernel function, and a bandwidth parameter of the adaptive kernel function is dynamically adjusted according to local data distribution characteristics;

[0123] The calculation formula of local data density value is as follows:

[0124] ;

[0125] Among them, ρ i is the local data density value of factory i, n is the total number of factories, K is the adaptive kernel function, x i is the low-dimensional feature vector of factory i, x j is the low-dimensional feature vector of factory j, is the Euclidean distance, h(x i ) is the adaptive bandwidth function;

[0126] Calculating a bandwidth parameter of the adaptive kernel function according to the local data distribution characteristics, wherein the bandwidth parameter is determined by the product of a reference bandwidth and a local coefficient of variation, wherein the local coefficient of variation represents a degree of dispersion of data around each chip manufacturing plant;

[0127] The adaptive kernel function is used to calculate the influence weight between each chip manufacturing plant and an adjacent plant, and the local data density value of each chip manufacturing plant is determined based on the influence weight;

[0128] Calculating the relative distance value of each chip manufacturing factory according to the local data density value;

[0129] The relative distance value calculation formula is as follows:

[0130] ;

[0131] Among them, δ i is the relative distance value of factory i, is the minimum Euclidean distance between all factories whose local data density values ​​are greater than factory i and factory i, is the relative distance value of the point with the maximum density, d ij is the Euclidean distance from factory i to factory j, ρ j is the local data density value of factory j;

[0132] When the local data density value of a chip manufacturing plant is the largest, the maximum distance from the chip manufacturing plant to other plants is taken as the relative distance value.

[0133] First, we obtain the raw data of the chip manufacturing plant, including multi-dimensional indicators such as the plant's production capacity, yield rate, equipment utilization rate, etc. Through the dimensionality reduction algorithm, these high-dimensional data are converted into low-dimensional feature representations for subsequent analysis and processing.

[0134] For each plant, its local data density value needs to be calculated. The local data density value reflects the degree of aggregation of the plant with surrounding plants in the feature space. The adaptive kernel function is used in the calculation, and the bandwidth parameter of the function is dynamically adjusted according to the local distribution characteristics of the data. Specifically, the bandwidth parameter is obtained by multiplying the baseline bandwidth by the local coefficient of variation. The baseline bandwidth can be determined by methods such as cross-validation, and the local coefficient of variation reflects the degree of discreteness of the data around the plant.

[0135] Take a chip manufacturing industrial park as an example. There are 10 factories in the park. First, the original indicator data of each factory is collected, including 15 indicators such as monthly production capacity, equipment comprehensive efficiency, and quality compliance rate. The 15-dimensional data is reduced to a 3-dimensional feature vector through the principal component analysis method. For each factory, the Euclidean distance with other factories is calculated, and the influence weight is calculated based on the adaptive kernel function. The local coefficient of variation of a factory is 1.2, which is multiplied by the baseline bandwidth of 0.5 to obtain the actual bandwidth parameter of 0.6.

[0136] After obtaining the local data density value, calculate the relative distance value of each factory. For each factory, find the value with the smallest Euclidean distance to the factory among all other factories with a density greater than that of the factory as its relative distance value. For the factory with the largest density, its relative distance value is taken as the maximum distance from the factory to all other factories.

[0137] In the example, the plant numbered 3 has the maximum local data density value of 0.85, and its relative distance value is the maximum distance from the plant to other plants, which is 2.3. For the plant numbered 5, its local data density value is 0.6. Among the plants with density values ​​greater than 0.6, the plant closest to plant 5 is plant 3, with a distance of 1.2, so the relative distance value of plant 5 is 1.2.

[0138] The solution of this application can:

[0139] This method dynamically adjusts the bandwidth parameters through an adaptive kernel function, which can more accurately characterize the local distribution characteristics of the data and improve the accuracy and reliability of density index calculation. The local data density value and relative distance value are used to comprehensively evaluate the factory characteristics, which not only considers the degree of clustering of factories in the feature space, but also considers the relative position relationship between factories, making the evaluation results more comprehensive and objective. The density index calculation method based on data-driven avoids the interference of subjective factors, can provide reliable data support for the operation management and decision optimization of chip manufacturing factories, and has good practical value.

[0140] In an optional implementation, differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node using homomorphic encryption; after receiving the encrypted local training gradient, the group management node uses an attention mechanism to calculate the historical yield contribution of each chip manufacturing plant, and uses it as a gradient aggregation weight including:

[0141] Performing density estimation on the local training gradient to construct a Gaussian mixture model, using an expectation maximization algorithm to calculate the mean vector and covariance matrix of the Gaussian mixture model, determining a mixing weight based on the mean vector and the covariance matrix, and obtaining a distribution characteristic of the local training gradient;

[0142] Calculate the sensitivity of the local training gradient to a single training sample, determine the noise intensity based on a preset privacy budget and the sensitivity, sample a noise vector of corresponding intensity from the Gaussian mixture model, and superimpose the noise vector on the local training gradient to obtain a noisy gradient;

[0143] Generate a homomorphic encryption key pair, the homomorphic encryption key pair includes a public key for encryption and a private key for decryption, use the public key to encrypt the noisy gradient to obtain an encrypted gradient, and transmit the encrypted gradient to the group management node;

[0144] Building a temporal attention module based on the group management node, the temporal attention module receives historical yield data of a chip manufacturing plant, uses position coding to maintain the temporal relationship of the historical yield data, and calculates the correlation strength of the yield state at each moment through an attention mechanism;

[0145] The current yield status is used as the query vector and the historical yield sequence is used as the key vector to calculate the attention score. The contribution of each chip manufacturing plant is evaluated based on the attention score and the degree of yield improvement. The contribution is normalized to obtain the gradient aggregation weight.

[0146] After local model training in the chip manufacturing plant, the density of the gradient data obtained from the local training is first estimated. By collecting a large number of gradient sample points, the parameters of the Gaussian mixture model are iteratively optimized using the expectation maximization algorithm. Specifically, the number of Gaussian mixture components is set to 3 to 5, and the gradient data is clustered to obtain the mean vector and covariance matrix of each Gaussian distribution. Based on the posterior probability of the sample point under each Gaussian distribution, the mixing weight is calculated to obtain the distribution characteristics of the gradient data.

[0147] To protect data privacy, it is necessary to calculate the sensitivity of the gradient to a single training sample. By adding or removing a sample in the training data set, the maximum change in the gradient is observed as a sensitivity indicator. The differential privacy budget is set to a decimal between 0.1 and 1, and the intensity of the Gaussian noise to be added is determined in combination with the sensitivity. A noise vector of corresponding intensity is sampled from the established Gaussian mixture model, and the noise vector is added to the original gradient to obtain a noisy gradient that meets the differential privacy requirements.

[0148] Next, generate a key pair for homomorphic encryption. Use the Paillier homomorphic encryption algorithm to generate a 2048-bit public key and private key. Use the public key to encrypt the noisy gradient to obtain the encrypted gradient in ciphertext form. Transmit the encrypted gradient to the group management node through a secure channel.

[0149] On the cluster management node side, a time series analysis module based on the attention mechanism is constructed. This module receives the monthly yield data of each chip manufacturing plant in the past 12 months as input. The time series data is positionally encoded and the time information is encoded into a 512-dimensional vector representation. By calculating the dot product similarity between the yield states at different times, the attention score matrix is ​​obtained, which reflects the degree of correlation between the yield states at each time.

[0150] The yield status of the current month is used as the query vector and the historical yield sequence is used as the key vector to calculate the attention score. Combined with the yield improvement of each factory in different periods, its contribution to the overall yield optimization is evaluated. The contribution is normalized to obtain a gradient aggregation weight between 0 and 1. The larger the weight, the more worthy of reference the factory's experience is.

[0151] The solution of this application can:

[0152] The gradient distribution is accurately modeled through the Gaussian mixture model, which can better characterize the multi-peak characteristics of the gradient compared to a single Gaussian distribution, and improve the accuracy and effectiveness of differential privacy protection. The homomorphic encryption technology is used to achieve end-to-end encrypted transmission of gradient data, ensuring the security of data during transmission and aggregation, effectively preventing the leakage of sensitive information, and protecting the commercial secrets of each factory. The time series analysis method based on the attention mechanism can adaptively capture the correlation between the yield states of each period, reasonably evaluate the contribution of each factory, and achieve a fairer and more efficient federated learning process.

[0153] In an optional implementation, when the yield index of any time window is continuously lower than the corresponding preset yield threshold, re-evaluating the optional action space based on the current state using the deep Q network model, and optimizing the network parameters of the deep Q network model by a meta-learning method in combination with historical optimization data includes:

[0154] Construct a multi-level time window to monitor the yield index, the multi-level time window includes a four-hour real-time window, a twenty-four-hour daily window and a seven-day weekly window, each time window corresponds to a preset yield threshold, and continuously monitors the change trend of the yield index in each time window; when the yield index in any time window is continuously lower than the corresponding preset yield threshold, collect the current process parameters, equipment status and yield data, construct the process parameters, the equipment status and the yield data as a state vector and input it into the deep Q network model;

[0155] The deep Q network model re-evaluates the optional action space under the current process state based on the state vector, and the optional action space includes multiple groups of process parameter adjustment schemes, and each group of process parameter adjustment schemes corresponds to different parameter combinations; extracts empirical data including state information, action information and optimization effects from historical optimization data to construct an optimization task library, and retrieves similar scenarios in the optimization task library based on the state vector to obtain optimization strategy features under the similar scenarios;

[0156] A meta-learning method is used to migrate the optimization strategy features to the current scenario, the network parameters of the deep Q network model are adjusted to adapt the deep Q network model to the current optimization needs, and the optional action space is evaluated based on the adjusted network parameters; the action value of each process parameter adjustment scheme in the optional action space is calculated, the process parameter adjustment scheme with the highest action value is selected as the optimization scheme, the optimization scheme is executed and the optimization effect is recorded.

[0157] The process parameter optimization system based on deep reinforcement learning first builds a multi-level time window monitoring system. The system includes three levels: four-hour real-time window, twenty-four-hour daily window, and seven-day weekly window. Set corresponding yield thresholds for different time windows, for example, the four-hour window yield threshold is 98%, the daily window yield threshold is 97.5%, and the weekly window yield threshold is 97%. The system continuously collects production data and calculates the yield indicators of each time window.

[0158] When the yield index of any time window is detected to be lower than the corresponding threshold for three consecutive times, the system automatically triggers the optimization process. First, the current state information is collected, including process parameters such as temperature, pressure, flow, equipment status information such as equipment operation time, maintenance records, and the yield data of the last 100 products. After standardizing these data, a state vector is constructed. For example, during a certain optimization, parameter data such as temperature 275 degrees, pressure 2.5MPa, and flow 120L / min are collected.

[0159] After receiving the state vector, the deep Q network model evaluates the optional parameter adjustment schemes based on the current process state. For example, the temperature can be adjusted within the range of plus or minus 10 degrees, the pressure can be adjusted within the range of plus or minus 0.2 MPa, and the flow rate can be adjusted within the range of plus or minus 10 L / min. Multiple sets of candidate schemes are generated through parameter combinations. At the same time, the system retrieves historical experience of similar scenarios from the optimization task library. The similarity calculation is based on the Euclidean distance of the process parameters, and the top five sets of historical data with the highest similarity are selected.

[0160] The system extracts the optimization strategy features from these five sets of historical data, including information such as parameter adjustment direction and adjustment range. Meta-learning methods are used to transfer these features to the current scenario by adjusting the network weights of the deep Q network. For example, if historical data shows that increasing temperature and reducing pressure are more effective in similar scenarios, the network weights are adjusted accordingly so that the model gives a higher score to such adjustment plans.

[0161] The optimized deep Q network evaluates the candidate solutions and calculates the action value of each solution. The solution with the highest action value is selected as the final optimization solution. For example, the temperature is increased by 5 degrees, the pressure is reduced by 0.1MPa, and the flow rate is kept unchanged. The system executes the optimization solution and continuously monitors the yield change in the next four hours, and records the optimization results in the task library for subsequent optimization.

[0162] The solution of this application can:

[0163] By building a multi-level time window monitoring system, we have achieved all-round monitoring of yield indicators, which can timely detect yield anomalies on different time scales, improve the timeliness of problem discovery and processing, and significantly reduce quality risks. By combining the deep Q network with meta-learning, we can make full use of historical optimization experience to guide current decisions, thereby improving the accuracy and efficiency of parameter optimization. By extracting optimization strategy features from similar scenarios and migrating them to the current scenario, the model's adaptation speed to new scenarios is accelerated, reducing the trial and error cost during the optimization process. A complete optimization closed loop has been established, including state monitoring, solution generation, strategy optimization, and effect evaluation, ensuring the controllability and traceability of the optimization process. By continuously accumulating optimization experience and dynamically updating optimization strategies, the system's optimization capabilities have been continuously improved, achieving continuous improvement in process parameter optimization.

[0164] Figure 2 FIG. 1 is a schematic diagram of the structure of a chip manufacturing process parameter optimization system based on federated learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0165] The first unit is used to collect process parameter data and yield data from multiple chip manufacturing plants, perform multi-scale decomposition of the process parameter data through wavelet transform to obtain parameter fluctuation characteristics; use mutual information entropy and Pearson correlation coefficient to perform dual evaluation on the parameter fluctuation characteristics, and screen out a subset of key process parameters; map the subset of key process parameters to a low-dimensional feature vector through a variational autoencoder; construct a reinforcement learning state space based on the low-dimensional feature vector, set the adjustment strategy of the process parameters as the action space, and establish a deep Q network model with the yield improvement level as the reward function; use a dynamic time warping algorithm to calculate the similarity of the process trajectories of different chip manufacturing plants based on the low-dimensional feature vector, and generate a process trajectory similarity matrix;

[0166] The second unit is used to divide the multiple chip manufacturing plants into process groups based on the process trajectory similarity matrix by adopting a hybrid strategy of spectral clustering and density peak clustering, and each process group is provided with an independent group management node; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model to obtain a local training gradient; differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node by homomorphic encryption; after receiving the encrypted local training gradient, the group management node calculates the historical yield contribution of each chip manufacturing plant by using the attention mechanism, and uses it as the gradient aggregation weight; weighted gradient aggregation is performed based on the gradient aggregation weight, and group optimization features are extracted by knowledge distillation, and the deep Q network model is updated to obtain a group optimization strategy model;

[0167] The third unit is used for each chip manufacturing factory to load the group optimization strategy model, and adopt the ε-greedy strategy to select the process parameter adjustment plan based on the real-time status of the low-dimensional feature vector; construct short-term, medium-term and long-term three-level sliding time windows to monitor the yield change trend after adjustment respectively; when the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with the historical optimization data, the network parameters of the deep Q network model are optimized by the meta-learning method; the latest low-dimensional feature vector is input into the dynamic time warping algorithm to recalculate the process trajectory similarity matrix, trigger the dynamic reconstruction of the process group, and realize the adaptive adjustment of the optimization strategy, wherein the corresponding preset yield threshold is dynamically updated according to the historical yield fluctuation range of each process group.

[0168] According to a third aspect of the embodiments of the present invention,

[0169] An electronic device is provided, comprising:

[0170] processor;

[0171] a memory for storing processor-executable instructions;

[0172] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0173] A fourth aspect of the embodiments of the present invention is:

[0174] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0175] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A chip manufacturing process parameter optimization method based on federated learning, characterized in that: include: Collecting process parameter data and yield data from multiple chip manufacturing plants, and performing multi-scale decomposition of the process parameter data through wavelet transform to obtain parameter fluctuation characteristics; The parameter fluctuation characteristics are double-evaluated by using mutual information entropy and Pearson correlation coefficient to screen out a subset of key process parameters; the subset of key process parameters is mapped into a low-dimensional feature vector through a variational autoencoder; a reinforcement learning state space is constructed based on the low-dimensional feature vector, the adjustment strategy of the process parameters is set as the action space, and a deep Q network model is established with the yield improvement level as the reward function; Calculating the similarity of process trajectories of different chip manufacturing plants based on the low-dimensional feature vector using a dynamic time warping algorithm to generate a process trajectory similarity matrix; Based on the process trajectory similarity matrix, a hybrid strategy of spectral clustering and density peak clustering is adopted to divide the multiple chip manufacturing plants into process groups, and an independent group management node is set for each process group; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model to obtain a local training gradient; differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node using homomorphic encryption; after receiving the encrypted local training gradient, the group management node uses the attention mechanism to calculate the historical yield contribution of each chip manufacturing plant and uses it as the gradient aggregation weight; weighted gradient aggregation is performed based on the gradient aggregation weight, and group optimization features are extracted through knowledge distillation, and the deep Q network model is updated to obtain a group optimization strategy model; Each chip manufacturing factory loads the population optimization strategy model, and adopts the ε-greedy strategy to select the process parameter adjustment plan based on the real-time status of the low-dimensional feature vector; constructs three-level sliding time windows of short-term, medium-term and long-term to monitor the yield change trend after adjustment respectively; when the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with historical optimization data, the network parameters of the deep Q network model are optimized by meta-learning method; The latest low-dimensional feature vector is input into the dynamic time warping algorithm to recalculate the process trajectory similarity matrix, triggering the dynamic reconstruction of the process group and realizing the adaptive adjustment of the optimization strategy, wherein the corresponding preset yield threshold is dynamically updated according to the historical yield fluctuation range of each process group.

2. The method according to claim 1, characterized in that The dynamic time warping algorithm is used to calculate the process trajectory similarity of different chip manufacturing plants based on the low-dimensional feature vector, and the process trajectory similarity matrix is ​​generated, including: Based on the low-dimensional feature vector, sampling is performed in a sliding time window to obtain time series feature points, the time series feature points are subjected to z-score normalization processing to obtain a standardized time series sequence, the variance of the standardized time series sequence in the local time window is calculated, and the elastic window size of the dynamic time warping algorithm is determined according to the variance, and the elastic window size is dynamically adjusted with the variance; Constructing a cumulative distance matrix of the dynamic time warping algorithm, calculating the Euclidean distance between different standardized time series based on the elastic window size, taking the Euclidean distance as the initial element of the cumulative distance matrix, and using the dynamic time warping algorithm to calculate the minimum cumulative distance of adjacent elements in the cumulative distance matrix, the path corresponding to the minimum cumulative distance is the optimal matching path, and calculating the dynamic time warping distance between two standardized time series based on the optimal matching path; The dynamic time warping distance is input into the Gaussian kernel function to calculate the process trajectory similarity between any two chip manufacturing plants. The kernel width parameter of the Gaussian kernel function is adaptively adjusted according to the distribution of the dynamic time warping distance. The calculated process trajectory similarity is filled into the corresponding position of the similarity matrix to generate a process trajectory similarity matrix reflecting the process trajectory similarity between different chip manufacturing plants.

3. The method according to claim 1, characterized in that Based on the process trajectory similarity matrix, the plurality of chip manufacturing plants are divided into process groups by adopting a hybrid strategy of spectral clustering and density peak clustering, and each process group is provided with an independent group management node; Each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model, and obtains the local training gradient including: Calculate the degree matrix of the process trajectory similarity matrix, wherein the diagonal elements of the degree matrix are the sum of the corresponding row elements of the process trajectory similarity matrix, construct a normalized Laplace matrix based on the degree matrix, extract the eigenvector corresponding to the minimum non-zero eigenvalue of the normalized Laplace matrix, and normalize the eigenvector to obtain a low-dimensional feature representation of each chip manufacturing plant; Calculating density indicators of each chip manufacturing plant based on the low-dimensional feature representation, wherein the density indicators include a local data density value and a relative distance value, and the local data density value calculates the data distribution density around each chip manufacturing plant through an adaptive kernel function; The product of the local data density value and the relative distance value is used as a decision value, a cluster center factory is selected based on the decision value, and the remaining chip manufacturing factories are allocated to the nearest cluster center factory according to the density reachability principle to form a process cluster; Calculating management indicators of chip manufacturing plants in each process group, the management indicators include connectivity, resource status and network performance, and selecting a group management node for each process group based on the management indicators, the group management node is used to coordinate parameter optimization of the group; Construct a deep Q network model including a strategy network and a target network. The strategy network inputs process parameters and outputs adjustment plans. The optimization value of the adjustment plans is evaluated based on the target network. Each chip manufacturing plant uses the deep Q network model to perform local parameter optimization training. The network gradient of the deep Q network model during the parameter optimization training process is calculated, and the network gradient is normalized to obtain a local training gradient.

4. The method according to claim 3, characterized in that Construct a deep Q network model including a strategy network and a target network. The strategy network inputs process parameters and outputs an adjustment plan. The optimization value of the adjustment plan is evaluated based on the target network. Each chip manufacturing plant uses the deep Q network model to perform local parameter optimization training, including: Constructing a deep Q network model including a strategy network and a target network, wherein the strategy network receives process parameter input and outputs a parameter adjustment scheme through a multi-layer neural network structure, and the target network is used to evaluate the optimization value of the parameter adjustment scheme; The process parameters are input into the strategy network, the strategy network generates a parameter adjustment plan based on the current process parameter state, the target network receives the parameter adjustment plan, and evaluates the optimization value of the parameter adjustment plan relative to the current process parameter state; Based on the optimization value of the target network evaluation, an optimal parameter adjustment scheme is selected, the optimal parameter adjustment scheme is executed to obtain the adjusted process parameters, and the corresponding yield change value is obtained; the current process parameter state, the optimal parameter adjustment scheme, the yield change value, and the adjusted process parameters are used to construct a training sample, and the deep Q network model is locally optimized and trained using the training sample.

5. The method according to claim 3, characterized in that: The density index of each chip manufacturing plant is calculated based on the low-dimensional feature representation, and the density index includes a local data density value and a relative distance value including: Calculating the local data density value of each chip manufacturing plant based on the low-dimensional feature representation, wherein the local data density value is calculated by an adaptive kernel function, and a bandwidth parameter of the adaptive kernel function is dynamically adjusted according to local data distribution characteristics; The local data density value calculation formula is as follows: ； Among them, ρ i is the local data density value of factory i, n is the total number of factories, K is the adaptive kernel function, x i is the low-dimensional feature vector of factory i, x j is the low-dimensional feature vector of factory j, is the Euclidean distance, h(x i ) is the adaptive bandwidth function; Calculating a bandwidth parameter of the adaptive kernel function according to the local data distribution characteristics, wherein the bandwidth parameter is determined by the product of a reference bandwidth and a local coefficient of variation, wherein the local coefficient of variation represents a degree of dispersion of data around each chip manufacturing plant; The adaptive kernel function is used to calculate the influence weight between each chip manufacturing plant and an adjacent plant, and the local data density value of each chip manufacturing plant is determined based on the influence weight; Calculating the relative distance value of each chip manufacturing factory according to the local data density value; The relative distance value calculation formula is as follows: ； Among them, δ i is the relative distance value of factory i, is the minimum Euclidean distance between all factories whose local data density values ​​are greater than factory i and factory i, is the relative distance value of the point with the maximum density, d ij is the Euclidean distance from factory i to factory j, ρ j is the local data density value of factory j; When the local data density value of a chip manufacturing plant is the largest, the maximum distance from the chip manufacturing plant to other plants is taken as the relative distance value.

6. The method according to claim 1, characterized in that The local training gradient is injected with differential privacy noise through Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node by homomorphic encryption; after receiving the encrypted local training gradient, the group management node uses the attention mechanism to calculate the historical yield contribution of each chip manufacturing plant, and uses it as the gradient aggregation weight including: Performing density estimation on the local training gradient to construct a Gaussian mixture model, using an expectation maximization algorithm to calculate the mean vector and covariance matrix of the Gaussian mixture model, determining a mixing weight based on the mean vector and the covariance matrix, and obtaining a distribution characteristic of the local training gradient; Calculate the sensitivity of the local training gradient to a single training sample, determine the noise intensity based on a preset privacy budget and the sensitivity, sample a noise vector of corresponding intensity from the Gaussian mixture model, and superimpose the noise vector on the local training gradient to obtain a noisy gradient; Generate a homomorphic encryption key pair, the homomorphic encryption key pair includes a public key for encryption and a private key for decryption, use the public key to encrypt the noisy gradient to obtain an encrypted gradient, and transmit the encrypted gradient to the group management node; Building a temporal attention module based on the group management node, the temporal attention module receives historical yield data of a chip manufacturing plant, uses position coding to maintain the temporal relationship of the historical yield data, and calculates the correlation strength of the yield state at each moment through an attention mechanism; The current yield status is used as the query vector and the historical yield sequence is used as the key vector to calculate the attention score. The contribution of each chip manufacturing plant is evaluated based on the attention score and the degree of yield improvement. The contribution is normalized to obtain the gradient aggregation weight.

7. The method according to claim 1, characterized in that When the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with historical optimization data, the network parameters of the deep Q network model are optimized by a meta-learning method, including: Construct a multi-level time window to monitor the yield index, the multi-level time window includes a four-hour real-time window, a twenty-four-hour daily window and a seven-day weekly window, each time window corresponds to a preset yield threshold, and continuously monitors the change trend of the yield index in each time window; when the yield index in any time window is continuously lower than the corresponding preset yield threshold, collect the current process parameters, equipment status and yield data, construct the process parameters, the equipment status and the yield data as a state vector and input it into the deep Q network model; The deep Q network model re-evaluates the optional action space under the current process state based on the state vector, and the optional action space includes multiple groups of process parameter adjustment schemes, and each group of process parameter adjustment schemes corresponds to different parameter combinations; extracts empirical data including state information, action information and optimization effects from historical optimization data to construct an optimization task library, and retrieves similar scenarios in the optimization task library based on the state vector to obtain optimization strategy features under the similar scenarios; A meta-learning method is used to migrate the optimization strategy features to the current scenario, the network parameters of the deep Q network model are adjusted to adapt the deep Q network model to the current optimization needs, and the optional action space is evaluated based on the adjusted network parameters; the action value of each process parameter adjustment scheme in the optional action space is calculated, the process parameter adjustment scheme with the highest action value is selected as the optimization scheme, the optimization scheme is executed and the optimization effect is recorded.

8. A chip manufacturing process parameter optimization system based on federated learning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect process parameter data and yield data of multiple chip manufacturing plants, and perform multi-scale decomposition of the process parameter data through wavelet transform to obtain parameter fluctuation characteristics; The parameter fluctuation characteristics are double-evaluated by using mutual information entropy and Pearson correlation coefficient to screen out a subset of key process parameters; the subset of key process parameters is mapped into a low-dimensional feature vector through a variational autoencoder; a reinforcement learning state space is constructed based on the low-dimensional feature vector, the adjustment strategy of the process parameters is set as the action space, and a deep Q network model is established with the yield improvement level as the reward function; Calculating the similarity of process trajectories of different chip manufacturing plants based on the low-dimensional feature vector using a dynamic time warping algorithm to generate a process trajectory similarity matrix; The second unit is used to divide the multiple chip manufacturing plants into process groups based on the process trajectory similarity matrix by adopting a hybrid strategy of spectral clustering and density peak clustering, and each process group is provided with an independent group management node; each chip manufacturing plant trains a local parameter optimization strategy based on the deep Q network model to obtain a local training gradient; differential privacy noise is injected into the local training gradient through a Gaussian mixture distribution, and the encrypted local training gradient is transmitted to the corresponding group management node by homomorphic encryption; after receiving the encrypted local training gradient, the group management node calculates the historical yield contribution of each chip manufacturing plant by using the attention mechanism, and uses it as the gradient aggregation weight; weighted gradient aggregation is performed based on the gradient aggregation weight, and group optimization features are extracted by knowledge distillation, and the deep Q network model is updated to obtain a group optimization strategy model; The third unit is used for each chip manufacturing factory to load the population optimization strategy model, and select the process parameter adjustment plan based on the real-time state of the low-dimensional feature vector using the ε-greedy strategy; construct short-term, medium-term, and long-term three-level sliding time windows to monitor the yield change trend after adjustment respectively; when the yield index of any time window is continuously lower than the corresponding preset yield threshold, the deep Q network model is used to re-evaluate the optional action space based on the current state, and combined with historical optimization data, the network parameters of the deep Q network model are optimized by the meta-learning method; The latest low-dimensional feature vector is input into the dynamic time warping algorithm to recalculate the process trajectory similarity matrix, triggering the dynamic reconstruction of the process group and realizing the adaptive adjustment of the optimization strategy, wherein the corresponding preset yield threshold is dynamically updated according to the historical yield fluctuation range of each process group.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images