Virtual measurement method and system based on important sample retention and convolutional neural network

Abstract

The invention provides a virtual measurement method and system based on important sample retention and a convolutional neural network. The modeling operation of the virtual measurement method based on important sample retention (KIS) and the convolutional neural network comprises classification of pairwise data and non-pairwise process data; a pre-training model is established by using non-pairwise process data, KIS operation is performed on pairwise data to generate important samples, and then the important samples are input to the pre-training model to establish a virtual measurement model based on a convolutional auto-encoder and KIS. The transfer learning stage operation includes performing calculation according to a virtual measurement model based on a convolutional auto-encoder and a KIS, and the number of important samples is smaller than the number of paired data. Therefore, offline modeling is realized by using a down-sampling KIS mechanism, and balance and improvement of prediction accuracy are achieved by using a two-stage algorithm with an online KIS mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to a virtual metrology method and system, and in particular to a virtual metrology method and system based on Keep Important Samples (KIS) and Convolutional Neural Network (CNN). BACKGROUND

[0002] Stable processing and high yield production has been the relentless pursuit of manufacturing industry, and off-line sampling inspection is the most commonly used method to achieve this goal. However, this method can only evaluate the quality of the sampled workpiece, and has time delay, and cannot monitor product quality in real time. Due to this deficiency, changes occurring during the measurement delay period can cause production loss. Automatic Virtual Metrology (AVM) technology can achieve online and real-time full inspection to solve these problems. With the increasing complexity of high-tech industries (such as semiconductors, TFT-LCD), the accuracy of virtual metrology prediction is also increasing.

[0003] However, in practical applications, the conventional AVM still has the following two problems to overcome. First, the scarcity and imbalance of collected measurement values result in poor prediction accuracy of extreme values. Second, the model can only be updated when sufficient measurement values are collected. Therefore, there is a lack of a virtual metrology method and system based on KIS and CNN in the market that considers data balance and improves prediction accuracy, so relevant industries are seeking solutions. SUMMARY

[0004] The purpose of the present application is to provide a virtual metrology method and system based on KIS and CNN, which uses a convolutional autoencoder, K-means algorithm, and KIS mechanism with cosine distance downsampling to achieve offline modeling, and uses a two-stage algorithm with online KIS mechanism to solve the problems of imbalance and poor prediction accuracy of the conventional method.

[0005] According to an embodiment of the method aspect of the present application, a virtual metrology method based on important sample reservation and convolutional neural network is provided, comprising the following steps: configuring a processor to obtain a plurality of sets of process data, wherein the sets of process data are used or generated when a plurality of workpieces are processed by a production machine, the sets of process data correspond to the workpieces in a one-to-one manner, each of the sets of process data comprises values of a plurality of parameters, the values of each of the parameters correspond to a plurality of sets of time series data of the workpieces respectively, and each of the sets of time series data has a data length; configuring the processor to perform a data rounding operation on the sets of process data, the data rounding operation comprising: performing a data length adjustment operation to repeatedly add at least one data point having a value of a last data point after the last data point of each of the sets of time series data of each of the parameters until the data length of each of the sets of time series data of each of the parameters is equal to a longest data length of the sets of process data; configuring the processor to obtain a plurality of actual measurement values of the workpieces; configuring the processor to perform a modeling operation, the modeling operation comprising: dividing the sets of process data and the actual measurement values into a plurality of paired data and at least one unpaired process data, each of the paired data comprising one of the sets of process data and one of the actual measurement values corresponding thereto; and using the at least one unpaired process data to establish at least one pre-trained model, performing an important sample reservation operation on the paired data to generate a plurality of important samples, and then inputting the important samples into the at least one pre-trained model to establish a virtual metrology model based on convolutional autoencoder and important sample reservation, wherein the virtual metrology model based on convolutional autoencoder and important sample reservation comprises at least one convolutional neural network model; and configuring the processor to perform a calculation operation, the calculation operation comprising: obtaining another set of process data of another workpiece and at least one of another actual measurement value, and performing one of a prediction stage operation and a transfer learning stage operation according to whether the another actual measurement value is obtained to calculate one of a first stage virtual measurement value and a second stage virtual measurement value of the another workpiece. The transfer learning stage operation comprises performing calculation according to the virtual metrology model based on convolutional autoencoder and important sample reservation, and the number of the important samples is less than the number of the paired data.

[0006] Therefore, the virtual metrology method based on important sample reservation and convolutional neural network of the present application uses the KIS mechanism of convolutional autoencoder, K-means algorithm and cosine distance downsampling to realize offline modeling, can reserve important samples in each value range to achieve balance, effectively solves the imbalance problem, and enhances the learning effect of the model during fine-tuning.

[0007] Other implementations of the aforementioned implementations are as follows: the aforementioned reserving important samples operation comprises: judging each of the pairs of data to belong to one of an extreme reserving group and a selected reserving group according to a distribution of the pairs of data, and down-sampling part of the pairs of data belonging to the selected reserving group to obtain a plurality of reserving data, the important samples comprising another part of the pairs of data belonging to the extreme reserving group and the reserving data.

[0008] Other implementations of the aforementioned implementations are as follows: the aforementioned distribution of the pairs of data is a normal distribution, and the reserving important samples operation further comprises: dividing the normal distribution into the extreme reserving group and the selected reserving group according to a judgment condition, and dividing the pairs of data into the part of the pairs of data and the another part of the pairs of data according to the extreme reserving group and the selected reserving group.

[0009] Other implementations of the aforementioned implementations are as follows: the aforementioned judgment condition conforms to the following formula:

[0010] and

[0011]

[0012] wherein p represents a parameter, the parameter being greater than 1 and less than or equal to 2; y i represents an i-th one of the actual measurement values; represents a mean value of y i ; σ y represents a standard deviation of y i , and σ y conform to the following formula:

[0013] and

[0014]

[0015] wherein n represents a sample number of y i , in the normal distribution, the part of the pairs of data conforming to the judgment condition belongs to the extreme reserving group, and the another part of the pairs of data not conforming to the judgment condition belongs to the selected reserving group.

[0016] Other implementations of the aforementioned implementations are as follows: the aforementioned reserving important samples operation further comprises: grouping the another part of the pairs of data according to a grouping algorithm to generate a plurality of data groups, and setting a threshold value of the data groups, and calculating a group center and two proportion parameters of each of the data groups according to the threshold value, wherein the threshold value is represented by and conforms to the following formula:

[0017]

[0018] and

[0019]

[0020] wherein, represents an average of cosine distances between two samples in the kth data group of the mth cluster; represents a standard deviation of cosine distances between two samples in the kth data group of the mth cluster; represents a cosine distance between a vector of sample b and a vector of sample c in the kth data group of the mth cluster; a represents a threshold setting factor; sample b, c belong to q and are different from each other; q represents a number of samples of the mth cluster.

[0021] Other implementations of the foregoing embodiment are as follows: the foregoing important sample reservation operation further comprises: dividing one of the data groups into multiple sections according to the cluster center and the two proportion parameters; and calculating a cosine distance between a sample in the one of the data groups and the cluster center, and performing a distributed sample operation according to the cosine distances between the samples in the one of the data groups and the cluster center, the distributed sample operation satisfying the following formula:

[0022]

[0023] wherein, represents the cluster center; represents the cosine distance between the sample in the one of the data groups and the cluster center; and represents the two proportion parameters; represents a vector of the bth sample in the kth data group of the mth cluster; represents a vector of the cluster center in the kth data group of the mth cluster; represents the bth sample in the kth data group of the mth cluster; S mks represents the s th section in the kth data group of the mth cluster, s is one of 1, 2, 3.

[0024] Other implementations of the foregoing embodiment are as follows: the foregoing important sample reservation operation further comprises: calculating a cosine distance between two samples in the s th section in the kth data group of the mth cluster, satisfying the following formula:

[0025]

[0026] wherein, represents a vector of the e th sample in the s th section in the kth data group of the mth cluster; a vector representing an fth sample of an sth section in a kth data group of an mth cluster, e, f being different from each other.

[0027] Other implementations of the foregoing embodiments include the following. The foregoing reserved important sample operation further includes: determining whether a number of section samples in the sth section of the kth data group of the mth cluster is greater than a preset section sample number to generate a determination result, and then determining to perform a selected important sample operation or an obtained important sample set operation according to the determination result. When the determination result is yes, sequentially performing the selected important sample operation, the obtained important sample set operation, and a confirmed all sample operation. When the determination result is no, sequentially performing the obtained important sample set operation and the confirmed all sample operation. The selected important sample operation includes selecting three samples with the largest cosine distance in the sth section of the kth data group of the mth cluster, and moving the three samples to an important sample set. The obtained important sample set operation includes obtaining the important samples of the important sample set. The confirmed all sample operation includes confirming whether the inspection of all samples is completed.

[0028] Other implementations of the foregoing embodiments include the following. The foregoing transfer learning stage operation of the calculation operation includes: regarding the another actual measurement value as a new sample, calculating a cosine distance between the new sample and a cluster center in the kth data group of the mth cluster, performing the assigned sample operation according to the cosine distance between the new sample and the cluster center in the kth data group of the mth cluster, and confirming whether the new sample becomes another important sample.

[0029] Other implementations of the foregoing embodiments include the following. The foregoing prediction stage operation includes calculating the first stage virtual measurement value from the another set of process data according to the virtual measurement model based on the convolutional autoencoder and the reserved important sample. The transfer learning stage operation includes calculating the second stage virtual measurement value from the another set of process data and the another actual measurement value according to the virtual measurement model based on the convolutional autoencoder and the reserved important sample. The virtual measurement model based on the convolutional autoencoder and the reserved important sample is used to control a production machine to process the workpieces. Either the first stage virtual measurement value output by the prediction stage operation or the second stage virtual measurement value output by the transfer learning stage operation corresponds to a dry etching process applied to a semiconductor process by the production machine.

[0030] According to another embodiment of the structure pattern of the present application, a virtual metrology system based on important sample reservation and convolutional neural network is provided, comprising a memory and a processor. The memory stores a plurality of sets of process data and a plurality of actual measurement values, wherein the sets of process data are used or generated when a plurality of workpieces are processed by a production machine, the sets of process data correspond to the workpieces in a one-to-one manner, each of the sets of process data comprises values of a plurality of parameters, the values of each of the parameters correspond to a plurality of sets of time series data of the workpieces respectively, and each of the sets of time series data has a data length. The processor is electrically connected to the memory, receives the sets of process data and the actual measurement values, and is configured to perform operations comprising: performing a data alignment operation on the sets of process data, the data alignment operation comprising: performing a data length adjustment operation to repeatedly add at least one data point having a value of a last data point to each of the sets of time series data of each of the parameters after the last data point until the data length of each of the sets of time series data of each of the parameters is equal to a longest data length of the sets of process data; performing a modeling operation, the modeling operation comprising: dividing the sets of process data and the actual measurement values into a plurality of paired data and at least one unpaired process data, each of the paired data comprising one of the sets of process data and one of the corresponding actual measurement values; and using the at least one unpaired process data to establish at least one pre-trained model, performing an important sample reservation operation on the paired data to generate a plurality of important samples, and then inputting the important samples into the at least one pre-trained model to establish a virtual metrology model based on convolutional autoencoder and important sample reservation, wherein the virtual metrology model based on convolutional autoencoder and important sample reservation comprises at least one convolutional neural network model; and performing a calculation operation, the calculation operation comprising: obtaining another set of process data of another workpiece and at least one of another actual measurement value, and performing one of a prediction stage operation and a transfer learning stage operation according to whether the another actual measurement value is obtained, to calculate one of a first stage virtual measurement value and a second stage virtual measurement value of the another workpiece. The transfer learning stage operation comprises performing calculation according to the virtual metrology model based on convolutional autoencoder and important sample reservation, and the number of the important samples is less than the number of the paired data.

[0031] Therefore, the virtual metrology system based on important sample reservation and convolutional neural network of the present application uses a two-stage algorithm with an online KIS mechanism to select important samples to realize instant model refreshing, thereby avoiding the problem of sample imbalance when new samples enter the system. Furthermore, in a time-varying system, the two-stage algorithm with the online KIS mechanism has model refreshing capability, which improves the learning efficiency of the model during fine-tuning and ensures good prediction accuracy.

[0032] Other implementations of the aforementioned implementations are as follows: the aforementioned reserving important samples operation comprises: judging each of the pairs of data to belong to one of an extreme reserving group and a selected reserving group according to a distribution of the pairs of data, and down-sampling part of the pairs of data belonging to the selected reserving group to obtain a plurality of reserved data, the important samples comprising another part of the pairs of data belonging to the extreme reserving group and the reserved data.

[0033] Other implementations of the aforementioned implementations are as follows: the aforementioned distribution of the pairs of data is a normal distribution, and the reserving important samples operation further comprises: dividing the normal distribution into the extreme reserving group and the selected reserving group according to a judgment condition, and dividing the pairs of data into the part of the pairs of data and the another part of the pairs of data according to the extreme reserving group and the selected reserving group.

[0034] Other implementations of the aforementioned implementations are as follows: the aforementioned judgment condition conforms to the following formula:

[0035] and

[0036]

[0037] wherein p represents a parameter, the parameter being greater than 1 and less than or equal to 2; y i represents an i-th one of the actual measured values; represents a mean value of y i ; σ y represents a standard deviation of y i , and σ y conform to the following formula:

[0038] and

[0039]

[0040] wherein n represents a sample number of y i , in the normal distribution, the part of the pairs of data conforming to the judgment condition belongs to the extreme reserving group, and the another part of the pairs of data not conforming to the judgment condition belongs to the selected reserving group.

[0041] Other implementations of the aforementioned implementations are as follows: the aforementioned reserving important samples operation further comprises: grouping the another part of the pairs of data according to a grouping algorithm to generate a plurality of data groups, and setting a threshold value of the data groups, and calculating a group center and two proportion parameters of each data group according to the threshold value, wherein the threshold value is represented by and conforms to the following formula:

[0042]

[0043] and

[0044]

[0045] wherein, represents an average of cosine distances between two samples in the kth data group of the mth cluster; represents a standard deviation of cosine distances between two samples in the kth data group of the mth cluster; represents a cosine distance between a vector of sample b and a vector of sample c in the kth data group of the mth cluster; a represents a threshold setting factor; sample b, c belong to q and are different from each other; q represents a number of samples of the mth cluster.

[0046] Other implementations of the foregoing embodiment are as follows: the foregoing important sample reservation operation further comprises: dividing one of the data groups into multiple sections according to the cluster center and the two proportion parameters; and calculating a cosine distance between a sample in the one of the data groups and the cluster center, and performing a distributed sample operation according to the cosine distances between the samples in the one of the data groups and the cluster center, the distributed sample operation satisfying the following formula:

[0047]

[0048] wherein, represents the cluster center; represents the cosine distance between the sample in the one of the data groups and the cluster center; and represents the two proportion parameters; represents a vector of the bth sample in the kth data group of the mth cluster; represents a vector of the cluster center in the kth data group of the mth cluster; represents the bth sample in the kth data group of the mth cluster; S mks represents the s th section in the kth data group of the mth cluster, s is one of 1, 2, 3.

[0049] Other implementations of the foregoing embodiment are as follows: the foregoing important sample reservation operation further comprises: calculating a cosine distance between two samples in the s th section in the kth data group of the mth cluster, satisfying the following formula:

[0050]

[0051] wherein, represents a vector of the e th sample in the s th section in the kth data group of the mth cluster; a vector representing an fth sample of an sth section in a kth data group of an mth cluster, e, f being different from each other.

[0052] Other implementations of the foregoing embodiments include the following. The foregoing retaining important sample operation further includes: determining whether a number of section samples in the sth section of the kth data group of the mth cluster is greater than a preset section sample number to generate a determination result, and then determining to perform a selecting important sample operation or an obtaining important sample set operation according to the determination result; when the determination result is yes, sequentially performing the selecting important sample operation, the obtaining important sample set operation, and a confirming all sample operation; and when the determination result is no, sequentially performing the obtaining important sample set operation and the confirming all sample operation. The selecting important sample operation includes selecting three samples with the largest cosine distance in the sth section of the kth data group of the mth cluster, and moving the three samples to an important sample set. The obtaining important sample set operation includes obtaining the important samples of the important sample set. The confirming all sample operation includes confirming whether the inspection of all samples is completed.

[0053] Other implementations of the foregoing embodiments include the following. The foregoing transfer learning stage operation of the computing operation includes: regarding the another actual measurement value as a new sample, calculating a cosine distance between the new sample and a cluster center in the kth data group of the mth cluster, and performing the assigning sample operation according to the cosine distance between the new sample and the cluster center in the kth data group of the mth cluster, and confirming whether the new sample becomes another important sample.

[0054] Other implementations of the foregoing embodiments include the following. The foregoing prediction stage operation includes calculating the first stage virtual measurement value from the another set of process data according to the virtual measurement model based on the convolutional autoencoder and the retaining important sample. The transfer learning stage operation includes calculating the second stage virtual measurement value from the another set of process data and the another actual measurement value according to the virtual measurement model based on the convolutional autoencoder and the retaining important sample. The virtual measurement model based on the convolutional autoencoder and the retaining important sample is used to control a production machine to process the workpieces. Either one of the first stage virtual measurement value output by the prediction stage operation and the second stage virtual measurement value output by the transfer learning stage operation corresponds to a dry etching process applied to a semiconductor process by the production machine. BRIEF DESCRIPTION OF DRAWINGS

[0055] For a more complete understanding of the embodiments and their advantages, reference is now made to the following description taken in conjunction with the accompanying drawings in which,

[0056] Figure 1 is a block schematic diagram illustrating a virtual measurement system based on important sample retaining and convolutional neural network of a first embodiment of the present disclosure;

[0057] Figure 2is a flowchart illustrating a virtual metrology method based on important sample reservation and convolutional neural network of a second embodiment of the present application;

[0058] Figure 3 is a flowchart illustrating a modeling operation based on convolutional autoencoder and reservation of important samples of Figure 2

[0059] Figure 4 is a schematic diagram of a network of a convolutional autoencoder of Figure 2

[0060] Figure 5 is a schematic diagram illustrating an important sample reservation operation of the present application;

[0061] Figure 6 is a flowchart illustrating reservation of all segments of each group of important samples of Figure 3

[0062] Figure 7A is a flowchart illustrating a two-stage mechanism based on transfer learning and reservation of important samples of a calculation operation of Figure 2

[0063] Figure 7B is a flowchart illustrating an online KIS mechanism of Figure 7A

[0064] Figure 8 is a schematic diagram illustrating results of modeling and testing of an application example of the present application with respect to different prediction algorithms;

[0065] Figure 9A is a schematic diagram illustrating depth values of an application example of the present application with respect to different prediction algorithms when a state change occurs in a dry etching process; and

[0066] Figure 9B is a schematic diagram illustrating cosine distances between sample number 274 and other sample numbers of Figure 9A

[0067]

Symbol Description

[0068] 100: virtual metrology system based on important sample reservation and convolutional neural network

[0069] 1002: memory

[0070] 1004: processor

[0071] 102: process data

[0072] 104: metrology data

[0073] 106: process data preprocessing module ​​​​​​

[0074] 108: ADAS

[0075] 110: Virtual metrology model with CAE, TL, and reserved important samples

[0076] 112: Metrology data pre-processing module

[0077] 114: Virtual metrology model based on CAE and reserved important samples

[0078] 116: Two-stage mechanism based on TL and reserved important samples

[0079] 120: RI model

[0080] 130: GSI model

[0081] 200: Virtual metrology method based on important sample reservation and convolutional neural network

[0082] 274: Sample number

[0083] 3D_1, 3D_2, 3D_p: CAE model decoder

[0084] 3E_1, 3E_2, 3E_p: CAE model encoder

[0085] 300: Prediction stage operation

[0086] 400: Transfer learning stage operation

[0087] IS: Important sample

[0088] L1, L2: Curve

[0089] NIS: Non-important sample

[0090] S02, S04, S042, S06, S062, S064, S06A, S06B, S06C, S06D, S06E, S06F, S06G, S06H, S06I, S06J, S06J1, S06J2, S06J3, S06J4, S06J5, S06J6, S06K, S08, S082, 302, 304, 306, 308, 310, 312, 402, 404, 406, 408, 410, 412, 414, 416, 417, 4171, 4172, 4173, 4174, 4175, 4176, 418, 420, 422, 424, 426, 428, 430: Step DETAILED DESCRIPTION

[0091] Embodiments of the present application will be described herein below with reference to the accompanying drawings. Many practical details will be set forth in the description that follows to provide a thorough understanding of embodiments of the application. It will be apparent, however, that the application can be practiced without these specific details. In other instances, well known structures and components are not described in detail or are shown in block diagram form in order to avoid unnecessarily obscuring the present application. Moreover, the use of the same reference numerals in different illustrations is intended to indicate similar or identical components.

[0092] In addition, as used herein, the term "connected" can mean directly connected, and can also mean indirectly connected via one or more other devices or connections. Where the context permits, "connected" can also refer to being "directly connected", i.e. not through intervening devices or connections. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The terms "first", "second", "third", etc. are used herein only to describe different elements, and are not intended to be limiting. Thus, a first element could also be termed a second element without departing from the teachings of the present application. As used herein, the term "comprises" or "comprising" or "includes" or "including" means covering the non-exclusive inclusion such that processes, methods, articles, or apparatuses that comprise, include, or consist of any of the specified steps or elements plus any other steps or elements are also covered.

[0093] Reference will now be made to the drawings, wherein: Figure 1 With reference to Figure 2 Wherein Figure 1 is a block diagram illustrating a convolutional neural network (CNN) and keeping important samples (KIS) based virtual metrology system 100 of a first embodiment of the present application; and Figure 2 is a flow diagram illustrating a keeping important samples and convolutional neural network based virtual metrology method 200 of a second embodiment of the present application. The keeping important samples and convolutional neural network based virtual metrology system 100 comprises a memory 1002 and a processor 1004.

[0094] The memory 1002 stores a plurality of sets of process data 102 and a plurality of sets of measurement data 104 (i.e., a plurality of actual measurement values). The sets of process data 102 are used or generated when a workpiece is processed by a production machine, and the sets of process data 102 are corresponded to the workpiece in a one-to-one manner. Each set of process data 102 includes a plurality of parameter values, each parameter value corresponding to a plurality of sets of timing data of the workpiece, each set of timing data having a data length. In other words, the memory 1002 obtains the aforementioned sets of process data 102 of historical workpieces from the production machine. Further, the sets of measurement data 104 (the actual measurement values) are obtained after the workpiece is measured by a measurement machine for quality items. For wafer processing (semiconductor processing), the production machine is a wafer processing machine, such as an etching machine, a deposition machine, a sputtering machine, etc.; the actual measurement values are film thickness, etching depth, etching side wall angle, critical dimension (CD), etc.; and the process data 102 includes temperature. For wafer cutting processing, the production machine is a wafer cutting machine; the actual measurement values are wafer breakage amount; and the process data 102 includes blade clogging, coolant flow rate, spindle speed, feed rate, wafer condition (such as thickness, coating, etc.), and / or kerf width. For tool machine processing, the production machine is a tool machine; the actual measurement values include roughness, straightness, angularity, perpendicularity, parallelism, and / or roundness; and the process data 102 includes working current, vibration and / or audio data obtained by a detector (such as a three-axis accelerometer and / or acoustic emission (AE) sensor) installed on the tool machine.

[0095] The processor 1004 is electrically connected to the memory 1002. The processor 1004 receives the process data 102 and the metrology data 104 (actual metrology values) and is configured to implement the virtual metrology method 200 based on the KIS scheme and the convolutional neural network. In detail, the processor 1004 includes a process data pre-processing module 106, a virtual metrology model 110 with a convolutional autoencoder (CAE), transfer learning (TL), and KIS scheme, a metrology data pre-processing module 112, a Reliance Index (RI) model 120, and a Global Similarity Index (GSI) model 130. The process data pre-processing module 106 receives a plurality of sets of process data 102. The process data pre-processing module 106 performs data alignment on the process data 102 according to an Automated Data Alignment Scheme (ADAS) 108 to remove process data 102 with different time distribution shapes and to make the data lengths of the process data 102 the same. Before or after the data alignment, the process data pre-processing module 106 can perform data quality evaluation of the process data 102 according to a Process Data Quality Index (DQI X ) model and arrange and standardize the original process data 102 from the production tool. The metrology data pre-processing module 112 performs data quality evaluation of the metrology data 104 (i.e., the actual metrology values) of the historical workpieces according to a Metrology Data Quality Index (DQI y ) model, removes outliers, and standardizes the metrology data 104. Then, the metrology data 104 of the historical workpieces and the aligned process data 102 are used as modeling samples, and the virtual metrology model 110 with the CAE, TL, and KIS scheme, the RI model 120, and the GSI model 130 are established according to the convolutional neural network algorithm and the KIS scheme.

[0096] The CAE, TL and important sample preserving virtual metrology model 110 includes a convolutional autoencoder and important sample preserving virtual metrology model 114 and a transfer learning and important sample preserving two-stage mechanism 116. After the CAE, TL and important sample preserving virtual metrology model 110, the RI model 120 and the GSI model 130 are modeled, the convolutional autoencoder and important sample preserving virtual metrology model 114 is generated, and then the transfer learning and important sample preserving two-stage mechanism 116 can be used to calculate the virtual metrology of subsequent workpieces. The convolutional autoencoder and important sample preserving virtual metrology model 114 is used to control the production machine to process workpieces. The transfer learning and important sample preserving two-stage mechanism 116 includes a prediction stage operation (Phase-I) and a transfer learning stage operation (Phase-II). In the prediction stage operation (Phase-I), after a set of process data 102 of a workpiece is obtained, the process data preprocessing module 106 performs data alignment operation according to the automatic data alignment system (ADAS) 108 and / or other data preprocessing. Then, the processed process data 102 is input to the CAE, TL and important sample preserving virtual metrology model 110, the RI model 120 and the GSI model 130 to calculate the first stage virtual metrology value (VM I ) and the confidence index value (RI) and the overall similarity index value (GSI) of the workpiece. In the transfer learning stage operation (Phase-II), when the workpiece is processed by the production machine, if the quality items of the workpiece are measured by the measuring machine to obtain the measurement data 104 (i.e. actual measurement value), the process data 102 and the measurement data 104 of the workpiece can be used to retrain or calibrate the CAE, TL and important sample preserving virtual metrology model 110, the RI model 120 and the GSI model 130 to calculate the second stage virtual metrology value (VM II ) and the confidence index value (RI) and the overall similarity index value (GSI) of the workpiece.

[0097] The RI model 120, the GSI model 130, the DQI X model and the DQI yThe model can refer to U.S. Patent No. 8,095,484 B2. That is, embodiments of the present disclosure incorporate by reference the relevant provisions of this U.S. patent. In addition, the aforementioned memory 1002 can include a random access memory (RAM) or other type of dynamic storage device that can store information and instructions for execution by the processor 1004. The processor 1004 can include any type of processors, microprocessors, or field programmable gate arrays (FPGAs) that can compile and execute instructions. The processor 1004 can include a single device (e.g., a single core) or a group of devices (e.g., multiple cores).

[0098] Therefore, the virtual metrology system 100 based on the important sample reservation and convolutional neural network of the present disclosure uses the KIS mechanism of the convolutional autoencoder, K-means algorithm, and cosine distance down-sampling to achieve offline modeling, which can reserve important samples in each value range to achieve balance, effectively solve the imbalance problem, and enhance the learning effect of the model during fine-tuning.

[0099] Figure 2 The virtual metrology method 200 based on the important sample reservation and convolutional neural network of the present disclosure includes steps S02, S04, S06, and S08. Step S02 includes an operation of data collection to obtain a plurality of sets of process data 102, wherein the sets of process data 102 are used or generated when a production machine processes a plurality of workpieces, the sets of process data 102 correspond to the workpieces in a one-to-one manner, each set of process data 102 includes a plurality of parameter values, each parameter value corresponds to a plurality of sets of time series data of the workpieces, and each set of time series data has a data length.

[0100] Step S04 includes a data alignment operation on the groups of process data 102, which includes a frequency distribution calculation on the data length of the multiple sets of timing data of each parameter to obtain a distribution of data occurrence frequency versus data length, wherein the data length with the highest occurrence frequency in the multiple sets of timing data of each parameter is a reference data length. Then, a reference timing data obtaining operation is performed to average the timing data of each parameter with the reference data length in the multiple sets of timing data of each parameter to obtain a group of reference timing data of each parameter. After obtaining the group of reference timing data of each parameter, a data distance calculation operation is performed to calculate a distance between each set of timing data of each parameter and its corresponding reference timing data using a dynamic time warping (DTW) algorithm. DTW is a method to measure the similarity between two timing data (time series), which is widely used in the field of speech recognition to identify whether two pieces of speech represent the same word. Then, a distance threshold is set. When the distance (data distance) between the timing data of a parameter and its corresponding reference timing data is greater than the distance threshold, the process data 102 corresponding to the distance is deleted, i.e. the group of process data 102 is too dissimilar to other groups of process data 102 to be used for modeling. Next, an upper limit of data length is set. Then, step S042 is performed, which includes a data length adjustment operation to repeatedly add at least one data point with the value of the last data point of each set of timing data of each parameter after the last data point until the data length of each set of timing data of each parameter is equal to the longest data length of all process data 102.

[0101] After performing the data length adjustment operation (step S042), multiple measurement data 104 (i.e. multiple actual measurement values) of the workpieces are obtained, and then step S06 is performed. Step S06 includes a modeling operation, which includes steps S062 and S064. Step S062 includes dividing the groups of process data 102 and the actual measurement values into multiple paired data and at least one unpaired process data, each paired data including one process data 102 and one corresponding actual measurement value. Step S064 includes using the at least one unpaired process data to establish at least one pre-trained model, and performing a reserved important sample operation on the paired data to generate multiple important samples, and then inputting the important samples to the at least one pre-trained model to establish a convolutional autoencoder and reserved important sample based virtual measurement model 114, wherein the convolutional autoencoder and reserved important sample based virtual measurement model 114 includes at least one convolutional neural network model. The number of important samples is less than the number of paired data.

[0102] Step S08 performs a calculation operation (test operation) including step S082 to obtain at least one of another set of process data 102 of another workpiece and another actual measurement value, and perform one of a prediction phase operation (Phase-I) and a transfer learning phase operation (Phase-II) to calculate one of a first phase virtual measurement value (VM I ) and a second phase virtual measurement value (VM II ) of the another workpiece, depending on whether the another actual measurement value is obtained. The transfer learning phase operation (Phase-II) includes performing calculation based on the convolutional autoencoder and the virtual measurement model 114 preserving important samples. Wherein the prediction phase operation (Phase-I) is calculated by the another set of process data 102 based on the convolutional autoencoder and the virtual measurement model 114 preserving important samples to obtain the first phase virtual measurement value (VM I ); and the transfer learning phase operation (Phase-II) is calculated by the another set of process data 102 and the another actual measurement value based on the convolutional autoencoder and the virtual measurement model 114 preserving important samples to obtain the second phase virtual measurement value (VM II ) of the another workpiece. The two-phase mechanism 116 based on transfer learning and preserving important samples of the present application is implemented by the prediction phase operation (Phase-I) and the transfer learning phase operation (Phase-II). The details of the modeling operation of step S06 and the calculation operation of step S08 are described in the following detailed embodiments.

[0103] Please refer to Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 , wherein Figure 3 is a flowchart illustrating the modeling operation (step S06) based on the convolutional autoencoder and preserving important samples of Figure 2 ; Figure 4 is a schematic diagram illustrating the network of the convolutional autoencoder of Figure 2 ; Figure 5 is a schematic diagram illustrating the operation of preserving important samples (steps S06F, S06G, S06H, S06I) of the present application; and Figure 6 is a schematic diagram illustrating Figure 3Schematic diagram of a process for retaining important samples from all segments of each group (step S06J). As shown, the operation of retaining important samples in step S06 includes determining whether each of these paired data belongs to an extreme retention group or a selected retention group based on a distribution of these paired data, and downsampling the portion of these paired data belonging to the selected retention group to obtain a plurality of retained data. These important samples include another portion of these paired data belonging to the extreme retention group and these retained data. Step S06 includes steps S06A, S06B, S06C, S06D, S06E, S06F, S06G, S06H, S06I, S06J, and S06K.

[0104] Step S06A includes collecting historical workpiece process data 102 and measurement data 104 (i.e., actual measurement values). Step S06B includes performing data alignment on the process data 102 using an automatic data alignment mechanism (ADAS) 108 to delete process data 102 with dissimilar temporal distribution shapes and to equalize the data lengths of the process data 102. Step S06C includes dividing the sets of process data 102 and the actual measurement values ​​into a plurality of paired data and at least one unpaired process data set, each paired data set comprising one piece of process data 102 and a corresponding one of the actual measurement values.

[0105] Step S06D includes determining whether the paired data are normally distributed and generating a distribution confirmation result. If the distribution confirmation result is yes (i.e., the distribution of the paired data is normally distributed), step S06E is executed; if the distribution confirmation result is no, step S06A is executed again.

[0106] Step S06E includes using the at least one unpaired process data to establish at least one pre-trained model. In step S06E, the unpaired process data is used to establish a pre-trained model, and the pre-trained model is formed by a CAE network, such as Figure 4 The pre-trained model includes multiple CAE model encoders 3E_1, 3E_2, 3E_p and multiple CAE model decoders 3D_1, 3D_2, 3D_p. The CAE model encoders 3E_1, 3E_2, 3E_p receive parameters 1, 2, and p respectively and generate outputs y1, y2, and y3 respectively. p ; CAE model decoders 3D_1, 3D_2, and 3D_p receive output y1, output y2, and output y respectively p And generate parameters 1', 2' and p' respectively. Using CAE algorithm, the characteristic value LS can be extracted i1 LS i2 ,…,LS ip , eigenvalue LSi1 LS i2 ,…,LS ip The output of the corresponding convolutional neural network model (i.e. output y1, output y2, ..., output y p ). In addition, the virtual measurement model 114 based on the convolutional autoencoder and the retained important samples includes multiple convolutional neural network models (such as CAE model encoders 3E_1, 3E_2, 3E_p) and an estimation model. The multiple inputs of these convolutional neural network models (i.e., parameter 1, parameter 2, ..., parameter p) are these paired data, and the multiple outputs of these convolutional neural network models (i.e., output y1, output y2, ..., output y p ) is the input of the estimation model. The convolutional neural network model typically includes a convolution layer, a pooling layer, a flattening layer, a dropout layer and a fully connected neural network, wherein the fully connected neural network includes at least one hidden layer and an output layer. The estimation model has the same structure as the fully connected neural network. The convolutional neural network model and the estimation model described above can refer to the prior U.S. patent No. US2023 / 0419107A1. That is, the embodiments of the present invention quote the relevant provisions of this prior U.S. patent (Incorporated by reference).

[0107] Step S06F includes dividing the paired data into multiple groups according to the normal distribution. Specifically, step S06F includes dividing the normal distribution into six groups according to a judgment condition. The six groups include a first group g1, a second group g2, a third group g3, a fourth group g4, a fifth group g5, and a sixth group g6. Figure 5 As shown. The first group g1 and the sixth group g6 belong to the extreme retention group, and the second group g2, the third group g3, the fourth group g4, and the fifth group g5 belong to the selected retention group. In other words, step S06F includes dividing the normal distribution into the extreme retention group and the selected retention group based on the judgment condition, and dividing the paired data into some of these paired data and another part of these paired data based on the extreme retention group and the selected retention group. The judgment condition (Criterion) meets the following formula:

[0108]

[0109] Wherein, p represents a parameter, which is greater than 1 and less than or equal to 2; q represents another parameter, which is greater than 0 and less than or equal to 1. In this embodiment, the parameters p and q are equal to 1.28 and 0.52 respectively, but the present invention is not limited thereto. i represents the i-th of these actual measured values; Represents yi The average value of (u); σ y Represents y i One standard deviation (σ), and σ y Conforms to the following formula:

[0110]

[0111] Where n represents y i For a sample size of , in a normal distribution, the paired data that meet the judgment criteria belong to the extreme retention group, and the paired data that do not meet the judgment criteria belong to the selected retention group. The judgment criteria for the first group g1, the second group g2, the third group g3, the fourth group g4, the fifth group g5, and the sixth group g6 correspond to equations (1), (2), (3), (4), (5), and (6), respectively. The probabilities of the first group g1, the second group g2, the third group g3, the fourth group g4, the fifth group g5, and the sixth group g6 are 10.00%, 20.12%, 19.88%, 19.88%, 20.12%, and 10.00%, respectively.

[0112] Step S06G involves determining whether the paired data belongs to the extreme holdout group. Specifically, step S06G involves determining whether each of the paired data belongs to the extreme holdout group (i.e., whether it belongs to the first group g1 and the sixth group g6) based on the six groups divided by the normal distribution, thereby generating a group determination result. If the group determination result is yes, step S06H is executed; if the group determination result is no, step S06K is executed.

[0113] Step S06H includes grouping and setting the threshold value of the cluster for model updating. Specifically, step S06H includes grouping the other part of these paired data (corresponding to Figure 4 The eigenvalue LS i1 LS i2 ,…,LS ip ) to generate multiple data groups (Cluster), and set a threshold value (Threshold value) of these data groups, and calculate the group center (Group center) and two proportion parameters (such as ). The threshold value is It is expressed and conforms to the following formula:

[0114]

[0115] in, represents an average value of a cosine distance between two samples in the k-th data set of the m-th group; represents a standard deviation of the cosine distance between the two samples in the k-th data set of the m-th group; represents the cosine distance between a vector of sample b and a vector of sample c in the kth data set of the mth group; α represents a threshold setting factor; samples b and c belong to q and are different from each other; q represents the number of samples in the mth group. In this embodiment, the clustering algorithm can be a K-means algorithm, but the present invention is not limited thereto.

[0116] Step S06I includes calculating the cosine distance between the sample and the cluster center and dividing the sample into multiple segments. Specifically, step S06I includes calculating the cosine distance between the sample and the cluster center (e.g. ) and two-scale parameters (such as ) one of these data sets (such as g 5,2 ) is divided into multiple sections (such as Section 1, Section 2, Section 3).

[0117] Step S06J includes retaining important samples from all segments of each group; in other words, step S06J includes selecting the most representative samples from each segment and retaining them as important samples, in order to retain the most data in each group. Figure 6 In the example, step S06J includes steps S06J1, S06J2, S06J3, S06J4, S06J5, and S06J6.

[0118] Step S06J1 includes calculating the cosine distance and allocating samples. Specifically, step S06J1 includes calculating a cosine distance between a sample in one of the data sets and the cluster center, and performing a sample allocation operation based on the cosine distance between the sample in one of the data sets and the cluster center. The sample allocation operation conforms to the following formula:

[0119]

[0120] in, Representative group center; represents the cosine distance between a sample in one of the data sets and the cluster center; and Represents these two proportional parameters; A vector representing the bth sample in the kth data set of the mth group; A vector representing the group center in the k-th data set of the m-th group; represents the bth sample in the kth data set of the mth group, which has a sample ID; S mks Represents the s-th segment in the k-th data group of the m-th group, where s is one of 1, 2, and 3.

[0121] Step S06J2 includes calculating S mks the cosine distance of the mth group. In detail, step S06J2 includes calculating a cosine distance between two samples in the s th segment (S mks ) of the k th data set of the mth group which satisfies the following equation:

[0122]

[0123] wherein, represents a vector of the e th sample of the s th segment of the k th data set of the mth group; represents a vector of the f th sample of the s th segment of the k th data set of the mth group, e and f being different from each other, the e th sample and the f th sample being the two samples

[0124] Step S06J3 includes confirming the number of samples of the s th segment S mks . In detail, step S06J3 includes confirming whether the number of segment samples in the s th segment (S mks ) of the k th data set of the mth group is greater than a preset segment sample number to generate a confirmation result, and then determining to perform a selected important sample operation (step S06J4) or an obtained important sample set operation (step S06J5) according to the confirmation result. When the confirmation result is yes, the selected important sample operation (step S06J4), the obtained important sample set operation (step S06J5), and a confirmed all sample operation (step S06J6) are sequentially performed; when the confirmation result is no, the obtained important sample set operation (step S06J5) and the confirmed all sample operation (step S06J6) are sequentially performed. In the present embodiment, the preset segment sample number can be 3, but the present application is not limited thereto.

[0125] Step S06J4 includes performing the selected important sample operation, and the selected important sample operation includes selecting three samples with the largest cosine distance in the s th segment (S mks ) of the k th data set of the mth group, and moving the three samples to an important sample set. Step S06J5 includes performing the obtained important sample set operation, and the obtained important sample set operation includes obtaining the important samples of the important sample set.

[0126] Step S06J6 includes performing the confirmed all sample operation, and the confirmed all sample operation includes confirming whether the inspection of all samples is completed. When the result of step S06J6 is yes, step S06 ends; when the result of step S06J6 is no, step S06J1 is re-executed. ​

[0127] Step S06K comprises inputting the important samples into the at least one pre-trained model to establish a convolutional autoencoder and important sample preserving based virtual metrology model 114, wherein the convolutional autoencoder and important sample preserving based virtual metrology model 114 comprises at least a convolutional neural network (CNN) model.

[0128] Please refer to Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7A and Figure 7B , wherein Figure 7A is a flowchart illustrating the computation operation (step S08) of the two-stage mechanism based on transfer learning and important sample preserving 116 of Figure 2 ; and Figure 7B is a flowchart illustrating the online KIS mechanism (step 417) of Figure 7A . As shown, in the computation operation, when the other actual measurement value is not obtained, the prediction stage operation 300 is performed to compute the first stage virtual measurement value (VM I ) of the other workpiece; otherwise, when the other actual measurement value is obtained, the transfer learning stage operation 400 is performed to compute the second stage virtual measurement value (VM II ) of the other workpiece.

[0129] In the prediction stage operation 300, first, step 302 is performed to collect the process data 102 of a workpiece sent out by a process device. Next, step 304 is performed to check whether the process data 102 of the workpiece is collected. If the result of step 304 is no, step 302 is continued; if the result of step 304 is yes, step 306 is performed to evaluate the DQI X of the process data 102. If the result of step 306 is abnormal, it means that the process data 102 is abnormal data, and a notification is issued (step 308); if the result of step 306 is normal, it means that the process data 102 is normal data, and step 310 is performed. Step 310 comprises performing data alignment operation on the process data 102 according to the automatic data alignment mechanism (ADAS) 108. Finally, step 312 is performed to input the other set of process data 102 of the other workpiece into the convolutional autoencoder and important sample preserving based virtual metrology model 114 to compute the first stage virtual measurement value (VM I ) of the other workpiece after performing data alignment operation on the other set of process data 102 of the other workpiece.

[0130] In the transfer learning phase operation 400, first, step 402 is performed to define a preset parameter value K, according to which the strategy selection can be periodically confirmed (i.e., the strategy selection confirmation step 422 is periodically executed), and the preset parameter value K is a positive integer. Next, step 404 is performed to set the initial value of a counting parameter value N to 0, and the counting parameter value N is an integer. Then, step 406 is performed to collect the actual measurement data 104 of a certain workpiece. Next, step 408 is performed to check whether the process data 102 of the same workpiece corresponding to this actual measurement data 104 exists, that is, to check the correlation between the measurement data 104 and the process data 102. Afterwards, step 410 is performed to determine whether the correlation check is successful. If the result of step 410 is no, then continue to step 406; if the result of step 410 is yes, then evaluate the DQI y Step 412 is performed to determine whether the actual measurement data 104 is normal. If the result of step 412 is abnormal, a notification is issued (step 414) and the process continues to step 406. If the result of step 412 is normal, the process proceeds to step 416 to confirm whether the production machine status has changed. If the result of step 416 is yes (the production machine status has changed), the process proceeds to step 404. If the result of step 416 is no (the production machine status has not changed), the process proceeds to step 417 to execute the online KIS mechanism.

[0131] Step 417 includes treating this other actual measurement value as a new sample, and calculating a cosine distance between the new sample and the group center in the kth data group of the mth group, and performing a sample allocation operation based on the cosine distance between the new sample and the group center in the kth data group of the mth group, and confirming whether the new sample becomes another important sample. In this way, the present invention uses a two-stage algorithm with an online KIS mechanism, which selects important samples to achieve real-time refresh of the model, thereby avoiding the problem of sample imbalance when new samples enter the system. Furthermore, since the virtual measurement system 100 based on important sample retention and convolutional neural network of the present invention is a time-varying system, if its corresponding two-stage algorithm has the ability to refresh the model, it can ensure good prediction accuracy. Figure 7B In the embodiment, step 417 includes steps 4171, 4172, 4173, 4174, 4175, and 4176.

[0132] Step 4171 includes assigning the new sample to the cluster to which it belongs. Specifically, step 4171 includes treating the other actual measurement value as a new sample and assigning the new sample to the cluster to which it belongs according to its actual measurement value. m (mth group), and extract the eigenvalue LS of the new sample through the CAE algorithm p , then the new sample is calculated based on the eigenvalue LS p Further assigned to the corresponding data group g m,k(kth data group of the mth cluster).

[0133] Step 4172 includes confirming whether the number of samples in the kth data group of the mth cluster (g m,k ) is less than a preset data group sample number. When the result of step 4172 is yes, step 4173 is performed; when the result of step 4172 is no, step 4174 is performed. In the present embodiment, the preset data group sample number can be 3, but the present application is not limited thereto.

[0134] Step 4173 includes calculating the threshold value Step 4174 includes assigning the new sample to the corresponding segment. In detail, step 4174 includes calculating the cosine distance between the new sample and the cluster center , and assigning the new sample to the corresponding segment (S mks ) according to the sample assignment operation of equation (12).

[0135] Step 4175 includes confirming whether the cosine distance of the most similar sample is less than the threshold value . In detail, step 4175 includes calculating the cosine distance between all old samples and the new sample in the s th segment (S mks ) of the kth data group of the mth cluster according to equation (13), and confirming whether it is less than the threshold value to find the most similar sample. When the result of step 4175 is yes (i.e., the cosine distance is less than the threshold value ), the corresponding similar sample replaces the old sample, and then step 4176 is performed; when the result of step 4175 is no (i.e., the cosine distance is greater than or equal to the threshold value ), the new sample is regarded as an important sample (i.e., the new sample is added to the important sample set), and then step 417 is ended.

[0136] Step 4176 includes confirming whether the most similar sample is the new sample, i.e., confirming whether the similar sample replacing the old sample is the new sample. If yes, step 406 (collecting actual measurement data 104) is re-performed; if no, step 417 is ended.

[0137] After that, step 418 is performed to set N = N + 1. Then, step 420 is performed to confirm whether the count parameter value N is equal to the preset parameter value K. If the result of step 420 is no, step 406 is continuously performed; if the result of step 420 is yes, the other actual measurement value of the other workpiece is subjected to a strategy selection confirmation step 422 to generate a confirmation result, and one of a first strategy step 424 (Strategy I) and a second strategy step 426 (Strategy II) is executed according to the confirmation result to update the virtual measurement model 114 based on convolutional autoencoder and important sample reservation. The strategy selection confirmation step 422 includes confirming whether the parts of the production machine are maintained or replaced. When the confirmation result is that the parts of the production machine are maintained or replaced, the first strategy step 424 is executed; otherwise, when the confirmation result is that the parts of the production machine are not maintained or replaced, the second strategy step 426 is executed.

[0138] The first strategy step 424 includes inputting the time series data of the plurality of parameters of the other set of process data 102 and the other actual measurement value of the other set of workpieces into the convolutional neural network models; and inputting the outputs of the convolutional neural network models into the estimation model to update the virtual measurement model 114 based on convolutional autoencoder and important sample reservation. In other words, both the convolutional neural network models and the estimation model are refreshed. In addition, the first strategy step 424 also includes retraining the RI model, the GSI model, the DQI X model, and the DQI y model. The so-called “retraining” is to retrain each model by using the updated historical process data 102 and historical measurement values.

[0139] The second strategy step 426 includes inputting the outputs of the convolutional neural network models into the estimation model to update part of the virtual measurement model 114 based on convolutional autoencoder and important sample reservation. In other words, the convolutional neural network models are not refreshed but are frozen, and only the estimation model is refreshed. In addition, the second strategy step 426 also includes calibrating the RI model, the GSI model, the DQI X model, and the DQI y model. Before performing the retraining step or the calibration step, the most recent set of process data 102 and actual measurement values obtained is used to replace the oldest data in the historical process data 102 and the historical measurement values. The so-called “calibration” is to adjust the weight values or parameter values of each model by using the updated historical process data 102 and historical measurement values. The execution time of the first strategy step 424 is less than that of the second strategy step 426.

[0140] After executing one of the first strategy step 424 and the second strategy step 426, the step 428 of updating the model is performed to replace the original set of virtual measurement models with the adjusted or retrained set of virtual measurement models, including the CNN model, RI model, GSI model, DQI model, and the like. X Model and DQI y These new virtual metrology models are also provided to steps 306, 312, and 412 to evaluate the quality (DQI) of the process data 102 of the next workpiece. X ), and calculate its first stage virtual measurement value (VM I ) and its accompanying confidence index (RI) and global similarity index (GSI); and evaluating the quality of the actual measurement data 104 of the next workpiece (DQI y Finally, step 430 is performed to input the other set of process data 102 of the other workpiece into the updated virtual measurement model 114 based on the convolutional autoencoder and the retained important samples to calculate the second stage virtual measurement value (VM) of the other workpiece. II ).

[0141] The technical details of the first strategy step 424 and the second strategy step 426 described above can be found in US Patent Application No. US2023 / 0419107A1. That is, the embodiments of the present invention incorporate by reference the relevant provisions of this US Patent Application.

[0142] Thus, the virtual measurement method 200 based on important sample retention and convolutional neural networks of the present invention utilizes a two-stage algorithm with an online KIS mechanism. This algorithm selects important samples to enable real-time model refresh, thus avoiding sample imbalance issues when new samples enter the system. Furthermore, in time-varying systems, the two-stage algorithm with an online KIS mechanism offers model refresh capabilities, improving model learning efficiency during fine-tuning and ensuring excellent prediction accuracy.

[0143] Please also refer to Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7A 、 Figure 7B and Figure 8 ,in Figure 8 This is a diagram showing the results of modeling and testing of different prediction algorithms in the application example of the present invention. This example is applied to the semiconductor dry etching process. Figure 8 In the figure, the horizontal axis is sampling and the vertical axis is thickness.Figure 3 After the offline model building operation (step S06), 64 paired data are retained and become important samples IS, while the unretained paired data are regarded as non-important samples NIS (Non-Important Samples). Curve L1 is the virtual measurement method based on convolutional neural network (Advanced AVM) CNN ) The thickness value prediction result obtained by the curve L2 is the virtual measurement method 200 (AdvancedAVM) based on important sample retention and convolutional neural network of the present invention CNN Table 1 shows the predicted results of thickness values ​​obtained with KIS). Figure 8 The prediction accuracy and R of thickness values ​​of different prediction algorithms 2 The prediction accuracy is expressed as Mean Absolute Error (MAE). Figure 8 As shown in Table 1, most of the non-important sample NIS are within ±1.28 standard deviations. Discarding these non-important sample NIS helps solve the imbalance problem of modeling samples, and using important sample IS can improve the accuracy of prediction. In addition, in terms of outliers, mean absolute error and R 2 The value improvement ratios (improvement rates) are 19.48%, 3.25% and 8.35% respectively, which shows the superiority of the architecture proposed in the present invention.

[0144] In addition, for the test set, the virtual measurement method 200 based on important sample retention and convolutional neural network of the present invention (Advanced AVM CNN The online model refresh operation with KIS makes up for the shortcomings of the old model in the conventional technology. It not only significantly enhances the overall model prediction ability, but also greatly improves the prediction trend in the dotted squares of the test set.

[0145] Table 1

[0146]

[0147] Please also refer to Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7A 、 Figure 7B 、 Figure 9A and Figure 9B ,in Figure 9A is a schematic diagram illustrating depth values ​​(Depth value) of different prediction algorithms when a state change occurs in a dry etching process according to an application example of the present invention; and Figure 9B This is an example of the application of the present invention.Figure 9A Figure 4 shows a plot of the cosine distance between sample No. 274 and other sample numbers. As shown, the first-stage virtual metrology values (VM I ) produced by the prediction stage operation 300 of the present application and the second-stage virtual metrology values (VM II ) produced by the transfer learning stage operation 400 of the present application correspond to a dry etching process applied to a semiconductor process. In Figure 9A Figure 4, the horizontal axis is the sampling and the vertical axis is the depth value; the curve L1 is the prediction result of the depth value obtained using only the virtual metrology method based on the convolutional neural network (Advanced AVM CNN ); the curve L2 is the prediction result of the depth value obtained using the virtual metrology method 200 based on the important sample reservation and the convolutional neural network (Advanced AVM CNN with KIS) of the present application. In Figure 9B Figure 4, the horizontal axis is the sampling and the vertical axis is the cosine distance. Table II shows the prediction accuracy of the depth value and the R Figure 9A value of different prediction algorithms in 2 Figure 4. From Figure 9A Figure 9B and Table II, it can be seen that the improvement ratios of the outlier, the mean absolute error and the R 2 value are 13.34%, 3.09% and 3.64%, respectively. Furthermore, since the virtual metrology method 200 based on the important sample reservation and the convolutional neural network (Advanced AVM CNN with KIS) of the present application simplifies the reservation of important samples IS and discards non-important samples NIS, the problem of data imbalance can be avoided, so that it has a superior performance in the outlier of sample No. 274.

[0148] Table II

[0149]

[0150]

[0151] ​It is to be understood that the virtual metrology method 200 based on important sample preservation and convolutional neural network of the present application is the implementation steps described above, and the computer program product of the present application is used to complete the virtual metrology method 200 based on important sample preservation and convolutional neural network as described above. The order of each implementation step described in the above embodiment can be adjusted, combined or omitted according to actual needs. The above embodiment can be realized by using a computer program product, which can include a machine-readable medium storing a plurality of instructions, and the instructions can program a computer to perform the steps in the above embodiment. The machine-readable medium can be, but is not limited to, a floppy disk, an optical disk, a read-only optical disk, a magneto-optical disk, a read-only memory, a random access memory, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an optical card or a magnetic card, a flash memory, or any machine-readable medium suitable for storing electronic instructions. Furthermore, the embodiment of the present application can also be downloaded as a computer program product, which can transfer the computer program product of the present application from a remote computer to a requesting computer by using a data signal through a communication connection such as a network connection.

[0152] It can also be noted that the present application can also be described in the context of a manufacturing system. Although the present application can be implemented in semiconductor manufacturing, the present application is not limited to semiconductor manufacturing and can also be applied to other manufacturing industries. The manufacturing system is configured to manufacture a workpiece or a product, which includes but is not limited to a microprocessor, a memory device, a digital signal processor, an application-specific circuit (ASIC) or other similar devices. The present application can also be applied to other workpieces or products other than semiconductor devices, such as vehicle wheels, screws. The manufacturing system includes one or more processing tools, which can be used to form one or more products or parts of a product on or in a workpiece (e.g., a wafer, a glass substrate). Those skilled in the art should know that the processing tool can be of any number and any type, including a lithography machine, a deposition machine, an etching machine, a grinding machine, an annealing machine, a tool machine and the like. In an embodiment, the manufacturing system also includes a scatterometer, an ellipsometer, a scanning electron microscope and the like.

[0153] From the above embodiments, the present application has the following advantages: first, the offline modeling is realized by using the convolutional autoencoder, the K-means algorithm and the KIS mechanism with cosine distance downsampling, important samples in each value range can be reserved to achieve balance, effectively solve the imbalance problem, and enhance the learning effect of the model during fine-tuning. Second, the two-stage algorithm with online KIS mechanism is used, which selects important samples to realize instant refreshing of the model, and can avoid the problem of sample imbalance when new samples enter the system. Furthermore, in the time-varying system, the two-stage algorithm with online KIS mechanism has the ability of model refreshing, which can improve the learning efficiency of the model during fine-tuning, and can ensure good prediction accuracy.

[0154] Although the present application has been disclosed as above with embodiments, it is not intended to limit the present application, and any person skilled in the art can make various modifications and decorations without departing from the spirit and scope of the present application, therefore the protection scope of the present application shall be subject to the appended claims.

Claims

1. A virtual metrology method based on important sample reservation and convolutional neural network, characterized in that, The method comprises the following steps: configuring a processor to obtain a plurality of sets of process data, wherein the plurality of sets of process data are used or generated when a production machine processes a plurality of workpieces, the plurality of sets of process data correspond to the plurality of workpieces in a one-to-one manner, each of the plurality of sets of process data comprises a plurality of parameter values, each of the plurality of parameter values corresponds to a plurality of sets of timing data of the plurality of workpieces respectively, and each of the plurality of sets of timing data has a data length; configuring the processor to perform a data alignment operation on the plurality of sets of process data, wherein the data alignment operation comprises: performing a data length adjustment operation to repeatedly add at least one data point having the value of a last data point of each of the plurality of sets of timing data of each of the plurality of parameters after the last data point until the data length of each of the plurality of sets of timing data of each of the plurality of parameters is equal to a longest data length of the plurality of sets of process data; configuring the processor to obtain a plurality of actual measurement values of the plurality of workpieces; configuring the processor to perform a modeling operation, wherein the modeling operation comprises: dividing the plurality of sets of process data and the plurality of actual measurement values into a plurality of paired data and at least one unpaired process data, each of the plurality of paired data comprises one of the plurality of sets of process data and one of the plurality of actual measurement values corresponding thereto; and using the at least one unpaired process data to establish at least one pre-trained model, performing a reserved important sample operation on the plurality of paired data to generate a plurality of important samples, and then inputting the plurality of important samples into the at least one pre-trained model to establish a convolutional autoencoder and reserved important sample-based virtual measurement model, wherein the convolutional autoencoder and reserved important sample-based virtual measurement model comprises at least one convolutional neural network model; and configuring the processor to perform a calculation operation, wherein the calculation operation comprises: obtaining another set of process data and at least one of another actual measurement value of another workpiece, and performing one of a prediction stage operation and a transfer learning stage operation according to whether the another actual measurement value is obtained to calculate one of a first stage virtual measurement value and a second stage virtual measurement value of the another workpiece; wherein the transfer learning stage operation comprises performing calculation according to the convolutional autoencoder and reserved important sample-based virtual measurement model, and the number of the plurality of important samples is less than the number of the plurality of paired data. The reserved important sample operation comprises: 2.The virtual metrology method based on importance sample reservation and convolutional neural network according to claim 1, wherein, judging each of the plurality of paired data to belong to one of an extreme reserved group and a selected reserved group according to a distribution of the plurality of paired data, and performing down-sampling on part of the plurality of paired data belonging to the selected reserved group to obtain a plurality of reserved data, wherein the plurality of important samples comprise another part of the plurality of paired data belonging to the extreme reserved group and the plurality of reserved data. The distribution of the plurality of paired data is a normal distribution, and the reserved important sample operation further comprises: 3.The virtual metrology method based on importance sample reservation and convolutional neural network according to claim 2, wherein, ​ The normal distribution is divided into the extreme reserved group and the selection reserved group according to a judgment condition, and the plurality of pairs of data is divided into the part of the plurality of pairs of data and the other part of the plurality of pairs of data according to the extreme reserved group and the selection reserved group. 4.The virtual metrology method based on importance sample reservation and convolutional neural network according to claim 3, wherein, The judgment condition satisfies the following equation: and where p represents a parameter greater than 1 and less than or equal to 2; y i represents an i-th one of the plurality of actual measurement values; represents an average value of y i ; σ y represents a standard deviation of y i , and σ y comply with the following equation and wherein n represents y i a number of samples, in the normal distribution, the part of the plurality of paired data meeting the judgment condition belongs to the extreme reservation group, and the other part of the plurality of paired data not meeting the judgment condition belongs to the selection reservation group. 5.The virtual metrology method based on important sample preservation and convolutional neural network according to claim 3, wherein, The reserved important sample operation further includes: According to a clustering algorithm, the other part of the plurality of pairs of data is clustered to generate a plurality of data groups, and a threshold value of the plurality of data groups is set, and a cluster center and two proportion parameters of each of the data groups are calculated according to the threshold value, wherein the threshold value is represented by and conforms to the following formula: ​ and wherein, an average value of a cosine distance between two samples in a kth data group of an mth cluster; a standard deviation of the cosine distance between the two samples in the kth data group of the mth cluster; a cosine distance between a vector of a sample b and a vector of a sample c in a kth data group of an mth cluster; a represents a threshold setting factor; samples b, c belong to q and are different from each other; q represents a sample number of the mth cluster.

6. The virtual metrology method based on importance sample reservation and convolutional neural network of claim 5, wherein, The reserved important sample operation further includes: One of the plurality of data groups is divided into a plurality of sections according to the group center and the two ratio parameters; and A cosine distance between a sample in the one of the plurality of data groups and the group center is calculated, and a distribution sample operation is performed according to the cosine distance between the sample in the one of the plurality of data groups and the group center, and the distribution sample operation satisfies the following equation: wherein, represents the centroid of the group; represents the cosine distance between the sample in the one of the plurality of data sets and the centroid of the group; and represents the two scaling parameters; represents a vector of the bth sample in the kth data set of the mth group; represents a vector of the centroid of the kth data set of the mth group; represents the bth sample in the kth data set of the mth group; S mks represents the st section in the kth data set of the mth group, s being one of 1, 2, 3.

7. The virtual metrology method based on importance sample reservation and convolutional neural network as claimed in claim 6, wherein, The reserved important sample operation further includes: A cosine distance between two samples in the s-th section in the k-th data group of the m-th group satisfies the following equation: wherein, a vector representing the e-th sample of the s-th segment in the k-th data group of the m-th cluster; a vector representing the f-th sample of the s-th segment in the k-th data group of the m-th cluster, e, f being different from each other.

8. The virtual metrology method based on importance sample reservation and convolutional neural network as claimed in claim 7, wherein, The reserved important sample operation further includes: A section sample number in the s-th section in the k-th data group of the m-th group is confirmed to be greater than a preset section sample number to generate a confirmation result, and then a selection important sample operation or an important sample set obtaining operation is determined to be performed according to the confirmation result; When the confirmation result is yes, the selection important sample operation, the important sample set obtaining operation, and a confirmation all sample operation are sequentially performed; when the confirmation result is no, the important sample set obtaining operation and the confirmation all sample operation are sequentially performed; The selection important sample operation includes selecting three samples with the largest cosine distance in the s-th section of the k-th data group of the m-th group, and moving the three samples to an important sample set. The important sample set obtaining operation includes obtaining the plurality of important samples of the important sample set. The confirmation all sample operation includes confirming whether the inspection of all samples is completed. 9.The virtual metrology method based on importance sample reservation and convolutional neural network according to claim 7, wherein, The transfer learning stage operation of the calculation operation includes: The other actual measurement value is regarded as a new sample, a cosine distance between the new sample and the group center in the k-th data group of the m-th group is calculated, the distribution sample operation is performed according to the cosine distance between the new sample and the group center in the k-th data group of the m-th group, and it is confirmed whether the new sample is another important sample.

10. The virtual measurement method based on important sample reservation and convolutional neural network according to claim 1, wherein The prediction stage operation includes calculating the first stage virtual measurement value from the other set of process data according to the virtual measurement model based on convolutional autoencoder and reserved important sample, and the transfer learning stage operation includes calculating the second stage virtual measurement value from the other set of process data and the other actual measurement value according to the virtual measurement model based on convolutional autoencoder and reserved important sample; The virtual measurement model based on convolutional autoencoder and reserved important sample is used to control the production machine to process the plurality of workpieces; and Either the first stage virtual measurement value output by the prediction stage operation or the second stage virtual measurement value output by the transfer learning stage operation is applied to a dry etching process of a semiconductor process for the production machine.

11. A virtual metrology system based on important sample reservation and convolutional neural networks, comprising: Comprising: a memory storing a plurality of sets of process data and a plurality of actual measurement values, wherein the plurality of sets of process data are used or generated when a plurality of workpieces are processed by a production machine, the plurality of sets of process data correspond to the plurality of workpieces in a one-to-one manner, each of the plurality of sets of process data comprises a plurality of values of parameters, each of the plurality of values of parameters corresponds to a plurality of sets of timing data of the plurality of workpieces, each of the plurality of sets of timing data has a data length; and a processor electrically connected to the memory, the processor receives the plurality of sets of process data and the plurality of actual measurement values and is configured to implement operations comprising: performing a data pairing operation on the plurality of sets of process data, the data pairing operation comprises: performing a data length adjustment operation to repeatedly add at least one data point having a value of a last data point of each of the plurality of sets of timing data of each of the plurality of parameters after the last data point until the data length of each of the plurality of sets of timing data of each of the plurality of parameters is equal to a longest data length of the plurality of sets of process data; performing a modeling operation, the modeling operation comprises: dividing the plurality of sets of process data and the plurality of actual measurement values into a plurality of paired data and at least one unpaired process data, each of the plurality of paired data comprises one of the plurality of sets of process data and one of the plurality of actual measurement values corresponding thereto; and using the at least one unpaired process data to establish at least one pre-trained model, performing a reserved important sample operation on the plurality of paired data to generate a plurality of important samples, and then inputting the plurality of important samples into the at least one pre-trained model to establish a convolutional autoencoder and reserved important sample-based virtual measurement model, wherein the convolutional autoencoder and reserved important sample-based virtual measurement model comprises at least one convolutional neural network model; and performing a calculation operation, the calculation operation comprises: obtaining another set of process data and at least one of another actual measurement value of another workpiece, and performing one of a prediction stage operation and a transfer learning stage operation according to whether the another actual measurement value is obtained to calculate one of a first stage virtual measurement value and a second stage virtual measurement value of the another workpiece; wherein the transfer learning stage operation comprises performing calculation according to the convolutional autoencoder and reserved important sample-based virtual measurement model, a number of the plurality of important samples is less than a number of the plurality of paired data. The reserved important sample operation comprises:

12. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 11, wherein, judging each of the plurality of paired data to belong to one of an extreme reserved group and a selected reserved group according to a distribution of the plurality of paired data, and performing down-sampling on part of the plurality of paired data belonging to the selected reserved group to obtain a plurality of reserved data, the plurality of important samples comprises another part of the plurality of paired data belonging to the extreme reserved group and the plurality of reserved data. The distribution of the plurality of paired data is a normal distribution, and the reserved important sample operation further comprises:

13. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 12, wherein, ​ The normal distribution is divided into the extreme reserved group and the selection reserved group according to a judgment condition, and the plurality of pairs of data is divided into the part of the plurality of pairs of data and the other part of the plurality of pairs of data according to the extreme reserved group and the selection reserved group.

14. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 13, wherein, The judgment condition satisfies the following equation: and where p represents a parameter greater than 1 and less than or equal to 2; y i represents an i-th one of the plurality of actual measurement values; represents an average value of y i ; σ y represents a standard deviation of y i , and σ y comply with the following equation and wherein n represents y i a number of samples, in the normal distribution, the part of the plurality of paired data meeting the judgment condition belongs to the extreme reservation group, and the other part of the plurality of paired data not meeting the judgment condition belongs to the selection reservation group.

15. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 13, wherein, The reserved important sample operation further includes: According to a clustering algorithm, the other part of the plurality of pairs of data is clustered to generate a plurality of data groups, and a threshold value of the plurality of data groups is set, and a cluster center and two proportion parameters of each of the data groups are calculated according to the threshold value, wherein the threshold value is represented by and conforms to the following formula: ​ and wherein, an average value of a cosine distance between two samples in a kth data group of an mth cluster; a standard deviation of the cosine distance between the two samples in the kth data group of the mth cluster; a cosine distance between a vector of a sample b and a vector of a sample c in a kth data group of an mth cluster; a represents a threshold setting factor; samples b, c belong to q and are different from each other; q represents a sample number of the mth cluster.

16. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 15, wherein, The reserved important sample operation further includes: One of the plurality of data groups is divided into a plurality of sections according to the group center and the two ratio parameters; and A cosine distance between a sample in the one of the plurality of data groups and the group center is calculated, and a distribution sample operation is performed according to the cosine distance between the sample in the one of the plurality of data groups and the group center, the distribution sample operation satisfying the following equation: wherein, represents the centroid of the cluster; represents the cosine distance between the sample in the one of the plurality of data sets and the centroid of the cluster; and represents the two scaling parameters; represents a vector of the bth sample in the kth data set of the mth cluster; represents a vector of the centroid of the kth data set of the mth cluster; represents the bth sample in the kth data set of the mth cluster; S mks represents the s th section in the kth data set of the mth cluster, s being one of 1, 2, 3.

17. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 16, wherein, The reserved important sample operation further includes: A cosine distance between two samples in the s-th section in the k-th data group of the m-th group satisfies the following equation: wherein, a vector representing the e-th sample of the s-th segment in the k-th data group of the m-th cluster; a vector representing the f-th sample of the s-th segment in the k-th data group of the m-th cluster, e, f being different from each other.

18. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 17, wherein, The reserved important sample operation further includes: Whether a section sample number in the s-th section in the k-th data group of the m-th group is greater than a preset section sample number is determined to generate a determination result, and then whether to perform a selection important sample operation or an important sample set obtaining operation is determined according to the determination result; When the determination result is yes, the selection important sample operation, the important sample set obtaining operation, and a confirmation all sample operation are sequentially performed; when the determination result is no, the important sample set obtaining operation and the confirmation all sample operation are sequentially performed; The selection important sample operation includes selecting three samples with the largest cosine distance in the s-th section of the k-th data group of the m-th group, and moving the three samples to an important sample set. The important sample set obtaining operation includes obtaining the plurality of important samples of the important sample set. The confirmation all sample operation includes confirming whether the inspection of all samples is completed.

19. The virtual metrology system based on importance sample reservation and convolutional neural network of claim 17, wherein, The transfer learning stage operation of the calculation operation includes: The other actual measurement value is regarded as a new sample, a cosine distance between the new sample and the group center in the k-th data group of the m-th group is calculated, the distribution sample operation is performed according to the cosine distance between the new sample and the group center in the k-th data group of the m-th group, and whether the new sample is another important sample is determined.

20. The virtual metrology system based on important sample reservation and convolutional neural network according to claim 11, wherein: The prediction stage operation includes calculating the first stage virtual measurement value from the other set of process data according to the virtual metrology model based on convolutional autoencoder and reservation of important samples, and the transfer learning stage operation includes calculating the second stage virtual measurement value from the other set of process data and the other actual measurement value according to the virtual metrology model based on convolutional autoencoder and reservation of important samples; The virtual metrology model based on convolutional autoencoder and reservation of important samples is used to control the production machine to process the plurality of workpieces; and Either the first stage virtual measurement value output by the prediction stage operation or the second stage virtual measurement value output by the transfer learning stage operation is applied to a dry etching process of a semiconductor process for the production machine.

Images