Prescription analysis in a highly collinear response space

Abstract

Methods, systems, and non-transitory computer-readable media for prescriptive analytics in a highly collinear response space are provided.SOLUTION: The method includes receiving film property data associated with manufacturing parameters of the manufacturing equipment and determining that the film property data is correlated and different from target data. The method further includes selecting a set of data points of the film property data that are orthogonal to the target data, performing feature extraction on the set of data points, and determining, based on the feature extraction, updates to the one or more manufacturing parameters to meet the target data.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001]

[0001] This disclosure relates to prescriptive analytics, and more particularly to prescriptive analytics in highly collinear response spaces. [Background technology]

[0002]

[0002] Manufacturing processes and equipment for producing products (such as in the semiconductor and display industries) can be complex. Determining updates to the parameters of the manufacturing processes and equipment to match target characteristics of the product can be time consuming and can depend on the expertise of the manufacturing facility managers. Summary of the Invention

[0003]

[0003] The following is a simplified summary of the present disclosure in order to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure, nor is it intended to delineate the scope of any particular implementation of the disclosure or the scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] In one aspect of the present disclosure, a method may include receiving film property data associated with a manufacturing parameter of a manufacturing equipment. The method further includes determining that the film property data is correlated and distinct from the target data. The method further includes selecting, by a processing device, a set of data points of the film property data that are orthogonal to the target data. The method further includes performing, by the processing device, feature extraction on the set of data points. The method further includes determining, based on the feature extraction, updates to one or more manufacturing parameters to match the target data.

[0005] In another aspect of the present disclosure, a system includes a memory and a processing unit coupled to the memory. The processing unit receives film property data associated with manufacturing parameters of a manufacturing equipment and determines that the film property data is correlated and different from target data. The processing unit further selects a set of data points of the film property data that are orthogonal to the target data and performs feature extraction on the set of data points. The processing unit further determines, based on the feature extraction, updates to one or more manufacturing parameters to match the target data.

[0006]

[0006] Another aspect of the present disclosure is a non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to receive film property data associated with production parameters of a manufacturing equipment and determine that the film property data is correlated and different from target data. The processor further selects a set of data points of the film property data that are orthogonal to the target data and performs feature extraction on the set of data points. The processor further determines, based on the feature extraction, updates to one or more production parameters to match the target data.

[0007]

[0007] The present disclosure is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary system architecture, according to certain embodiments. [Figure 2] 1 is an exemplary dataset generator for creating a dataset for a machine learning model, according to certain embodiments. [Figure 3] FIG. 10 is a block diagram illustrating determining updates to manufacturing parameters to match target data, according to certain embodiments. [Figure 4]FIG. 1 is a flow diagram illustrating an example method for determining updates to manufacturing parameters to match target data, according to certain embodiments. [Figure 5] FIG. 1 is a flow diagram illustrating an example method for determining updates to manufacturing parameters to match target data, according to certain embodiments. [Figure 6] FIG. 1 is a flow diagram illustrating an example method for determining updates to manufacturing parameters to match target data, according to certain embodiments. [Figure 7] 8A-8C are graphs illustrating determining updates to manufacturing parameters to match target data, according to certain embodiments. [Figure 8] FIG. 1 is a block diagram illustrating a computer system according to certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009]

[0014] Described herein are techniques directed to prescriptive analysis in highly collinear response spaces. A manufacturing device (e.g., a semiconductor or display processing tool) executes a manufacturing process to produce a product (e.g., a semiconductor wafer, a semiconductor display, etc.) having resulting characteristic data (e.g., film characteristic data). The resulting characteristic data may be compared to target data (e.g., target characteristic data, specifications). In response to the resulting characteristic data not matching the target data, manufacturing parameters (e.g., hardware parameters, process parameters) of the manufacturing device may be updated to match the target data. Traditionally, updating the manufacturing parameters to match the target data may depend on the expertise of an administrator, which may be ad hoc and limited. The administrator may use trial and error to determine how to update the manufacturing parameters to match the target data. The administrator may not be able to update the manufacturing parameters to match the target data. The target data may include two or more characteristics. For example, the target data may be one or more of a multi-parameter, multi-objective optimization problem, a constrained optimization problem, an unconstrained optimization problem, etc. Conventionally, an administrator may select a specific characteristic of the target data to match that characteristic, and then update the manufacturing parameters. When attempting to match one characteristic of the resulting characteristic data to the target data, other characteristics of the resulting characteristic data may not match the target data, and deviations from the target data may increase. Conventionally, the amount or types of manufacturing parameters that an administrator can update (e.g., consider updating simultaneously) may be limited. An administrator may not be able to update the manufacturing parameters to match target data that is one or more of multi-parameter, multi-objective optimization, constrained optimization problems, unconstrained optimization problems, etc.

[0010]

[0015] Product characteristic data may be correlated (e.g., collinear film characteristic data). For example, in a graph where a first axis is a first characteristic and a second axis is a second characteristic, the characteristic data may form a line. The line may be substantially parallel to (e.g., offset from) the target data. Traditionally, managers are unable to adjust manufacturing parameters so that the correlated characteristic data (e.g., collinear film characteristic data) matches the target data.

[0011]

[0016] The apparatus, systems, and methods disclosed herein use prescriptive analysis in a highly collinear response space to determine updates to one or more manufacturing parameters (e.g., process parameters, equipment parameters, hardware design changes, etc.) to match target data. A processing device receives film property data related to manufacturing parameters of a manufacturing device and determines that the film property data is correlated and different from the target data (e.g., does not match the target data, cannot match the target data, is offset from the target data). The processing device selects a set of data points of the film property data that are orthogonal to the target data and performs feature extraction on the set of data points. The processing device determines updates to one or more manufacturing parameters to match the target data based on the feature extraction.

[0012]

[0017] In some embodiments, a machine learning model can be trained using data inputs (e.g., historical or experimental manufacturing parameters) and target outputs (e.g., historical or experimental membrane property data) corresponding to the data inputs. An inverse solution can be obtained from the trained machine learning model based on the target data. The inverse solution can include updates to the manufacturing parameters. In some embodiments, to obtain the inverse solution, the trained machine learning model can be inverted, the target data can be input to the inverted trained machine learning model, and the inverted machine learning model can output updates to the manufacturing parameters to match the target data.

[0013]

[0018] In some embodiments, updates to the manufacturing parameters may be displayed via a graphical user interface. In some embodiments, updates to the manufacturing parameters may be performed to match target data.

[0014]

[0019] Aspects of the present disclosure provide technical advantages such as significant reductions in energy consumption (e.g., battery consumption), required bandwidth, processor overhead, etc. In some embodiments, a technical advantage results from determining updates to manufacturing parameters to match target data without performing ad hoc trial and error using an administrator's expertise. Updates to manufacturing parameters to match target data may be determined using less energy, less bandwidth, and less processor overhead than ad hoc trial and error relying on user expertise. Updates to manufacturing parameters determined via the present disclosure may produce products with characteristic data that are closer to the target data than traditional approaches. Traditionally, administrators may be unable to determine updates to manufacturing parameters to match the target data, potentially resulting in products that do not match specifications. Updates to manufacturing parameters determined by embodiments described herein may produce products with characteristic data that match different characteristics of the target data (e.g., match specifications rather than simply approximating one characteristic of the target data).

[0015]

[0020] 1 is a block diagram illustrating an exemplary system architecture 100 according to certain embodiments. System architecture 100 includes client device 120, manufacturing equipment 124, measurement equipment 126, prescriptive analysis server 130, and data store 140. Prescriptive analysis server 130 may be part of prescriptive analysis system 110. Prescriptive analysis system 110 may further include server machines 170 and 180.

[0016]

[0021] The metrology equipment 126 may include one or more of a metrology system 127 or a sensor 128. The metrology equipment 126 may determine film property data (e.g., historical or experimental film property data 144, film property data 150, tested film property data 156) of products (e.g., wafers) manufactured by the manufacturing equipment 124 (e.g., via the metrology equipment 127). The metrology equipment 126 may determine manufacturing parameters (e.g., historical or experimental manufacturing parameters 146, etc.) associated with the manufacturing equipment 124 (e.g., via the sensor 128).

[0017]

[0022] Client device 120, manufacturing equipment 124, measurement equipment 126, prescriptive analysis server 130, data store 140, server machine 170, and server machine 180 may be coupled to one another via network 160 to determine updates to manufacturing parameters 154 to match target data 152. In some embodiments, network 160 is a public network that provides client device 120 with access to prescriptive analysis server 130, data store 140, and other publicly available computing devices. In some embodiments, network 160 is a private network that provides client device 120 with access to prescriptive analysis server 130, data store 140, and other privately available computing devices. Network 160 may include one or more wide area networks (WANs), local area networks (LANs), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks or Wi-Fi networks), cellular networks (e.g., Long Term Evolution (LTE) networks), routers, hubs, switches, server computers, cloud computing networks, and / or combinations thereof.

[0018]

[0023] The client device 120 may include a computing device such as a personal computer (PC), laptop, mobile phone, smartphone, tablet computer, netbook computer, network-connected television ("smart TV"), network-connected media player (e.g., Blu-ray player), set-top box, over-the-top (OTT) streaming device, operator box, etc. The client device 120 may be capable of receiving film property data (e.g., historical or experimental data 142, film property data 150, tested film property data 156) from the measurement equipment 126 and receiving updates to the manufacturing parameters 154 from the prescriptive analysis system 110 or the like via the network 160. The client device 120 may be capable of transmitting film property data (e.g., historical or experimental data 142, film property data 150, target data 152, tested film property data 156) to the prescriptive analysis system 110 and transmitting updates to the manufacturing parameters 154 to the manufacturing equipment 124 or the like via the network 160. In some embodiments, the client devices 120 may change manufacturing parameters (e.g., process parameters, hardware parameters, etc.) of the manufacturing equipment 124 based on the updates to the manufacturing parameters 154. Each client device 120 may include an operating system that enables a user to one or more of generate, view, or edit data (e.g., target data 152, updates to the manufacturing parameters 154, tested film property data 156, etc.).

[0019]

[0024] The client device 120 may include a manufacturing parameter modification component 122. The manufacturing parameter modification component 122 may receive user input of target data 152 (e.g., via a graphical user interface displayed via the client device 120). The target data 152 may include property data (e.g., film property data). In some embodiments, the client device 120 transmits the target data 152 to the prescriptive analysis server 130, and the client device 120 receives updates to the manufacturing parameters 154 from the prescriptive analysis server 130 to match the target data 152. The client device 120 may update the manufacturing parameters of the manufacturing equipment 124 based on the updates to the manufacturing parameters 154 (e.g., transmit the updates to the manufacturing parameters 154 to the manufacturing equipment 124 and execute the updates to the manufacturing parameters 154). The client device 120 may receive tested film property data 156 in response to the manufacturing parameter updates being performed by the manufacturing equipment 124. The client device 120 may send the tested film property data 156 to the prescriptive analysis system 110 (e.g., the prescriptive analysis server 130) for updating the trained machine learning model 190.

[0020]

[0025] The prescriptive analytics server 130 may include one or more computing devices, such as a rack-mount server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a graphics processing unit (GPU), an accelerator application-specific integrated circuit (ASIC) (e.g., a tensor processing unit (TPU)), etc. The prescriptive analytics server 130 may include a prescriptive analytics component 132. In some embodiments, the prescriptive analytics component 132 may use historical or experimental data 142 to determine updates to the manufacturing parameters 154 to match the target data 152. The historical or experimental data 142 may include historical data, experimental data, or a combination thereof. The experimental data may include design of experiments (DOE) data. In some embodiments, the prescriptive analytics component 132 may use a trained machine learning model 190 to determine updates to the manufacturing parameters 154 to match the target data 152. The trained machine learning model 190 may learn key process and hardware parameters. The determination of updates to the manufacturing parameters 154 by the trained machine learning model 190 may include the prescription of optimal process conditions (e.g., process parameters) and / or space (e.g., hardware parameters) by the trained machine learning model 190.

[0021]

[0026] The prescriptive analysis component 132 may receive (e.g., retrieve from the data store 140) film property data 150 associated with production parameters of the production equipment 124, determine that the film property data 150 is correlated and different from the target data 152, and determine updates to the production parameters 154 to match the target data 152. In some embodiments, the prescriptive analysis component 132 determines updates to the production parameters 154 by selecting a set of data points of the film property data 150 that are orthogonal to the target data 152 and performing feature extraction on the set of data points, where the updates to the production parameters 154 are based on the feature extraction. In some embodiments, an inverse solution (e.g., updates to the production parameters) may be obtained from a machine learning model trained based on the target data. The inverse solution may include updates to the production parameters. In some embodiments, to obtain the inverse solution, the prescriptive analysis component 132 determines updates to the production parameters 154 by providing the target data 152 to the trained machine learning model. For example, the inverse solution (e.g., updates to the production parameters 154) may be obtained from a trained machine learning model based on the target data 152. In some embodiments, the prescriptive analysis component 132 determines updates to the manufacturing parameters 154 by providing the target data 152 to an inverted trained machine learning model (e.g., trained inverted model 190), obtaining output from the inverted trained machine learning model, and extracting updates to the manufacturing parameters 154 from the output. The inverted trained machine learning model can select a set of data points in the film property data 150 that are orthogonal to the target data 152 and perform feature extraction on the set of data points, and the output of the inverted trained machine learning model (e.g., updates to the manufacturing parameters 154) is based on the feature extraction.

[0022]

[0027] The data store 140 may be memory (e.g., random access memory), a drive (e.g., a hard drive, a flash drive), a database system, or another type of component or device capable of storing data. The data store 140 may include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). The data store 140 may store one or more of historical or experimental data 142, film property data 150, target data 152, updates to production parameters 154, or tested film property data 156. The historical or experimental data 142 may include historical or experimental film property data 144 and historical or experimental production parameters 146 over a period of time or across multiple runs of the manufacturing equipment 124. Each instance of the historical or experimental film property data 144 may correspond to a respective instance of the historical or experimental production parameters 146 (e.g., an instance of the historical or experimental production parameters 146 used by the manufacturing equipment 124 to manufacture a product having the historical or experimental film property data 144). Each instance of the tested film characteristic data 156 may correspond to a respective instance of an update of the manufacturing parameters 154 (e.g., an instance of an update of the manufacturing parameters 154 used by the manufacturing equipment 124 to manufacture a product having the tested film characteristic data 156).

[0023]

[0028] In some embodiments, the manufacturing parameters include one or more settings or components (e.g., size, type, etc.) of the manufacturing equipment 124. The manufacturing parameters may include one or more of temperature (e.g., heater temperature), spacing (SP), pressure, high frequency radio frequency (HFRF), electrostatic chuck (ESC) voltage, current, first precursor, first diluent, second diluent, first reactant, second reactant, second precursor, etc.

[0024]

[0029] In some embodiments, the film property data includes wafer-space film properties based on metrology tool data (e.g., obtained from metrology tool 126 coupled to fabrication tool 124). The film property data may include one or more of dielectric constant, dopant concentration, growth rate, density, etc.

[0025]

[0030] In some embodiments, the client device 120 can store the target data 152 and the tested film property data 156 in the data store 140, and the server for prescription analysis 130 can retrieve the target data 152 and the tested film property data 156 from the data store 140. In some embodiments, the server for prescription analysis 130 can store updates to the manufacturing parameters 154 in the data store 140, and the client device 120 can retrieve updates to the manufacturing parameters 154 from the data store 140.

[0026]

[0031] In some embodiments, prescriptive analysis system 110 further includes server machine 170 and server machine 180. Server machines 170 and 180 may be one or more computing devices (such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, etc.), a GPU, an ASIC (such as a TPU), a data store (such as a hard disk, a memory database, etc.), a network, a software component, or a hardware component.

[0027]

[0032] The server machine 170 includes a dataset generator 172 that can generate datasets (e.g., a set of data inputs and a set of target outputs) for training, validating, or testing the machine learning model 190. Some operations of the dataset generator 172 are described in detail below with respect to FIGS. 2 and 6. In some embodiments, the dataset generator 172 can divide the historical or experimental data 142 into a training set (e.g., 60 percent of the historical or experimental data 142), a validation set (e.g., 20 percent of the historical or experimental data 142), and a test set (e.g., 20 percent of the historical or experimental data 142). In some embodiments, the prescriptive analysis component 132 generates multiple sets of features. For example, a first set of features can be a first set of manufacturing parameters corresponding to each of the datasets (e.g., the training set, the validation set, and the test set), and a second set of features can be a second set of manufacturing parameters corresponding to each dataset (e.g., different from the first set of manufacturing parameters).

[0028]

[0033] Server machine 180 includes a training engine 182, a validation engine 184, a selection engine, and a test engine 186. The engines (e.g., training engine 182, validation engine 184, selection engine, and test engine 186) may refer to hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, processing unit, etc.), software (e.g., instructions executed on a processing unit, general-purpose computer system, or dedicated machine), firmware, microcode, or a combination thereof. Training engine 182 may be capable of training machine learning model 190 using one or more sets of features associated with a training set from dataset generator 172. Training engine 182 may generate multiple trained machine learning models 190, each corresponding to a distinct set of features in the training set. For example, a first trained machine learning model may be trained using all features (e.g., X1-X5), a second trained machine learning model may be trained using a first subset of features (e.g., X1, X2, X4), and a third trained machine learning model may be trained using a second subset of features (e.g., X1, X3, X4, and X5) that may partially overlap with the first subset of features.

[0029]

[0034] The validation engine 184 may validate the trained machine learning models 190 using a corresponding set of features of the validation set from the dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features of the training set may be validated using a first set of features of the validation set. The validation engine 184 may determine the accuracy of each of the trained machine learning models 190 based on the corresponding set of features of the validation set. The validation engine 184 may discard trained machine learning models 190 with an accuracy that does not match a threshold accuracy. In some embodiments, the selection engine 185 may select one or more trained machine learning models 190 with an accuracy that matches the threshold accuracy. In some embodiments, the selection engine 185 may select the trained machine learning model 190 with the highest accuracy of the trained machine learning models 190.

[0030]

[0035] Test engine 186 may test trained machine learning models 190 using a corresponding set of features of a test set from dataset generator 172. For example, a first trained machine learning model 190 trained using a first set of features of the training set may be tested using a first set of features of the test set. Test engine 186 may determine, based on the test set, the trained machine learning model 190 with the highest accuracy among all of the trained machine learning models.

[0031]

[0036] Machine learning model 190 may refer to a model artifact created by training engine 182 using a training set that includes data inputs and corresponding target outputs (correct answers for each training input). Patterns in the dataset that map the data inputs to the target outputs (correct answers) may be sought, and mappings that capture these patterns may be provided to machine learning model 190. Machine learning model 190 may use one or more of linear regression, random forests, neural networks (e.g., artificial neural networks), etc.

[0032]

[0037] The trained machine learning model 190 may be inverted (e.g., by the prescriptive analytics component 132, etc.). The prescriptive analytics component 132 may provide target data 152 (e.g., target film property data) as input to the inverted trained machine learning model 190 and may run the inverted trained machine learning model 190 on the input to obtain one or more outputs. As described in more detail below with respect to FIG. 5 , the prescriptive analytics component 132 may extract updates to the manufacturing parameters 154 to match the target data 152 from the output of the trained machine learning model 190 and extract confidence data from the output indicating a confidence level that one or more products manufactured using the updates to the manufacturing parameters 154 match the target data 152 (e.g., are within specifications). The prescriptive analytics component 132 may use the confidence data to determine whether to have the manufacturing equipment 124 implement the updates to the manufacturing parameters 154.

[0033]

[0038] The confidence data may include or indicate a confidence level of the product matching the target data 152 generated using the updates to the manufacturing parameters 154. In one example, the confidence level is a real number between 0 and 1, where 0 indicates no confidence in the product matching the target data 152 and 1 indicates absolute confidence in the product matching the target data 152.

[0034]

[0039] For purposes of illustration and not limitation, embodiments of the present disclosure describe training a machine learning model using historical or experimental data 142, inverting the trained machine learning model, and inputting target data 152 into the inverted, trained machine learning model to determine updates to the manufacturing parameters 154. In other implementations, updates to the manufacturing parameters 154 are determined using a heuristic or rule-based model (e.g., without using a trained machine learning model). The prescriptive analysis component 132 may monitor the historical or experimental data 142. All of the information described with respect to the data input 210 in FIG. 2 may be monitored or otherwise used in the heuristic or rule-based model.

[0035]

[0040] In some embodiments, fewer machines may provide the functionality of client device 120, prescriptive analysis server 130, server machine 170, and server machine 180. For example, in some embodiments, server machines 170 and 180 may be combined into a single machine, while in other embodiments, server machine 170, server machine 180, and prescriptive analysis server 130 may be combined into a single machine.

[0036]

[0041] In general, functions described in one embodiment as being performed by client device 120, prescriptive analysis server 130, server machine 170, and server machine 180 may also be performed by prescriptive analysis server 130 in other embodiments, as appropriate. Additionally, functions performed by a particular component may be performed by a different component or multiple components working together. For example, in some embodiments, prescriptive analysis server 130 may send updates to manufacturing parameters 154 to manufacturing equipment 124. In another example, client device 120 may determine updates to manufacturing parameters 154 based on output from an inverted trained machine learning model.

[0037]

[0042] Additionally, the functionality of a particular component may be performed by a different component or multiple components working together. One or more of prescriptive analysis server 130, server machine 170, or server machine 180 may be accessed as a service offered to other systems or devices via an appropriate application programming interface (API).

[0038]

[0043] In embodiments, a "user" may be represented as a single individual. However, other embodiments of the present disclosure encompass a "user" that is an entity controlled by multiple users and / or automated sources. For example, a set of individual users aggregated as a group of administrators may be considered a "user."

[0039]

[0044] Although embodiments of the present disclosure have been described with respect to updating manufacturing parameters 154 of manufacturing equipment 124 at a manufacturing facility (e.g., a semiconductor manufacturing facility) to match target data 152, the embodiments may also be applied to matching target data generally. The embodiments may also be applied to optimizing product characteristic data (e.g., collinear characteristic data).

[0040]

[0045] 2 illustrates an exemplary dataset generator 272 (e.g., dataset generator 172 of FIG. 1) for creating a dataset for a machine learning model (e.g., model 190 of FIG. 1) using historical or experimental data 242 (e.g., historical or experimental data 142 of FIG. 1), according to certain embodiments. System 200 of FIG. 2 illustrates dataset generator 272, data input 210, and target output 220.

[0041]

[0046] In some embodiments, the dataset generator 272 generates a dataset (e.g., a training set, a validation set, a test set) that includes one or more data inputs 210 (e.g., training inputs, validation inputs, test inputs) and one or more target outputs 220 corresponding to the data inputs 210. The dataset may also include mapping data that maps the data inputs 210 to the target outputs 220. The data inputs 210 may also be referred to as “features,” “attributes,” or “information.” In some embodiments, the dataset generator 272 may provide the dataset to the training engine 182, the validation engine 184, or the test engine 186, where the dataset is used to train, validate, or test the machine learning model 190. Some embodiments of generating a training set may be further described with respect to FIG. 6.

[0042]

[0047] In some embodiments, the data input 210 may include one or more sets of characteristics 212A of historical or experimental manufacturing parameters 246 (e.g., historical or experimental manufacturing parameters 146 of FIG. 1 ). Each instance of the historical or experimental manufacturing parameters 246 may include one or more of the process parameters 214 or hardware parameters 216. The target output 220 may include historical or experimental film property data 244 (e.g., historical or experimental film property data 144 of FIG. 1 ).

[0043]

[0048] In some embodiments, the dataset generator 272 may generate a first data input corresponding to a first set of features 212A for training, validating, or testing a first machine learning model, and the dataset generator 272 may generate a second data input corresponding to a second set of features 212B for training, validating, or testing a second machine learning model.

[0044]

[0049] In some embodiments, the data set generator 272 may discretize one or more of the data inputs 210 or the target outputs 220 (e.g., for use in a classification algorithm for a regression problem). Discretizing the data inputs 210 or the target outputs 220 may convert continuous values ​​of variables into discrete values. In some embodiments, the discrete values ​​of the data inputs 210 represent discrete manufacturing parameters for obtaining the target outputs 220.

[0045]

[0050] The data inputs 210 and target outputs 220 for training, validating, or testing a machine learning model may include information about a particular facility (e.g., a particular semiconductor manufacturing facility). For example, the historical or experimental manufacturing parameters 246 and the historical or experimental film property data 244 may be for the same manufacturing facility as the film property data 150, the target data 152, the manufacturing parameter 154 updates, and the tested film property data 156.

[0046]

[0051] In some embodiments, the information used to train the machine learning model may be from a particular type of manufacturing equipment 124 at a manufacturing facility having particular characteristics, allowing the trained machine learning model to determine an outcome for a particular group of manufacturing equipment 124 based on input for particular target data 152 associated with one or more components that share the characteristics of the particular group. In some embodiments, the information used to train the machine learning model may be about components from more than one manufacturing facility, allowing the trained machine learning model to determine an outcome for a component based on input from one manufacturing facility.

[0047]

[0052] In some embodiments, after generating the dataset and using the dataset to train, validate, or test the machine learning model 190, the machine learning model 190 can be further trained, validated, or tested (e.g., using updates to the manufacturing parameters 154 and the tested film property data 156 of FIG. 1 ), or adjusted (e.g., by adjusting weights associated with the input data of the machine learning model 190, such as connection weights in a neural network).

[0048]

[0053] 3 is a block diagram illustrating a system 300 for generating updates to manufacturing parameters 354 (e.g., updates to manufacturing parameters 154 of FIG. 1 ), according to certain embodiments. System 300 can be a feedback system for determining updates to manufacturing parameters 354 to match target data 352 (e.g., target data 152 of FIG. 1 ) based on historical or experimental data 342 (e.g., historical or experimental data 142 of FIG. 1 ).

[0049]

[0054] In block 310, the system 300 (e.g., the prescriptive analytics system 110 of FIG. 1 ) performs data partitioning (e.g., via the data set generator 172 of the server machine 170 of FIG. 1 ) of historical or experimental data 342 (e.g., the historical or experimental data 142 of FIG. 1 ) to generate a training set 302, a validation set 304, and a test set 306. For example, the training set may be 60% of the historical or experimental data 342, the validation set may be 20% of the historical or experimental data 342, and the validation set may be 20% of the historical or experimental data 342. The system 300 may generate multiple sets of features for each of the training set, validation set, and test set. For example, if the historical or experimental data 342 has 20 manufacturing parameters (e.g., process parameters, hardware parameters) and 100 runs for each manufacturing parameter, then the first set of features may be manufacturing parameters 1-10, the second set of features may be manufacturing parameters 11-20, the training set may be runs 1-60, the validation set may be runs 61-80, and the test set may be runs 81-100. In this example, the first set of features for the training set would be manufacturing parameters 1-10 for runs 1-60.

[0050]

[0055] At block 312, the system 300 performs model training (e.g., via training engine 182 of FIG. 1 ) using the training set 302. The system 300 may train multiple models using multiple sets of features from the training set 302 (e.g., a first set of features from the training set 302, a second set of features from the training set 302, etc.). For example, the system 300 may train machine learning models to generate a first trained machine learning model using the first set of features from the training set (e.g., manufacturing parameters 1-10 from runs 1-60) and generate a second trained machine learning model using the second set of features from the training set (e.g., manufacturing parameters 11-20 from runs 1-60). In some embodiments, the first trained machine learning model and the second trained machine learning model may be combined to generate a third trained machine learning model (e.g., which may itself be a better predictor than the first or second trained machine learning models). In some embodiments, the sets of features used to compare models may overlap (e.g., a first set of features are manufacturing parameters 1-15, and a second set of features are manufacturing parameters 5-20). In some embodiments, hundreds of models may be generated, including models with various permutations of features and combinations of models.

[0051]

[0056] At block 314, the system 300 performs model validation using the validation set 304 (e.g., via validation engine 184 of FIG. 1 ). The system 300 may validate each of the trained models using a corresponding set of features in the validation set 304. For example, the system 300 may validate a first trained machine learning model using a first set of features in the validation set (e.g., production parameters 1-10 for runs 61-80) and validate a second trained machine learning model using a second set of features in the validation set (e.g., production parameters 11-20 for runs 61-80). In some embodiments, the system 300 may validate hundreds of models (e.g., models with various permutations of features, combinations of models, etc.) generated at block 312. At block 314, the system 300 may determine the accuracy of each of the one or more trained models (e.g., via model validation) and determine whether one or more of the trained models have an accuracy that matches a threshold accuracy. In response to a determination that none of the trained models have an accuracy that matches the threshold accuracy, flow returns to block 312, where system 300 performs model training using a different set of features from the training set. In response to a determination that one or more of the trained models have an accuracy that matches the threshold accuracy, flow proceeds to block 316. System 300 may discard trained machine learning models that have an accuracy lower than the threshold accuracy (e.g., based on a validation set).

[0052]

[0057] In block 316, the system 300 performs model selection (e.g., via selection engine 315) to determine which of the one or more trained models that meet the threshold accuracy has the highest accuracy (e.g., model 308 selected based on the validation of block 314). In response to a determination that two or more of the trained models that meet the threshold accuracy have the same accuracy, flow can return to block 312, where the system 300 performs model training using a more accurate training set corresponding to the more accurate set of features to determine the trained model with the highest accuracy.

[0053]

[0058] At block 318, the system 300 performs model testing (e.g., via test engine 186 of FIG. 1 ) using the test set 306 to test the selected model 308. The system 300 may test the first trained machine learning model using the first set of features of the test set (e.g., manufacturing parameters 1-10 of runs 81-100) to determine whether the first trained machine learning model meets a threshold accuracy (e.g., based on the first set of features of the test set 306). In response to the accuracy of the selected model 308 not meeting the threshold accuracy (e.g., the selected model 308 overfits the training set 302 and / or the validation set 304 and is not applicable to other datasets, such as the test set 306), flow proceeds to block 312, where the system 300 performs model training (e.g., retraining) using a different training set corresponding to a different set of features (e.g., different manufacturing parameters). In response to a determination that the selected model 308 has an accuracy that meets the threshold accuracy based on the test set 306, flow proceeds to block 320. At least in block 312, the model may learn patterns in historical or experimental data 342 to make predictions, and in block 318, the system 300 may apply the model to the remaining data (e.g., the test set 306) to test the predictions.

[0054]

[0059] In block 320, the system 300 inverts the trained model (e.g., the selected model 308). For the trained model, predicted film property data can be extracted from the output of the trained model in response to input of manufacturing parameters. For the inverted trained model, updates to manufacturing parameters 354 can be extracted from the output of the inverted trained model in response to input of target data 352 (e.g., target film property data).

[0055]

[0060] In block 322, the system 300 receives target data 352 (e.g., target film property data, target data 152 in FIG. 1 ) using the inverted trained model (e.g., selected model 308) and extracts updates to manufacturing parameters 354 (e.g., updates to manufacturing parameters 154 in FIG. 1 ) from the output of the inverted trained model.

[0056]

[0061] In response to the manufacturing equipment 124 using the updates to the manufacturing parameters 354 to manufacture a product (e.g., a semiconductor, a wafer, etc.), the product may be tested (e.g., via the measurement equipment 126) to determine tested film property data 356 (e.g., the tested film property data 156 of FIG. 1 ). In response to receiving the tested film property data 356, the flow may proceed to block 312 (e.g., via a feedback loop), where the updates to the manufacturing parameters 354 and the tested film property data 356 are compared, and the trained model is updated via model training (e.g., model retraining).

[0057]

[0062] In some embodiments, one or more of steps 310-322 may be performed in various orders and / or with other steps not shown and described herein. In some embodiments, one or more of steps 310-322 may not be performed. For example, in some embodiments, one or more of data partitioning of block 310, model validation of block 314, model selection of block 316, or model testing of block 318 may not be performed.

[0058]

[0063] 4-6 are flow diagrams illustrating exemplary methods 400-600 related to determining a manufacturing parameter update (e.g., updating manufacturing parameter 154 of FIG. 1 ), according to certain embodiments. Methods 400-600 are performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, a processing unit, etc.), software (e.g., instructions executing on a processing unit, a general-purpose computer system, or a dedicated machine, etc.), firmware, microcode, or a combination thereof. In one embodiment, methods 400-600 may be performed, in part, by prescriptive analysis system 110. In some embodiments, methods 400-600 may be performed by prescriptive analysis server 130. In some embodiments, a non-transitory storage medium stores instructions that, when executed by a processing unit (e.g., of prescriptive analysis system 110), cause the processing unit to perform methods 400-600.

[0059]

[0064] For ease of explanation, methods 400-600 are shown and described as a series of steps. However, steps according to the present disclosure may occur in various orders and / or simultaneously, and concurrently with other steps not shown and described herein. Moreover, not all illustrated steps may be performed to implement methods 400-600 according to the disclosed subject matter. Furthermore, those skilled in the art will understand and appreciate that methods 400-600 may alternatively be represented as a series of interrelated states via a state diagram or events.

[0060]

[0065] FIG. 4 is a flow diagram of a method 400 for determining updates to manufacturing parameters (e.g., updates to manufacturing parameters 154 of FIG. 1) to match target data (e.g., target data 152 of FIG. 1) according to certain embodiments.

[0061]

[0066] 4 , at block 402, processing logic receives film property data 150 associated with manufacturing parameters of a manufacturing equipment 124. In some embodiments, the client device 120 receives the film property data 150 from a metrology device 126 (e.g., a metrology system 127). In some embodiments, the film property data 150 is measured via the metrology device 126 while the manufacturing equipment 124 is processing or producing a product (e.g., a semiconductor wafer). In some embodiments, the film property data 150 is measured via the metrology device 126 after the manufacturing equipment 124 has processed the product (e.g., a finished semiconductor wafer, a semiconductor wafer that has undergone a stage of processing). The film property data 150 may correspond to multiple types of measurements via multiple types of metrology devices 126. The prescriptive analysis component 132 may receive the film property data 150 from one or more of the client device 120, the metrology device 126, or the data store 140.

[0062]

[0067] At block 404, processing logic determines that the film property data 150 is correlated and different from (e.g., does not match) the target data 152. In some embodiments, processing logic determines that the film property data 150 does not intersect with (e.g., the film property data 150 is substantially parallel to) the target data 152.

[0063]

[0068] At block 406, processing logic selects a set of data points of the film property data 150 that are orthogonal to the target data 152 (see FIG. 7A).

[0064]

[0069] At block 408, processing logic performs feature extraction on the set of data points, which may be performed via one or more of principal component analysis (PCA), clustering (such as k-means clustering or hierarchical clustering), factor analysis (FA), discriminant analysis, or correlation matrix.

[0065]

[0070] At block 410, processing logic determines, based on the feature extraction, updates to one or more fabrication parameters (e.g., updates to fabrication parameters 154) to match target data 152. Processing logic may filter solutions based on the feasibility or cost of updating fabrication parameters 154 to match target data 152. For example, a first update or a second update of fabrication parameters may match the film property data to target data 152. The first update may include a change in process parameters (e.g., feasible, low cost), and the second update may include an update in hardware parameters (e.g., ordering new fabrication equipment, high cost). The first update may be used instead of the second update because it is more feasible and less costly.

[0066]

[0071] At block 412, processing logic displays one or more updates to the fabrication parameters 154 via a graphical user interface (GUI). The GUI may be displayed via the client device 120. The GUI may display two or more different options for updating the fabrication parameters 154 to match the film property data to the target data 152. The GUI may indicate the cost (e.g., cost increase, cost decrease, etc.) associated with the fabrication parameter 154 update (e.g., equipment update, processing cost, time required, etc.) compared to the current cost. The GUI may display selectable GUI elements for selecting and implementing the fabrication parameter 154 update.

[0067]

[0072] At block 414, processing logic implements updates to one or more manufacturing parameters of the manufacturing equipment 124 to match the target data 152. In some embodiments, block 414 is in response to receiving user input selecting a GUI element to implement updates to the manufacturing parameters 154. In some embodiments, receiving user input selecting a GUI element may update process parameters of the manufacturing equipment 124 (e.g., by the client device 120, by the prescriptive analysis component 132, etc.). In some embodiments, receiving user input selecting a GUI element may update hardware parameters of the manufacturing equipment 124 (e.g., by changing components of the manufacturing equipment 124, by changing settings of the manufacturing equipment 124, etc.).

[0068]

[0073] FIG. 5 is a flow diagram of a method 500 for determining updates to manufacturing parameters (e.g., updates to manufacturing parameters 154) using an inverted machine learning model to match target data 152, according to certain embodiments.

[0069]

[0074] Referring to FIG. 5, at block 502 , processing logic receives film property data 150 associated with manufacturing parameters of manufacturing equipment 124 .

[0070]

[0075] At block 504, processing logic determines that the film property data 150 is correlated and different from (e.g., does not match) the target data 152. The film property data 150 may be correlated between two or more variables (e.g., multi-dimensional correlation). In some embodiments, the desired output target (e.g., target data 152, such as film properties including one or more of refractive index (RI), stress, uniformity, etc.) is highly collinear (e.g., substantially parallel to the film property data 150) and does not appear to be achievable or plausible based on the existing experimental data set (e.g., historical or experimental data 142) used in learning / training. This may be due to one or more of the narrow process space explored, limitations in hardware design, suitable chemistries (reactants / precursors, etc.), etc.

[0071]

[0076] In some embodiments, at block 506, processing logic inverts the trained machine learning model to generate an inverted machine learning model. In some embodiments, at block 506, processing logic generates a cost function similar to the trained machine learning model, and processing logic generates an inverse solution to the cost function. As described in connection with FIGS. 1-3 , data input 210 (e.g., training data) may include historical or experimental data 142. Historical or experimental data 142 (e.g., historical or experimental film property data 144 and historical or experimental manufacturing parameters 146) may include process parameters, hardware parameters, curated paste Excel files, classical design experiment schemas, hardware component information, radio frequency (RF) time, coordinate measuring machine (CMM) data, infrared (IR) data, color array (COA) data, condition data, design of experiment (DOE) data, etc. The data input 210 may be used as a training set for a statistical and / or machine learning model (e.g., a group of statistical and / or machine learning algorithms) to learn an approximation function (G) of a transfer function (F) relating manufacturing parameters (S) (e.g., hardware and / or process parameters) to film properties (P).

[0072]

[0077] At block 506, an inverse operator may be used on the function G in a regression-based method. If a closed-form solution for G does not exist (e.g., neural networks, random forests, etc.), simulation and optimization methods may be employed. At block 508, processing logic provides the target data 152 to the (inverted) trained machine learning model. The machine learning model may be inverted or pseudo-inverted (e.g., an inverse solution of a cost function similar to the machine learning model). The (inverted) trained machine learning model may select a set of data points of film property data orthogonal to the target data and perform feature extraction on the set of data points (see blocks 406-408 in FIG. 4 ). In some embodiments, the (inverted) trained machine learning model may use orthonormal vector-based principal component analysis (PCA) to define directions for process space exploration or hardware modifications and design improvements.

[0073]

[0078] As shown in Figure 7A, there may be high collinearity between characteristics Y2 and Y1. As shown in graph 700B of Figure 7B, an orthogonal line is formed between the set (Y1) defined by the distance metric d (e.g., distance from the target data 152). 0 , Y2 0 ) constitutes the point (Y1 0 =D1, Y2 0 =D2).

[0074]

[0079] Then, {Y1 0 , Y2 0}, the predictor / input vectors {X o} can be identified. Feature extraction (e.g., PCA) can be performed on the identified desired predictors / input vectors {X o} and is performed on the vector space {X o Principal components that account for the variability of {e.g., 90% of the variability, most of the variability} can be identified.

[0075]

[0080] At block 510, processing logic obtains an inverse solution comprising one or more outputs from the (inverted) trained machine learning model. In some embodiments, the inverse solution is obtained by inverting the trained machine learning model. In some embodiments, the inverse solution is obtained by pseudo-inversion (e.g., in response to an inability to invert the trained machine learning model). Pseudo-inversion may include generating a cost function similar to the trained machine learning model (e.g., via a nonlinear optimization technique) and determining an inverse solution of the cost function to obtain one or more outputs.

[0076]

[0081] At block 512, processing logic extracts from the one or more outputs one or more updates to manufacturing parameters 154 (e.g., recipe, inverse prediction) to match target data 152. In some embodiments, processing logic extracts from the one or more outputs a confidence level that the product will produce using the one or more updates to manufacturing parameters 154 will match target data 152. The processing device may determine whether the confidence level matches a confidence level threshold. Depending on the confidence level matching the confidence level threshold, flow may proceed to one or more of blocks 514 or 516.

[0077]

[0082] At block 514, processing logic displays the updates to one or more manufacturing parameters via a graphical user interface (see block 412 of Figure 4).

[0078]

[0083] At block 516, processing logic implements updates to one or more manufacturing parameters of manufacturing equipment 124 to match target data 152 (see block 414 of FIG. 4).

[0079]

[0084] At block 518, processing logic receives tested film property data (e.g., tested film property data 156 of FIG. 1 ) associated with one or more updates to manufacturing parameters 154. Tested film property data 156 may represent actual film property data of a product manufactured using the updates to manufacturing parameters 154.

[0080]

[0085] At block 520, processing logic updates the trained machine learning model based on updates to one or more of the tested film property data 156 and the manufacturing parameters 154. In some embodiments, in response to the tested film property data 156 differing from the target data 152 (e.g., a product manufactured using updates to the manufacturing parameters 154 that do not match the target data 152), processing logic may update the trained machine learning model with the tested film property data 156 and the updates to the manufacturing parameters 154 (e.g., updating the historical or experimental data 142 by storing the tested film property data 156 in the historical or experimental film property data 144 and storing the updates to the manufacturing parameters 154 in the historical or experimental manufacturing parameters 146). Processing logic may update (e.g., retrain, revalidate, and / or retest) the trained machine learning model based on the updated historical or experimental data.

[0081]

[0086] 6 is a flow diagram of a method 600 for generating a dataset for a machine learning model to determine updates to manufacturing parameters to match target data, according to certain embodiments. The prescriptive analytics system 110 may use the method 600 according to embodiments of the present disclosure to at least one of train, validate, or test a machine learning model. In some embodiments, one or more steps of the method 600 may be performed by the dataset generator 172 of the server machine 170, as described with respect to FIGS. 1 and 2. Note that the components described with respect to FIGS. 1 and 2 may be used to illustrate aspects of FIG. 6.

[0082]

[0087] Referring to FIG. 6, in some embodiments, at block 602, processing logic implementing method 600 initializes a training set T to an empty set.

[0083]

[0088] At block 604, processing logic generates a first data input (e.g., a first training input, a first validation input) that includes historical or experimental production parameters (e.g., historical or experimental production parameters 146 of FIG. 1 , historical or experimental production parameters 246 of FIG. 2 ). In some embodiments, the first data input may include a first set of characteristics of the historical or experimental production parameters, and the second data input may include a second set of characteristics of the historical or experimental production parameters (e.g., as described with respect to FIG. 2 ). In some embodiments, the third data input may include updates to the production parameters (e.g., updates to production parameters 154 of FIG. 1 , updates to production parameters 354 of FIG. 3 ).

[0084]

[0089] At block 606, processing logic generates a first target output for one or more data inputs (e.g., a first data input), where the first target output provides a representation of the membrane property data (e.g., historical or experimental membrane property data 144 of FIG. 1, historical or experimental membrane property data 244 of FIG. 2, tested membrane property data 156 of FIG. 1, tested membrane property data 356 of FIG. 3).

[0085]

[0090] At block 608, processing logic optionally generates mapping data indicating an input / output mapping. The input / output mapping (or mapping data) may refer to a data input (e.g., one or more of the data inputs described herein), a target output for the data input (e.g., where the target output identifies film property data), and an association between the data input(s) and the target output.

[0086]

[0091] At block 610, processing logic adds the mapping data generated at block 610 to the data set T.

[0087]

[0092] At block 612, processing logic branches based on whether dataset T is sufficient for at least one of training, validation, or testing of machine learning model 190. If so, execution proceeds to block 614; if not, execution returns to block 604. Note that while in some embodiments, the sufficiency of dataset T may be determined simply based on the number of input / output mappings of the dataset, in some other implementations the sufficiency of dataset T may be determined based on one or more other criteria (e.g., a measure of diversity of the data examples, accuracy, etc.) in addition to or instead of the number of input / output mappings.

[0088]

[0093] At block 614, processing logic provides a dataset T for training, validating, or testing the machine learning model 190. In some embodiments, dataset T is a training set and is provided to training engine 182 of server machine 180 to perform training. In some embodiments, dataset T is a validation set and is provided to validation engine 184 of server machine 180 to perform validation. In some embodiments, dataset T is a test set and is provided to test engine 186 of server machine 180 to perform testing. For example, in the case of a neural network, input values ​​(e.g., numerical values ​​associated with data inputs 210) of a given input / output mapping are input to the neural network, and output values ​​(e.g., numerical values ​​associated with target outputs 220) of the input / output mapping are stored in output nodes of the neural network. The connection weights of the neural network are then adjusted according to a learning algorithm (e.g., backpropagation, etc.), and the procedure is repeated for other input / output mappings of dataset T. After block 614, the machine learning model (e.g., machine learning model 190) may be at least one of trained using training engine 182 of server machine 180, validated using validation engine 184 of server machine 180, or tested using test engine 186 of server machine 180. The trained machine learning model may be inverted and implemented by prescriptive analysis component 132 (of prescriptive analysis server 130) to determine updates to production parameters to match target data.

[0089]

[0094] 7A-B are graphs 700A-B illustrating the determination of updates to manufacturing parameters to match target data, according to certain embodiments.

[0090]

[0095] Graph 700A in FIG. 7A illustrates high collinearity between characteristics Y2 and Y1. Client device 120 may receive film property data from a measurement device 126 (e.g., metrology system 127) coupled to fabrication equipment 124. Client device 120 may receive first film property data corresponding to a first property from a first portion of measurement device 126A and second film property data corresponding to a second property from a second portion of measurement device 126B. In some embodiments, the first film property data and the second film property data are the same type of property measured at different locations on the semiconductor wafer (e.g., one measured on a first surface and the other measured on a second surface).

[0091]

[0096] The client device 120 may plot the first characteristic data and the second characteristic data on a graph 700A. Each point on the graph 700A may correspond to a value of a first characteristic (Y1) and a value of a second characteristic (Y2) measured at the same time for the same product (e.g., a semiconductor wafer). As shown in the graph 700A, the film characteristic data plotted for the first characteristic and the second characteristic may be correlated (e.g., linearly, collinearly, Y1 and Y2 may at least partially describe each other). The correlated film characteristic data may substantially form a film characteristic data line on the graph 700A. The correlated film characteristic data may be determined by measuring a coefficient of determination (R 2 or Rsq) (e.g., Rsq>0.8). The correlated membrane property data may meet a threshold Spearman's rank correlation coefficient. Spearman's rank correlation coefficient may be a nonparametric measure of rank correlation (e.g., statistical dependence between the ranks of a first property (Y1) and a second property (Y2)). The correlated membrane property data may meet a threshold P-value. The P-value is the probability that a given statistical model (a line fitting between Y1 and Y2) will produce a statistical summary (e.g., no linear dependence between Y1 and Y2) that exceeds or equals the actual observed results if the null hypothesis is true. In some embodiments, the target data 152 plotted for the first property (Y1) and the second property (Y2) may be substantially parallel to the membrane property data 150 plotted for Y1 and Y2.

[0092]

[0097] Graph 700B in FIG. 7B illustrates an orthogonal line passing through the points that make up the set defined by the distance metric. As shown in graph 700B in FIG. 7B, a line that is orthogonal (e.g., perpendicular, orthonormal) to the line formed by plotting the target data 152 may intersect the set of points of the film property data. One or more first points of the set of points may be closer to the plot of the target data 152 than one or more second points of the set of points. The one or more first points may correspond to a first manufacturing parameter, and the one or more second points may correspond to a second manufacturing parameter that is different from the first manufacturing parameter. Feature extraction may be performed on the set of data points to determine one or more manufacturing parameters that bring the first points of the set of points closer to the target data 152 than the second points. Based on the feature extraction, updates to one or more manufacturing parameters 154 may be determined to cause the set of points to match the target data 152.

[0093]

[0098] In some embodiments, selecting a set of data points orthogonal to the target data 152 and performing feature extraction on the set of data points is performed via an inverted trained machine learning model. In some embodiments, selecting a set of data points orthogonal to the target data 152 and performing feature extraction on the set of data points is performed via prescriptive analysis (e.g., statistical modeling, clustering, etc.) that does not use a machine learning model.

[0094]

[0099] 8 is a block diagram illustrating a computer system 800 according to certain embodiments. In some embodiments, computer system 800 may be connected to other computer systems (e.g., via a network such as a local area network (LAN), an intranet, an extranet, or the Internet). Computer system 800 may operate in the capacity of a server or a client computer in a client-server environment, or as a peer computer in a peer-to-peer or distributed network environment. Computer system 800 may be provided by a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, switch, or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be performed by that device. Furthermore, the term "computer" is intended to include any collection of computers that individually or collectively execute a set (or sets) of instructions to perform one or more of the methods described herein.

[0095]

[0100] In a further aspect, computer system 800 may include a processing unit 802, a volatile memory 804 (e.g., random access memory (RAM)), a non-volatile memory 806 (e.g., read-only memory (ROM) or electrically erasable programmable ROM (EEPROM)), and a data storage device 816, which may communicate with each other via a bus 808.

[0096]

[0101] The processing unit 802 may be provided by one or more processors, such as a general-purpose processor (e.g., a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, a microprocessor implementing another type of instruction set, or a microprocessor implementing a combination of instruction set types) or a specialized processor (e.g., an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), or a network processor).

[0097]

[0102] The computer system 800 may further include a network interface device 822. The computer system 800 may also include a video display unit 810 (e.g., an LCD), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and a signal generating device 820.

[0098]

[0103] In some implementations, the data storage device 816 may include a non-transitory computer-readable storage medium 824 that may store instructions 826 encoding any one or more of the methods or functions described herein, including instructions for encoding the prescription analysis component 132 or the manufacturing parameter modification component 122 of FIG. 1 and implementing the methods described herein.

[0099]

[0104] The instructions 826 may also reside, completely or partially, within the volatile memory 804 and / or within the processing unit 802 during its execution by the computer system 800; thus, the volatile memory 804 and the processing unit 802 may also constitute machine-readable storage media.

[0100]

[0105] Although the computer-readable storage medium 824 is shown as a single medium in the illustrated example, the term "computer-readable storage medium" is intended to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated cache, and / or associated caches and servers) that store one or more sets of executable instructions. The term "computer-readable storage medium" is also intended to include any tangible medium that can store or encode a set of instructions for execution by a computer that cause the computer to perform any one or more of the methods described herein. The term "computer-readable storage medium" is intended to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0101]

[0106] The methods, components, and features described herein may be implemented by discrete hardware components or may be integrated into the functionality of other hardware components, such as an ASIC, FPGA, DSP, or similar device. Furthermore, the methods, components, and features may be implemented by firmware modules or functional circuits within a hardware device. Furthermore, the methods, components, and features may be implemented in any combination of hardware devices and computer program components, or in a computer program.

[0102]

[0107] Unless otherwise specified, terms such as "receive," "determine," "select," "execute," "train," "generate," "provide," "invert," "obtain," "implement," "display," "optimize," "nonlinear optimization," and the like refer to actions and processes performed or implemented by a computer system that manipulate data represented as physical (electronic) quantities in the computer system registers and memory and convert it into other data similarly represented as physical quantities in the computer system's memory or registers or other such information storage, transmission, or display. Also, terms such as "first," "second," "third," "fourth," and the like, as used herein, are intended as labels to distinguish different elements and may not have an ordinal significance due to their numerical designations.

[0103]

[0108] The examples described herein also relate to apparatus for performing the methods described herein. This apparatus may be specially constructed to perform the methods described herein, or may include a general-purpose computer system selectively programmed by a computer program stored on the computer system. Such a computer program may be stored on a computer-readable tangible storage medium.

[0104]

[0109] The methods and illustrated examples described herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems can be used in accordance with the teachings described herein, or it may prove convenient to construct more specialized apparatus to perform the methods described herein and / or each of their individual functions, routines, subroutines, or steps. Examples of structures for these various systems are set forth in the description above.

[0105]

[0110] The above description is intended to be illustrative, not limiting. While the present disclosure has been described with reference to certain illustrated examples and implementations, it will be recognized that the present disclosure is not limited to the described examples and implementations. The scope of the present disclosure should be determined with reference to the following claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. 1. A method comprising: receiving film property data associated with manufacturing parameters of the manufacturing equipment; determining that the film property data is correlated and different from target data; selecting, by a processing unit, a set of data points of the film property data that are orthogonal to the target data; performing, by the processing unit, feature extraction on the set of data points; determining updates to one or more of the manufacturing parameters to match the target data based on the feature extraction; and A method comprising:

2. The method of claim 1 , wherein the manufacturing parameters include one or more of hardware parameters or process parameters.

3. The method of claim 1 , wherein determining that the film property data is correlated and different from the target data comprises determining that the film property data is substantially parallel to the target data.

4. The method of claim 1 , wherein the feature extraction is performed via one or more of principal component analysis (PCA), clustering, factor analysis (FA), discriminant analysis, or correlation matrix.

5. training a machine learning model to generate a trained machine learning model for determining the updates to the one or more manufacturing parameters, the training comprising: generating training data for the machine learning model, generating a first training input comprising historical production parameters of the manufacturing equipment; generating a first target output for the first training input, the first target output including historical membrane property data; generating training data for the machine learning model, providing the training data for training the machine learning model to (i) a set of training inputs including the first training input, and (ii) a set of target outputs including the first target output; Training machine learning models, including The method of claim 1 further comprising:

6. providing the target data to a trained machine learning model, wherein the selection of the set of data points and the performance of the feature extraction are performed via the trained machine learning model; obtaining an inverse solution comprising one or more outputs from the trained machine learning model, wherein the determining of the update comprises deriving the update of the one or more manufacturing parameters from the one or more outputs; The method of claim 1 further comprising:

7. The method of claim 1 , further comprising: performing the update of the one or more manufacturing parameters of the manufacturing equipment to match the target data.

8. The method of claim 1 , further comprising displaying the updates of the one or more manufacturing parameters via a graphical user interface.

9. 1. A system comprising: Memory and A processing device, receiving film property data associated with manufacturing parameters of the manufacturing equipment; determining that the film property data is correlated and different from target data; selecting a set of data points of the film property data that are orthogonal to the target data; performing feature extraction on the set of data points; determining updates to one or more of the manufacturing parameters to match the target data based on the feature extraction; a processing unit coupled to the memory for performing A system comprising:

10. 10. The system of claim 9, wherein to determine that the film property data is correlated and different from the target data, the processing unit determines that the film property data is substantially parallel to the target data.

11. the manufacturing parameters include one or more of hardware parameters or process parameters; the processing unit performs the feature extraction via one or more of principal component analysis (PCA), clustering, factor analysis (FA), discriminant analysis, or correlation matrix; The system of claim 9.

12. The processing device further trains the machine learning model to generate a trained machine learning model for determining the updates to the one or more manufacturing parameters, and to train the machine learning model, the processing device: generating training data for the machine learning model, generating a first training input comprising historical production parameters of the manufacturing equipment; generating a first target output for the first training input, the first target output including historical membrane property data; generating training data for the machine learning model, providing the training data for training the machine learning model to (i) a set of training inputs including the first training input, and (ii) a set of target outputs including the first target output; The system of claim 9 , further comprising:

13. The processing device further comprises: providing the target data to a trained machine learning model, wherein the processing unit selects the set of data points and performs the feature extraction through the trained machine learning model; obtaining an inverse solution including one or more outputs from the trained machine learning model, wherein the processing unit derives the updates to the one or more manufacturing parameters from the one or more outputs to determine the updates; and The system of claim 9 , further comprising:

14. The processing device further comprises: performing the update of the one or more manufacturing parameters of the manufacturing equipment to match the target data; or Displaying the updates of the one or more manufacturing parameters via a graphical user interface. The system of claim 9 , further comprising:

15. A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by a processing device, causing the processing device to: receiving film property data associated with manufacturing parameters of the manufacturing equipment; determining that the film property data is correlated and different from target data; selecting, by a processing unit, a set of data points of the film property data that are orthogonal to the target data; performing feature extraction on the set of data points; determining updates to one or more of the manufacturing parameters to match the target data based on the feature extraction; and A non-transitory computer-readable medium for causing

16. 16. The non-transitory computer-readable medium of claim 15, wherein to determine that the film property data is correlated and different from the target data, the processing unit determines that the film property data is substantially parallel to the target data.

17. the manufacturing parameters include one or more of hardware parameters or process parameters; the processing unit performs the feature extraction via one or more of principal component analysis (PCA), clustering, factor analysis (FA), discriminant analysis, or correlation matrix; 16. The non-transitory computer-readable medium of claim 15.

18. The processing device further trains the machine learning model to generate a trained machine learning model for determining the updates to the one or more manufacturing parameters, and to train the machine learning model, the processing device: generating training data for the machine learning model, generating a first training input comprising historical production parameters of the manufacturing equipment; generating a first target output for the first training input, the first target output including historical membrane property data; generating training data for the machine learning model, providing the training data for training the machine learning model to (i) a set of training inputs including the first training input, and (ii) a set of target outputs including the first target output; 16. The non-transitory computer-readable medium of claim 15,

19. The processing device further comprises: providing the target data to a trained machine learning model, wherein the processing unit selects the set of data points and performs the feature extraction through the trained machine learning model; obtaining an inverse solution including one or more outputs from the trained machine learning model, wherein the processing unit derives the updates to the one or more manufacturing parameters from the one or more outputs to determine the updates; and 16. The non-transitory computer-readable medium of claim 15,

20. The processing device further comprises: performing the update of the one or more manufacturing parameters of the manufacturing equipment to match the target data; or Displaying the updates of the one or more manufacturing parameters via a graphical user interface. perform one or more of the following:

16. The non-transitory computer-readable medium of claim 15.

Images