Scaling shared-data-induced predictive quality and manufacturing
Patent Information
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- MERCK PATENT GMBH
- Filing Date
- 2024-07-08
- Publication Date
- 2026-05-20
AI Technical Summary
Existing methods for scalable shared-data-included production process improvements in the semiconductor industry are inefficient and resource-intensive, requiring robust, reproducible, and efficient integration of data merging, machine learning model training, and result visualization across companies.
A method utilizing a cloud platform to acquire and merge data from multiple sources, deploy internal exploratory analyses, train and validate machine learning models, and provide post-processing results through a Frontend Graphical User Interface, specifically using Foundry-based data and analytics platforms with proprietary Python libraries and Streamlit for intuitive result representation.
This approach significantly accelerates production process improvements by reducing timelines from 6-8 months to 3-6 weeks, enabling efficient data-driven decision-making and scalable implementation across multiple products and companies in the semiconductor industry.
Smart Images
Figure EP2024069209_16012025_PF_FP_ABST
Abstract
Description
Scaling Shared-Data-lnduced Predictive Quality and ManufacturingBACKGROUND OF THE INVENTION
[0001] The disclosed invention relates to a method and a system for ensuring scalable shared-data-included production process improvements.
[0002] The invention belongs to the technical fields process automation.
[0003] The state of the art is the utilization of shared data, e.g. between chipmakers and semiconductor material suppliers, to derive insights for production process improvements. An example of such a utilization has been described in the international patent application WO2022195001 about a Shared Data Induced Production Process Improvement.
[0004] This approach outlines the general process flow as well as exemplary drawings for multivariate analyses that are applied to identify process improvements.
[0005] A scalable implementation of this state-of-the-art requires the integration of the process with technical implementation. This refers especially to the part of the process that starts after the completion of data readiness, meaning process mapping, data mapping, data collection, and aggregation per party, and therefore explicitly includes the scalable merging of data between companies, deploying internal exploratory analyses, training machine learning models for selected target parameters, optimizing / tuning hyperparameters of the respective models, validating respective models, and analyzing, post-processing, and visualizing the results in a format that is meaningful to data scientists as well as subject matter experts.
[0006] Realizing shared-data-included production process improvements at scale therefore requires the development of a clean technical process that is robust, reproducible, and efficient in terms of human resource needs and implementation.
[0007] To comply with and enhance such state of the art it would therefore be desirable to find a new approach to provide a method and a system forimproving the efficiency of shared-data-included industrial production processes in regard to a scalable implementation.BRIEF SUMMARY OF THE INVENTION
[0008] This task can be solved by a method for ensuring scalable shared- data-included production process improvements via at least one Computer comprising the following steps of acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results.
[0009] Advantageous and therefore preferred further developments of this invention emerge from the associated sub claims and from the description and the associated drawings.
[0010] Preferred further developments of the method include for example but are not limited to that:• acquiring data about the production process and its relevant process parameters from at least two different sources includes the merging of data sets from at least two different companies.• the at least one user comprises of at least one data science and / or subject matter expert.• as at least one Computer a cloud platform is used.
[0011] In the following further preferred developments of the method are described a bit more in detail. The invented method is a direct mapping of data-based solutions to each step of the technical process outlined in the problem section and their scalable implementation on a cloud platform. The method can be divided in a further preferred development into a technical backbone and a front-end application.
[0012] The Backbone Solution then covers all the described method steps including the merging of the data sets between companies, deploying internal exploratory analyses, training machine learning models for selected target parameters, optimizing / tuning hyperparameters of the respective models, validating respective models, and analyzing and post-processing the results for downstream use in the Front-end Solution.
[0013] This Backbone Solution is preferably implemented Foundry-based data and analytics platform. This implementation then uses various Foundrynative applications and is packaged in a ‘Foundry Templates’ module. The approach of using templates for proliferating pipelines itself is not the core of the invention. This invention itself concentrates more on the hereby described custom implementation and integration into the technical process for production process improvements.
[0014] Preferred functionalities of the inventive Foundry Template I Backbone Solution include the following:1 . Merging of data between parties• Taking assumptions on join keys• Including context of X parameters and y (target) parameters• Merging data sets on desired granularity, e.g., material supplier batch ID or semiconductor device marker’s wafter lot ID2. Machine learning pipeline from training over optimization to validation. The machine room is based on the proprietary python library that we developed for all related machine learning steps.• Descriptive modelling including Principal Component Analysis (PCA)• Training of machine learning models for all target parameters of choice, currently the following three: Partial least squares (PLS), Random Forest (RF), XGBoost; the models were selected based on their different strengths and constitute a balanced set so that at least one model has a good chance of performing well on different data sets (e.g., small number of rows, very complex behavior)• Optimization and hyperparameter tuning of all machine learning models, including the application of Bayesian hyperparameter optimization going beyond the standard methods of grid- and random-search• Validation of all machine learning models with K-fold cross-validation• Analysis of model results based on metrics that were decided with subject matter experts and data science experts:Sample sizeError metrics for the test set: mean and standard deviation of the coefficient of determination (R2), mean and standard deviation of the mean squared error (MSE), root-mean-square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE)Error metrics for the training set:Mean R2,Mean MSE, RMSE, MAE, MAPEResiduals on cross-validation (test set) and training setTime series of ground truth and predicted values for the target parametersPermutation feature importance (weighted to sum up to the R2for increased interpretability and comparability between models)SHAP valuesDevelopment of objective function scores on training and cross-validation set over dataset size (learning curve)MSE over hyper-optimization trials3. Postprocessing of results• Creation of a folder structure for output files:Raw preprocessing descriptive modeling, contains principal component analysis (PCA) for both parties predictive modeling• Generation of output files into respective folder structures, containing the contents of point 2. (ML pipeline)• Preparation of consolidated data sets collecting e.g., model metrics, feature importances across target parameters and modeling cases, to constitue the input to the front-end solution. This includes the aggregation of feature importances across model for each target parameter, weighted by the model performance, e.g., feature importances of models with R2below 0.4 don’t contribute at all, and models with feature importances above 0.7 contribute the most, according to the ranking above.
[0015] The Frontend Solution is preferably tailored to the results of the Backend Pipeline and creates an interactive, intuitive, and efficient representation of the results to be investigated with data science and subject matter experts. The Frontend Solution is also suitable for business users without hands-on coding experience and is essential in democratizing such a data-driven approach in a semiconductor business context.
[0016] The Front-end Solution could be implemented using the publicly available “streamlit” library for python and this implementation is beyond thestate-of-the-art in terms of the capabilities it offers for democratizing the previously mentioned results and implementation within a platform-based Foundry system.
[0017] The core of the Front-end Solution contains multiple pages, that respond to different user needs, which could be implemented respectively as follows:• Homepage page• The goal of the homepage is to get a quick overview of model performances to find entry points to dive into feature importances for well-performing models• This page contains three elements: the models overview, parameter overview, and model per target■ Models overview shows the R2coefficient for all target parameters (rows) and trained models (columns) in a heatmap, where the value per call that is also represented via color depicts the R2. This approach provides expert an intuitive and quick overview into model performances for further deep dives. We assign R2values below 0.4 as unacceptable, between 0.4 and 0.5 as acceptable at higher risk, between 0.5 and 0.7 as OK for further investigation, and above 0.7 as good moel.■ Parameter overview shows the different parameter types clustered across the value chain (n = device maker, n-1 = material suppler, n-2 = sub supplier, etc.) as well as split by variable type as defined by the party (for the inventors’ material parmeters this is split by labels, raw materials QC, in-process QC, finished goods QC, manufacturing process, storage, packaging, and miscellaneous)■ Model per target allows a filtered investigation with the same functionality as the heatmap, but the parameter ofchoice can be selected from an automatically fed dropdown menu for better readabilityFeature importances page• This page will be selected to deep dive into the models of interested that can be derived from the homepage. The goal of this page is to identify features of interest (regarding their impact for respective target parameters) that can be further investigated in following pages• It contains a drop-down to select the parameter(s) of choice, a slider to select how many top features of importances are to be investigated (normally only the top few are relevant, standard sliders allows to select from 1 to 10), as well as two visualizations: the pareto chart of feature importances and the correlations among top parameters■ Feature importances show the adjusted feature importance per top parameter (as per slider selection). The feature importances are only interpretable in relation to one another and are normalized so that the sum of all feature importances sums up to the total R2.■ Correlation among top parameters shows the 1 -1 correlations of all parameters according to the slider seting. It is an industry-standard correlation matrix including a color-coded visualization for easier interpretability.Scatterplots page• The goal of this page is to investigate the features of interest in terms of the target parameter’s value in dependence of the feature’s value as well as the development over time for both. This is a view which helps subject matter experts develop scientific hypotheses as leads for process improvements.• This page contains a drop-down menu to select the target parameter(s) of choice, a slider to select N representing the top number of features to be considered, as well as N analysis sections, each showing a scatterplot and a timeseries■ The scatterplot shows the target parameter (y-axis) in relation to the values of the feature of interest (x-axis). This allows for quick insights such as visual inspection of trends and the availability of distinct unique values for the feature of interest, which are both essential to derive downstream process improvement ideas with subject matter experts.■ The timeseries plot shows a line-plot for the target parameter and feature of interest over time. This view is also required for visual inspection and helps to understand trends that may go beyond the relationship of the two parameters such as a general trend in the underlying data (e.g., product improvements for a new introduction and optimization)• Target parameters page• This page offers a quick overview for the development of the desired target parameters over time. It is a comparable functionality to the timeseries of point 3 (scatterplots), but makes it more convenient for business end-users to plot multiple target parameters at the same time as well as their standard deviation.
[0018] It is understood that all features of those preferred further developments can be combined together, if not stated otherwise explicitly or obviously being impossible due to the nature of the respective features.
[0019] A further component of the claimed invention is a system for ensuring scalable shared-data-included production process improvements comprisingof at least one Computer configured to perform the following method steps of acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results.
[0020] Another component of the disclosed invention is a Computer program comprising instructions which cause the involved computers to carry out the following method steps of acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results. The program parts responsible for the single method steps are running on the respective computer parts. How the program itself is partitioned depends on the computer hardware being involved. It is possible to use a main software running on one of the mentioned computers or a separate computer which controls local client programs. Other options include equal instances of the software who communicate with each other and so on.
[0021] Only requirement for this computer program to perform the whole method as described is, that the used program and its respective hardware components are able to perform the method completely and automatically. Such a program can then be stored on a Computer-readable storage medium and / or data carrier signal which cause the involved computers to carry out the method steps of acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results. The storage medium can be stored on any suitable digital memory like an usb drive, a harddisk, a flashdrive and so on. From that memory it can also be provided via remota communication means using respective data carrier signals, like ethernet, wired or wireless, or any other suitable network transmission means, for transmitting the software to its target hardware.BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS
[0022] Fig. 1 : Version overview of template for Backend Pipeline
[0023] Fig. 2: Template deployment settings
[0024] Fig. 3: Parameter setting for template deployment
[0025] Fig. 4: Frontend Homepage
[0026] Fig. 5: Front-end Feature Importances
[0027] Fig. 6: Scatterplots for target parameters
[0028] Fig. 7: Target parameters over timeDETAILED DESCRIPTION OF THE INVENTION
[0029] The invention will be explained in more detail by presenting a real working example which is constituted by the inventors’ work on a Palantir- Foundry-based platform, where all the full solution as described above was implemented and is fully operational.
[0030] Backend Pipeline:
[0031] The starting point for the deployment of the Backend Pipeline is the template element that was developed on the Foundry-based platform. It utilized the native versioning functionality for robust iterative development and the template can be deployed by selecting the desired version and clicking “deploy”.
[0032] Fig. 1 shows therefore a version overview of the template for the Backend Pipeline
[0033] The template is located in a private namespace and deployed in a directory of choice, as shown in Fig. 2.
[0034] The template deployment consists now of three steps, as visible in Fig. 3. In a first step, inputs can be mapped to the template in terms of data files. This is not required for the standard pipeline of this invention, as the standard pipeline automatically identified the correct files based on the naming convention of folder structures. It can however be used if further information shall be included, e.g., additional less automated inputs from Fusion Sheets.
[0035] The second step, which is shown in Fig. 3, allows for parameters to be set, including the data sets, health checks, and transforms. It also provides an overview of artifacts that will be created. No changes are required for the standard deployment of this invention, which makes the realization very efficient. The health checks ensure the robustness of downstream results, as it maintains data integrity and consistency, e.g., checking the number of columns, nulls, etc..
[0036] Once the template is deployed, the results can be directly read out by the consecutive Front-end Solution.
[0037] Frontend Solution:
[0038] The Frontend Solution is implemented as a streamlit application. In this best practice / working example, the application is implemented in Code Workspaces of a platform Foundry. The Frontend Solution is not part of a Foundry Template.
[0039] The contents of Fig. 4 depict real best practice results for the invention described in the previous section.
[0040] This exemplary heatmap of Fig. 4 shows good results (R2>0.7) for the third-row element. Results will be further explored in the next paragraph.
[0041] The Feature Importances, as shown in Fig. 5 with three selected features, are visualized as well as internal correlations. One feature stands out with a very high importance for the selected target parameter.
[0042] The feature of interest can be further investigated with the scatterplot of the target parameter depending on the feature as well as the development of both over time. Fig. 6 shows an example for the scatterplot of the target parameter and its development over time compared to the development of the respective feature of interest. The development of desired target parameters over time is then visualized in Fig. 7.
[0043] With this best practice implementation, a massive efficiency gain was realized. The general data-included approach to quality issue resolutions and production process improvements accelerated timelines from 6-8 months to 3- 6 weeks. This approach was able to test one to a few scientific hypotheses in the latter timeframe. With the approach depicted in this invention report, users were able to train more than 200 models within 3 weeks. With that, by using the invention described in this report, we are moving from the, already accelerated timeline of 3-6 weeks, to 1-3 weeks according to our working experience. The entire pipeline is scalable and is quintessential for thesemiconductor industry to scale data-induced process improvements across more products and companies.
Claims
CLAIMS1. A method for ensuring scalable shared-data-included production process improvements via at least one Computer comprising the following steps: acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results.
2. The method of claim 1 , wherein acquiring data about the production process and its relevant process parameters from at least two different sources includes the merging of data sets from at least two different companies.
3. The method of claim 1 , wherein the at least one user comprises of at least one data science and / or subject matter expert.
4. The method of claim 1 , wherein as at least one Computer a cloud platform is used.
5. A System for ensuring scalable shared-data-included production process improvements comprising of at least one Computer configured to perform the following method steps: acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results.
6. A Computer program comprising instructions which cause the involved at least one Computer to carry out the following method steps: acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer;training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results.
7. A Computer-readable storage medium and / or data carrier signal having stored thereon the computer program of claim 6 which cause the involved computers to carry out the following method steps: acquiring data about the production process and its relevant process parameters from at least two different sources by using the at least one Computer; deploying internal exploratory analyses of the acquired data by using the at least one Computer; training at least one selected machine learning model for selected target parameters by using the at least one Computer; optimizing hyperparameters of the selected at least one model and validating the optimized, selected at least one model by using the at least one Computer; analyzing the acquired data with the optimized and validated at least one model and post-processing the results for downstream use by at least one user using a Frontend Graphical User Interface provided by the at least on Computer; and improving the analyzed production process with the thereby gained results.