Quality control of products for mass production

Abstract

A method (100) for quality control of mass-produced products (1) with the following steps: • recording (110) a plurality of parameters (11) characterizing the manufacturing process during the manufacture of a product (1); • controlling (120) the product (1), the control comprising at least one physical observation of the product (1) and / or at least one physical functional test of the product (1) and comparing the results obtained at the time of observation or at the time of functional test with pre-given references, by comparison a quality assessment (12) of the product (1) is derived; • in response to the quality assessment (12) meeting pre-given criteria (130), the parameters (11) recorded during the manufacture of the product (1) are fed to a trained machine learning model (2) and mapped by the trained machine learning model (2) onto a quality prediction (13) of the product (1) (140); • checking whether the quality prediction (13) is consistent with the quality assessment (12) (150); • if this is the case, the recorded parameters (11) are fed to an explainer (21) of the machine learning model (2) (160), the explainer (21) assigning a quantitative contribution (14) to the quality prediction (13) to individual recorded parameters (11) and / or combinations of recorded parameters (11); • from the quantitative contributions (14), possible causes (15) of the quality assessment (12) obtained at the time of control (120) are evaluated (170).

Description

Technical Field

[0001] This invention relates to quality control for mass-produced products, and particularly to quality control of products that undergo functional final inspection after multiple processing steps. Background Technology

[0002] For complex products manufactured through multiple processing steps, quality control becomes increasingly difficult with the number of steps. Performing quality control after each step is generally too time-consuming to achieve 100% control in mass production. Typically, time is only sufficient for a final inspection (“end-of-line” test) at the end of processing. If problems arise in this final inspection, the specific issue does not always provide useful indications of its underlying cause. Investigating the cause is labor-intensive and under significant time pressure, as quality problems or resulting downtime incur high operating costs until the cause is found and eliminated.

[0003] A manufacturing method for complex products is known from DE 101 38 760A1, in which a product-specific electronic data carrier is assigned to the product during manufacturing. Manufacturing steps are recorded on this data carrier. The electronic data carrier remains permanently assigned to the product after production is completed, enabling traceability of quality issues. Summary of the Invention

[0004] Within the framework of this invention, a method for quality control of mass-produced products has been developed.

[0005] The method includes recording multiple parameters characterizing the manufacturing process during product manufacturing. This manufacturing process can be referenced in this respect by a "black box" function, which is known only to have at least the recorded parameters as independent variables and to map these independent variables to measures for product quality.

[0006] As parameters characterizing the manufacturing process, considerations are particularly given to settings such as welding current, the force applied to the product, or the sputtering current of a coating device, which are used to process the product during the manufacturing process. It is not always feasible to simply keep these parameters constant for good reproducibility of the product during mass production. Rather, these parameters typically must be set according to the circumstances, for example, to adjust a specific parameter of interest to a desired value.

[0007] The duration of time since the last maintenance and / or setup work was performed on at least one machine (which processes products during the manufacturing process) can also have a positive or negative impact on product quality. A machine being readjusted after maintenance can, for example, have a positive effect. A negative effect, for example, is that the vacuum chamber will not be as clean immediately after maintenance pumping as it would be after months of continuous vacuuming.

[0008] Similarly, a measure of the wear condition of at least one tool that comes into contact with the product during the manufacturing process can be correlated with product quality. Many tools used in machining are wear parts, designated for a defined service life, and whose properties (Beschaffenheit) change gradually over this service life. This measure can be obtained, for example, through physical measurements at the tool; however, alternatively or in combination with it, it can also be obtained, for example, using a service life counter that counts the duration of tool operation and / or the number of products processed with that tool. The physical measurement can involve, for example, the dimensions of the tool, but can also involve, for example, its hardness or toughness.

[0009] Furthermore, any measured value from measuring at least one physical measurement parameter can be considered as a parameter, which is on the product under manufacture, on the primary product used for said manufacture, and / or in the environment in which said product is manufactured. Thus, for example, the primary product can be routinely inspected for its physical properties or material composition (e.g., its purity), making it possible to determine, if necessary, whether changes in properties or material composition affect the quality of the product. These properties can, for example, involve dimensions, hardness, toughness, crystal structure, or magnetic properties. Climatic conditions (e.g., temperature and / or air humidity) in the spatial environment in which the product is manufactured can also be recorded. Thus, for example, the final strength of an adhesive can depend on the dominant temperature and air humidity during processing.

[0010] Finally, for example, at least one timestamp at at least one point in time can be considered as a possible parameter affecting the quality of the product, at which at least one processing step has been performed on the product. If, for example, a new, previously unknown quality problem arises, the problem can be at least limited by a general understanding of the time period in which it occurred.

[0011] The product is controlled during or at the end of the manufacturing process, such as through end-of-line testing (EoL). This control includes at least one physical observation of the product and / or at least one physical functional test of the product, and comparing the results obtained at the time of the observation or the functional test with a pre-given reference. Thus, it is possible, for example, to expose a sensor (such as a pressure sensor or air quality meter) to one or more test stimuli, and then to verify whether the sensor's response to the test stimuli is within a pre-given tolerance band. The quality assessment of the product is obtained through this comparison.

[0012] The quality assessment can include, for example, classifications into levels such as "Super", "OK", "NOK", and any intermediate level. For instance, a product can be classified into the "NOK" quality level in response to identified defects or damage (such as cracks or missing materials).

[0013] In response to the quality assessment meeting pre-defined criteria—that is, the quality assessment leading to a classification in the corresponding grade and, for example, indicating that the product's quality does not meet requirements (grade "NOK")—measures for conducting a root cause investigation are introduced. For this purpose, parameters recorded during product manufacturing are fed into a trained machine learning model and mapped by the trained machine learning model to a quality prediction for the product. However, conversely, it is also entirely possible to search for reasons why a specific sample of the product is particularly good and has been assigned to the grade "Excellent." If this can be reasonably explained, the manufacturing process may be optimized. The quality predictions are predictions of the quality assessment.

[0014] Machine learning models are specifically considered to be models that embody powerful generalization capabilities through parameterization using adaptable parameters. These parameters are adapted during training to reproduce, as closely as possible, the corresponding pre-known learning output data when inputting learning input data into the machine learning model. Machine learning models can, in particular, include and / or be KNN artificial neural networks. However, gradient boosted trees or support vector machines can also be used, for example.

[0015] Now we examine whether the quality prediction is consistent with the quality assessment obtained through physical controls. This is, for example, when both the physical controls of the product and the quality predictions obtained through a machine learning model assign the product the same or similar evaluation numbers or grades for its quality. Here, "similar" can be understood, for example, as the deviation between the quality prediction and the quality assessment not exceeding a predetermined number of grades or a predetermined value of evaluation numbers.

[0016] In this context, it is assumed that the machine learning model adequately explains the quality state as determined by the physical controls of the product. To further investigate the underlying causes of quality problems, the recorded parameters are fed into an interpreter for the machine learning model. This interpreter assigns a quantitative contribution to the quality prediction to each recorded parameter and / or combination of recorded parameters. The “quantitative contribution of a parameter or combination of parameters” can be understood in particular as the weight by which the parameter or combination of parameters is included in the quality prediction.

[0017] Assess the possible, most probable, and / or anticipated causes of the quality assessment obtained during control from the quantitative contributions.

[0018] Therefore, the interpreter can, for example, provide the following result: the "not OK" quality prediction provided by the machine learning model can be attributed with high or highest probability to:

[0019] • The diameter of the borehole is within the lower tolerance limit.

[0020] The bolts to be inserted into the borehole must come from a designated supplier, and

[0021] An attempt was made to insert the bolt into the borehole at the critical temperature, but the bolt became stuck and could not be inserted as deeply as intended.

[0022] In particular, it is possible to distinguish, for example, whether a given parameter has a decisive influence on quality prediction on its own or only in combination with other parameters.

[0023] If, for example, the quantitative contribution of the parameter “hole diameter” exceeds a first pre-given threshold, the interpreter is able to provide the following result: the hole diameter itself has also led to a “not OK” quality prediction.

[0024] Conversely, if the quantitative contributions of the parameters “drill hole diameter”, “bolt supplier”, and “temperature” are only significantly greater than a second pre-defined threshold that is significantly smaller than the first pre-defined threshold, then the interpreter can provide the following result: the combination of these three parameters is decisive for the quality prediction of “not OK”.

[0025] On the one hand, the physical control of the product and on the other hand, the causal research using machine learning models work synergistically in many ways.

[0026] Since machine learning models are only considered in cases where, from a physical control perspective, they are specific according to pre-given standards and therefore require more detailed investigation, the computational time allocated to machine learning models is not used for "uninteresting" normal cases. In these normal cases, machine learning models cannot provide new insights because the manufacturing process operates precisely as expected, i.e., unchanged after physical control. Therefore, it is advantageous to concentrate the computational time allocated to machine learning models on a significantly smaller number of special cases. This then allows for a greater, on average, allocation of computational time to any cases where such investigations will not be a limiting factor for productivity.

[0027] Physical control not only motivates the use of machine learning models but also serves to validate the reliability of their conclusions. If a contradiction arises—for example, if the machine learning model predicts that a product identified as defective during physical control should have met requirements—then the interpretation of the quality prediction obtained using the interpreter is no longer reliable from the outset. Therefore, computational costs for the interpreter can still be saved here. Simultaneously, such contradictions suggest that the error causes learned by the machine learning model may no longer be correct, and the true causes should be sought outside of these learned error causes. This also highlights the importance of explicitly limiting possible error causes, which saves valuable time.

[0028] Therefore, the reasons why products fail physical quality control can be identified more quickly and with better accuracy. This allows for faster elimination of the cause of the error, resulting in a lower defect rate or less downtime in production.

[0029] In a particularly advantageous configuration, 50 to 10,000, preferably 100 to 2,000, parameters are recorded during product manufacturing. This range is such that, on the one hand, simplified methods for cause analysis without using machine learning models reach their limits, and on the other hand, the computational cost of evaluating machine learning models and, in particular, the associated interpreters can still be processed quickly enough as the number of parameters increases.

[0030] For similar reasons, it is advantageous to record the parameters during the 10 to 200, preferably 50 to 150, manufacturing steps the product undergoes.

[0031] In a particularly advantageous configuration, an interpreter is selected that comprises an approximation of a machine learning model. The approximation is chosen to at least locally mimic the characteristics of the machine learning model while simultaneously having lower complexity, i.e., being easier to interpret. The approximation can in particular include, for example, approximations of the model that can be computed with less complexity and / or at greater speed. Here, the term "complexity" can particularly refer to the requirements for memory or clock cycles, and alternatively, or in combination with them, include the requirements for parameters and / or relationships between parameters used to represent the approximation. In particular, a smaller number of parameters and / or relationships between parameters allows the approximation to be interpreted more easily compared to the original machine learning model.

[0032] The underlying understanding here is that such an approximation allows for the relatively quick identification of quality problems caused precisely by relatively small variations in manufacturing process parameters. At the same time, such quality problems are precisely the ones that have been difficult to diagnose until now, as illustrated by the example of the bolt stuck in the drill hole mentioned at the beginning. Conversely, the severity of a failure regarding one or more parameters (which deviate from the applicability of localized imitations) often has causes that are obvious even without the use of machine learning, such as a completely faulty machining machine.

[0033] In particular, for example, it is possible to select an interpreter that provides at least one locally interpretable model-agnostic explanation (LIME) for the quality prediction. Such an interpreter approximates the characteristics of the machine learning model in a locally multilinear manner.

[0034] In another particularly advantageous configuration, an interpreter is chosen that is constructed to calculate the marginal contribution to the quality prediction from a sequence of parameters, where no specific parameter or combination of parameters appears in the sequence, and the marginal contribution is achieved by adding the specific parameter or combination of parameters. In particular, the average marginal contribution of a specific parameter or combination of parameters under study can be calculated, for example, by corresponding evaluations for many sequences and averaging these sequences. Thus, the case with parameters or combinations of parameters is compared with the case without parameters or combinations of parameters. This is somewhat analogous to measuring assistive devices at an optician's, where the client is repeatedly asked which of the two currently available configurations they can better identify the rows of numbers or letters.

[0035] In this way, it is particularly possible to study the impact of, for example, multiple parameters or combinations of parameters on quality prediction. These parameters can also have opposite effects on quality prediction. Therefore, for example, for a specific sample of a product, certain aspects of the manufacturing process may be very successful, while other certain aspects may not be successful at all. Here, some unsuccessful aspects may be "cured" by successful aspects, while others may not. The calculation of marginal contribution is model-agnostic. That is, the model can be used as a "black box" without requiring simplified adoption.

[0036] In particular, it is possible to select, for example, an interpreter constructed for calculating the Shapley value for specific parameters or combinations of parameters related to the machine learning model or the conditional expectation of the machine learning model. For the Shapley value, there is a mathematical guarantee that the sum of the quantitative contributions of all parameters forms the difference between the average quality prediction and the quality prediction for the specific product under study. Furthermore, it is possible to calculate Shapley values ​​for any permutation of the order and to average them over these orders, with the parameters added to the aforementioned sequence in the stated order.

[0037] The Shapley value derived in relation to the conditional expectation of a machine learning model is also called the SHAP value (Shapley Additive Explanations). The SHAP value can be specifically determined as, for example, the Shapley value of the conditional expectation function of the machine learning model, as described below: “A Unified Approach to Interpreting Model Predictions” by SMLundberg and S.-I. Lee, 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, https: / / papers.nips.cc / paper / 7062-a-unified-approach-to-interpreting-model-predictions. In particular, the conditional expectation function can be defined to a simplified input to the machine learning model.

[0038] The cost of the aforementioned mathematical guarantees is that calculating the Shapley value becomes exponentially more expensive with the increase in the number of parameters. Advantageously, in this case, the interpreter does not have to work on every single sample of the product during mass production, but only on those samples where the product's performance is significantly positive or negative depending on physical control. This accompanies the ability to use machine learning models that can be interpreted particularly easily by the interpreter. If, for example, a gradient boosting tree-based machine learning model is used, then SHAP with a "tree interpreter" can be utilized as the interpreter, which operates particularly fast and reliably for the model's underlying structure.

[0039] The Shapley value, or SHAP value, provides the marginal contribution of a parameter or the marginal contribution of a combination of parameters, and thus makes causal investigation for determining quality predictions easier. Therefore, it becomes possible to identify, for example, whether the workpiece's material composition, the temperature during processing, or the quality of machining are the causes of poor final workpiece quality.

[0040] This is somewhat analogous to efforts to identify the primary causes of the novel coronavirus SARS-CoV-2 infection at the most localized level possible, so as to precisely eliminate these primary causes while allowing as much public life as possible. Here, for example, a strategy could be developed that allocates the contribution to reducing the replication number R (i.e., the average number of people infected by one infected person) to each measure from a pre-given catalogue (e.g., contact restrictions, mask requirements, or border closures). Since the cost of each measure is known or at least can be estimated, these measures could be optimized, for example, with the objective of achieving a replication number R of less than 1 with the smallest possible total cost.

[0041] In another advantageous configuration, at least one recommendation for a process-guided change to the manufacturing process is additionally evaluated from the quantitative contribution used for quality evaluation and / or from previously assessed possible, most likely, and / or anticipated causes. Products subsequently manufactured after implementing the change are expected, at control, to be closer to a pre-given reference than products manufactured before implementing the change. This recommendation can be communicated to the production line operator, for example, in any form. If, for example, the recommendation is to adapt the welding current at a welding machine, then the recommendation can be displayed directly at the welding machine. The recommendation can also be, for example, to reduce the temperature in the processing chamber, wherein how this is specifically implemented is determined by the operator.

[0042] The present invention also relates to a method for training a machine learning model for predicting the quality of products used in mass production. This training is particularly designed to enable the machine learning model to be used in the aforementioned methods.

[0043] This method provides multiple parameters that characterize the product manufacturing process and are recorded separately during the manufacture of multiple samples of the product.

[0044] These parameters are first preprocessed before they are used to train a machine learning model. For this purpose, significant changes are searched for in the parameters according to at least one pre-given criterion. This pre-given criterion can, for example, include outliers and / or significant jumps in the parameter over time. In response to finding at least one significant change, the significant change is removed, and / or an operator is requested to interpret the significant change and add the interpretation to the parameter. The operator's interpretation can, for example, include statements such as: maintenance has been performed or tools or consumables have been updated. Removing the significant change can in particular include, for example: clearing or completely removing values ​​reflecting the significant change from the set of available parameters. Therefore, for each sample of the product, the set of available parameters is coordinated and complete such that the quality prediction and the parameters used to obtain that quality prediction are identifiable and independent of additional parameters not included in the set of mentioned parameters.

[0045] Such preprocessing is crucial for training effectiveness. Data sources providing parameters from industrial manufacturing processes are often not designed to ensure that data collected over long periods or even on different devices remains comparable and thus reasonably aggregated for training machine learning models. Therefore, measurements recorded before and after machine maintenance, for example, can sometimes only be conditionally compared because the measuring instruments or processes may be affected by maintenance. Such effects can lead to significant variations in the measurement data. Consequently, without preprocessing, inconsistencies can arise in the training data, which can severely hinder the training process or distort the learned model.

[0046] For each sample of the product for which parameters have been provided, a quality metric is provided, which is detected based on at least one physical observation and / or at least one physical functional test of that sample of the product during or after the manufacturing process. Conversely, the previously provided parameters are mapped by a machine learning model to a quality prediction for the corresponding sample of the product. The quality prediction is then compared to the corresponding quality metric.

[0047] The goal of optimizing model parameters that characterize the features of a machine learning model is to make quality predictions more closely approximate quality metrics when the parameters are further processed by the machine learning model. Model parameters can be, for example, weights in a neural network. Any optimization algorithm can be used for this. Therefore, for example, a pre-given cost function can be used to evaluate the deviation between quality predictions and quality metrics. Here, the value of the cost function can be backpropagated to the model parameters based on the gradients of the model parameters. Thus, the model parameters can be selectively modified at each training step, providing a reasonable prospect to improve the value of the cost function.

[0048] The quality assessment can include, for example, classifications into levels such as "Super," "OK," "NOK," and any intermediate level. For instance, a product can be classified into the "NOK" quality level in response to the identification of specific defects or damage (such as cracks or missing materials). The quality metric can also be a quality score obtained according to any pre-defined inspection scheme. The durability of a product, determined after its eventual failure, can also be used as a quality metric. Therefore, the lifespan (Brenndauer) of a lighting device until its failure can, for example, be used as a measure of its quality.

[0049] In a particularly advantageous configuration, parameters that have been detected at least partially during the product pattern construction phase prior to the commencement of mass production are selected. This is particularly relevant, for example, by the so-called "C-pattern construction," which has already been performed on the production line set up for later mass production. The way the machine learning model interprets the salience of the product in the aforementioned approach is relatively robust to changes imposed on the training data by the transition from C-pattern construction to mass production. Instead, training benefits from the fact that parameters have significantly greater variability precisely in C-pattern construction than in later mass production. In particular, some parameters (e.g., expected values ​​for settings at a given processing machine) are still formulated or fine-tuned during C-pattern construction.

[0050] In particular, training can begin using the training data available at that point in time, for example, before mass production. Therefore, as new training data is added, the machine learning model can always continue training, for example, incrementally. Since the machine learning model requires relatively little training and can thus be started especially before mass production begins, it is immediately available if problems suddenly arise during mass production and a rapid explanation is needed.

[0051] In another advantageous configuration, the interaction between different parameters concerning the quality of the product is evaluated from a trained machine learning model, and / or at least one tolerance range for at least one parameter that is not critical to the quality of the product is evaluated.

[0052] For example, the fact that the value of the first parameter is within a first range can itself have a first effect. The fact that the value of the second parameter is within a second range can itself have a second effect. Here, the interaction between the first and second parameters can, for example, include the following: the simultaneous occurrence of the value of the first parameter within the first range and the value of the second parameter within the second range has an effect that differs from the sum of the first and second effects. As mentioned earlier, for example, the limiting diameter of the borehole into which the bolt is to be inserted, the imprecise manufacturing of the bolt, and the limiting temperature can work together to cause the bolt to jam when inserted into the borehole.

[0053] "The tolerance range of a parameter" can be understood, for example, as the range within which the value of the parameter can move without deviating from the nominal state. For instance, if a drill hole is supposed to have a nominal diameter of 0.2 mm, then the product can still be OK if the drill hole actually has a diameter of 0.19 or 0.21 mm.

[0054] The interaction relationships and tolerance ranges can be obtained, for example, with the aid of an interpreter in the manner described above. However, the interaction relationships and tolerance ranges can also be obtained, for example, directly from the quality predictions provided by the machine learning model. Thus, for example, quality predictions can be obtained separately for multiple clusters (Konstellationen) of parameters. If, for example, a combination of changes in multiple parameters emanating from such a cluster results in a change in the quality prediction (conversely, no change in the quality prediction occurs when one of the changes is omitted), then the interaction relationships between the parameters can be identified.

[0055] If, starting from a cluster of parameters, a parameter can be changed within a certain range without altering the quality prediction, then this range can be identified as the tolerance range of that parameter. Conversely, it can be determined in such a way that even a small change in a given parameter has resulted in a change in the quality prediction. This parameter can here be identified as the one on which the product quality is particularly critical.

[0056] Therefore, trained machine learning models appear to be able to serve as a “digital twin” of the manufacturing process, where the effects of parameter variations can be studied significantly faster than in the manufacturing process itself.

[0057] Therefore, the understanding developed by machine learning models can be directly used to improve the reliability and economy of manufacturing processes.

[0058] Trained machine learning models offer the potential for providing good incentives, particularly for determining tolerances. Shape and position tolerances, in particular, can be controlled as broadly as possible in this way. This simplifies production and reduces costs that would otherwise increase disproportionately with more stringent tolerances.

[0059] Further knowledge can be extracted from the trained model, and thus decoupled from the specific production line. Therefore, the understanding that, for example, product quality is particularly critically dependent on specific parameters can incentivize the expansion of product and / or manufacturing processes precisely when this dependency is less critical. Here, the product can be produced with less overhead and / or a lower scrap rate.

[0060] Furthermore, the extracted knowledge makes it easy to replicate the production line in the same location or another. This is especially true for heavy and / or bulky components (such as vehicle batteries) used in the automotive industry's supply chain, which are typically not centrally shipped from one location worldwide but are instead produced in a decentralized manner. Here, it is crucial that the product consistently exhibits precisely the same performance regardless of which factory it originates from.

[0061] Advantageously, XGBoost, Support Vector Machines, and Interpretable Boosting Models (Boosting-Modell) are chosen as machine learning models. These types of models work particularly well with the aforementioned interpreters based on LIME, or Shapley values.

[0062] The methods can be implemented, in particular, entirely or partially, by a computer. Therefore, the invention also relates to a computer program having machine-readable instructions that, when implemented on one or more computers, enable the one or more computers to implement one of the described methods. In this sense, controllers for vehicles and embedded systems for technical devices can also be considered computers, which are similarly capable of implementing machine-readable instructions.

[0063] Similarly, the present invention also relates to a machine-readable data carrier and / or a downloadable product having a computer program. A downloadable product is a digital product that can be transmitted via a data network (i.e., downloaded by a user of the data network), which, for example, can be sold in an online store for immediate download.

[0064] In addition, computers can be equipped with computer programs, machine-readable data carriers, or downloadable products. Attached Figure Description

[0065] Further improvements to the invention are presented below in more detail together with the description of preferred embodiments of the invention with reference to the accompanying drawings.

[0066] It shows:

[0067] Figure 1 An embodiment of a method 100 for quality control of mass-produced product 1;

[0068] Figure 2 An embodiment of a method 200 for training a machine learning model 2. Detailed Implementation

[0069] Figure 1 This is a schematic flowchart of an embodiment of a method 100 for quality control of mass-produced product 1.

[0070] In step 110, multiple parameters 11 characterizing the manufacturing process are recorded during the manufacturing of product 1. Here, according to block 111, for example, 50 to 10,000, preferably 100 to 2,000 parameters 11 can be recorded. According to block 112, the parameters 11 can be recorded during 10 to 200, preferably 50 to 150, manufacturing steps of product 1.

[0071] In step 120, physical control is performed on product 1. This physical control leads to quality evaluation 12.

[0072] In step 130, it is checked whether the quality evaluation 12 meets a pre-given standard, such as "quality score difference from pass". If this is the case (true value 1), then in step 140, the parameters 11 recorded during the manufacturing of product 1 are fed into the trained machine learning model 2 and mapped by the trained machine learning model 2 onto the quality prediction 13 for product 1.

[0073] Then, in step 150, it is checked whether the quality prediction 13 is consistent with the quality assessment 12. If so (true value 1), then in step 160, the recorded parameters 11 are fed to the interpreter 21 for the machine learning model 2. The interpreter 21 assigns the quantitative contribution 14 to the quality prediction 13 to the individual recorded parameters 11 and / or combinations of recorded parameters 11. Conversely, if the quality prediction 13 is inconsistent with the quality assessment 12 (true value 0 in step 150), the model can be retrained in an improved form and / or with additional parameters.

[0074] According to box 105, as a machine learning model 2, it is particularly possible to select, for example, XGBoost models, support vector machines, and / or interpretable boosting models.

[0075] According to block 161, for example, an interpreter 21 can be selected, which includes an approximation of machine learning model 2. This approximation at least locally mimics the characteristics of machine learning model 2 and can be interpreted more easily than machine learning model 2. In particular, for example, according to block 161a, an interpreter 21 can be selected that provides at least one locally interpretable, model-agnostic, LIME explanation for quality prediction 13.

[0076] According to box 162, an interpreter 21 can be selected, for example, to calculate the marginal contribution to the quality prediction from the sequence of parameters 11, wherein no specific parameter 11' or a specific combination of parameters 11' appears in the parameters, and the marginal contribution is achieved by adding specific parameters 11' or specific combinations of parameters 11'. Specifically, according to box 162a, an interpreter 21 can be selected, for example, to calculate the Shapley value, which is used for specific parameters 11' or specific combinations of parameters 11' related to the conditional expectation of the machine learning model 2 or the machine learning model.

[0077] In step 170, the possible, most probable, and / or anticipated causes 15 of the quality evaluation 12 obtained at control 120 are evaluated from the quantitative contribution 14. Optionally, in step 180, at least one suggestion 16 for process-guided changes to the manufacturing process of product 1 can be additionally evaluated from the quantitative contribution 14 and / or from the previously evaluated possible, most probable, and / or anticipated causes 15. Such changes are implemented such that product 1 manufactured after the changes are implemented in the manufacturing process is closer to a pre-given reference at physical control 120 than product 1 manufactured before the changes were implemented.

[0078] Figure 2 This is a schematic flowchart of an embodiment of a method 200 for training a machine learning model 2 for quality control of a mass-produced product 1. This machine learning model 2 is particularly applicable to the method 100 described above.

[0079] In step 210, a plurality of parameters 11 are provided, which characterize the manufacturing process of product 1 and are recorded respectively during the manufacture of multiple samples of product 1. Here, according to block 211, in particular, parameters can be selected, for example, those parameters that have been detected at least in part during the pattern construction stage of product 1 before the start of mass production of product 1.

[0080] In step 220, within the framework of preprocessing, significant changes are searched for in parameter 11 according to at least one pre-given criterion. If such a change (truth value 1) is identified, it is removed in step 230, and / or, in step 240, the operator is requested to interpret the corresponding significant change and the interpretation is added to parameter 11.

[0081] In step 260, the preprocessed parameters 11 are mapped by the machine learning model 2 to the quality prediction 13 for the corresponding sample of product 1. Furthermore, in step 250, a quality metric 13# is provided for each of these samples of product 1, the quality metric being determined based on at least one physical observation of that sample of product 1 and / or based on at least one physical functional test of that sample of product 1 detected during or after the manufacturing process.

[0082] According to box 205, as a machine learning model 2, it is particularly possible to select, for example, XGBoost models, support vector machines, and / or interpretable boosting models.

[0083] In step 270, the quality prediction 13 is compared with its corresponding quality metric 13#. Based on the result of comparison 270, in step 280, the model parameters 2a characterizing the features of the machine learning model 2 are optimized, with the goal that the quality prediction 13 becomes closer to the quality metric 13# when the parameters 11 are further processed by the machine learning model 2. Training can continue until any termination criterion is met. Such termination criteria can include, for example, meeting the accuracy of the quality prediction 13 as measured by test or validation data, or the difference between the quality prediction and the quality metric not exceeding a pre-given threshold. The state of completed training of model parameters 2a is indicated by reference numeral 2a*.

[0084] Optionally, in step 290, it is possible to additionally evaluate the interaction relationship 11* between different parameters 11 and / or at least one tolerance range 11** for at least one parameter 11 from the trained machine learning model 2.

Claims

1. A method (100, 200) for quality control of mass-produced products (1), the method comprising the following steps: • Record (110) multiple parameters (11) characterizing the manufacturing process during the manufacture of the product (1). • The product (1) is controlled by (120), wherein, The control includes at least one physical observation of the product (1) and / or at least one physical functional test of the product (1), and comparing the results obtained during the observation or the functional test with a pre-given reference, wherein a quality evaluation (12) of the product (1) is obtained through the comparison. • In response to the quality evaluation (12) meeting the pre-given criteria (130), the parameters (11) recorded during the manufacturing of the product (1) are fed into the trained machine learning model (2) and mapped (140) by the trained machine learning model (2) onto the quality prediction (13) for the product (1); • Verify whether the quality prediction (13) stated in (150) is consistent with the quality evaluation (12); • If this is the case, the recorded parameters (11) are fed (160) to the interpreter (21) for the machine learning model (2), wherein the interpreter (21) assigns the quantitative contribution (14) to the quality prediction (13) to each of the recorded parameters (11) and / or the combination of the recorded parameters (11). • From the quantitative contribution (14), evaluate (170) the possible reasons (15) for the quality assessment (12) obtained during the control (120).

2. The method (100, 200) according to claim 1, wherein, From the quantitative contribution (14), evaluate (170) the most likely cause (15) of the quality assessment (12) obtained when the control (120) is applied.

3. The method (100, 200) according to claim 1 or 2, wherein, The parameter (11) characterizing the manufacturing process includes • The setup of at least one machine that processes the product (1) during the manufacturing process, and / or • The duration since the last maintenance and / or setup work was performed on at least one machine that processed the product (1) during the manufacturing process, and / or • A measurement of the wear condition of at least one tool that came into contact with the product (1) during the manufacturing process, and / or • Measurements derived from measuring at least one physical measurement parameter, said at least one physical measurement parameter being on the product (1) under manufacture, on a primary product used for said manufacture, and / or in the environment in which said product (1) is manufactured, and / or • At least one timestamp at at least one point in time, at which at least one processing step has been performed on the product (1) at the at least one point in time.

4. The method (100, 200) according to claim 1 or 2, wherein, During the manufacture of the product (1), 50 to 10,000 parameters (11) are recorded (111).

5. The method (100, 200) according to claim 4, wherein, During the manufacture of the product (1), 100 to 2000 parameters (11) are recorded (111).

6. The method (100, 200) according to claim 1 or 2, wherein, During the process of the product (1) undergoing 10 to 200 manufacturing steps, the parameter (11) is recorded (112).

7. The method (100, 200) according to claim 6, wherein, During the 50 to 150 manufacturing steps of the product (1), the parameters (11) are recorded (112).

8. The method (100, 200) according to claim 1 or 2, wherein, The selection (161) includes an interpreter (21) that approximates the machine learning model (2) at least locally, and the approximation has lower complexity than the machine learning model (2).

9. The method (100, 200) according to claim 8, wherein, Select (161a) an interpreter (21) that provides an explanation of at least one locally interpretable model that is not known for the quality prediction (13).

10. The method (100, 200) according to claim 1 or 2, wherein, Select (162) an interpreter (21) configured to calculate the marginal contribution to the quality prediction from the sequence of the parameters (11), wherein no specific parameter (11') or specific combination of parameters (11') appears in the parameters, and the marginal contribution is achieved by adding the specific parameter (11') or specific combination of parameters (11').

11. The method (100, 200) according to claim 10, wherein, Select (162a) an interpreter (21) configured to obtain a Shapley value for a specific parameter (11') or a specific combination of parameters (11') associated with the machine learning model (2) or the conditional expectation of the machine learning model.

12. The method (100, 200) according to claim 1 or 2, wherein, From the quantitative contribution (14) and / or from the assessed possible causes (15), at least one recommendation (16) for process-guided changes to the manufacturing process is additionally evaluated (180) such that the product (1) manufactured after the implementation of the changes in the manufacturing process is expected to be closer to the pre-given reference when under the control (120) than the product (1) manufactured before the implementation of the changes.

13. The method (100, 200) according to claim 1 or 2, wherein the machine learning model is used for quality prediction of mass-produced products (1), the method comprising the following steps: • Provide (210) multiple parameters (11) that characterize the manufacturing process of the product (1) and are recorded during the manufacture of multiple samples of the product (1); • Search for significant changes in (220) in the parameter (11) according to at least one pre-given criterion; • In response to finding at least one significant change, remove (230) the significant change, and / or request the operator to explain the significant change and add (240) the explanation to the parameter (11). • For each sample of the product (1) for which parameters (11) have been provided, a quality metric (250) (13#) is provided, which is determined based on at least one physical observation of the sample of the product (1) and / or based on at least one physical functional test of the sample of the product (1) during or after the manufacturing process. • The provided parameters (11) are mapped (260) by the machine learning model (2) to the quality prediction (13) of the corresponding sample for the product (1); • Compare the quality prediction (13) with the corresponding quality metric (13#) (270); • Optimize (280) the model parameters (2a) that characterize the features of the machine learning model (2), with the goal that the quality prediction (13) is closer to the quality metric (13#) when the parameters (11) are further processed by the machine learning model (2).

14. The method (100, 200) according to claim 13, wherein, Select (211) parameters that have been detected at least in part during the style construction phase of the product (1) prior to the commencement of mass production of the product (1).

15. The method (100, 200) according to claim 13, wherein, From the trained machine learning model (2), evaluate (290) the interaction (11*) between different parameters (11) and / or at least one tolerance range (11**) for at least one parameter (11).

16. The method (100, 200) according to claim 1 or 2, wherein, XGBoost models, support vector machines, and / or interpretable boosting models were selected (105, 205) as machine learning models (2).

17. A computer program product containing machine-readable instructions that, when executed on one or more computers, enable the one or more computers to perform the method (100, 200) according to any one of claims 1 to 16.

18. A machine-readable data carrier having a computer program according to claim 17.

19. A computer equipped with a computer program according to claim 17 and / or equipped with a machine-readable data carrier and / or download product according to claim 18.

Images