QUALITY CONTROL FOR PRODUCTION PRODUCTS

Description

[0001] The present invention relates to quality control for mass-produced products, and in particular for products that are subjected to a functional final manufacturing inspection only after a large number of processing steps. State of the art

[0002] For complex products manufactured in numerous processing steps, quality control becomes increasingly difficult as the number of steps increases. Performing a quality check after each step is simply too time-consuming to be practical for 100% inspection in mass production. Often, there is only enough time for a final production inspection ("end-of-line" test) at the end of the manufacturing process.

[0003] If a problem is discovered during this final inspection, the specific issue doesn't always provide a useful clue to its underlying cause. Root cause analysis is labor-intensive and subject to considerable time pressure, as the quality problem, or even a resulting production stoppage, incurs high ongoing costs until the cause is identified and resolved.

[0004] German patent DE 101 38 760 A1 discloses a manufacturing process for complex products in which a product-specific electronic data carrier is assigned to a product under construction. The manufacturing steps are recorded on this data carrier. The electronic data carrier remains permanently assigned to the product even after completion, so that quality problems can be traced.

[0005] EP 3 667 445 A1 discloses a method and a device for manufacturing a product. The product is manufactured in at least one production step. Optionally, quality control is performed after at least one of the production steps to determine a quality index for the respective product. To eliminate the need for quality control, a quality indicator for the respective product is determined based on production data. The production data is advantageously provided by sensors. The calculation of the quality indicator for the respective product is preferably carried out using a machine learning algorithm. The machine learning algorithm can be trained and / or improved using quality indices from a quality control unit and the corresponding production data. Training the machine learning algorithm is preferably carried out using an additional computing unit, particularly in a cloud environment.

[0006] EP 3 352 013 A1 discloses the generation of production data for the control or monitoring of a production process.

[0007] US patent 2020 / 0166909 A1 discloses machine learning-based methods and systems for automated object defect classification and adaptive real-time control of manufacturing processes.

[0008] The publication SHICHANG DU ET AL.: "A robust approach for root causes identification in machining processes using hybrid learning algorithm and engineering knowledge", JOURNAL OF INTELLIGENT MANUFACTURING, KLUWER ACADEMIC PUBLISHERS, BO, Vol. 23, No. 5, December 28, 2010, pages 1833-1847, XP035112206, reveals a new robust approach for root cause identification in machine processes using hybrid learning algorithms to improve product quality and productivity.

[0009] The publication RIBEIRO MARCO TULIO MARCOTCR@GMAIL COM ET AL: '°'Why Should I Trust You?" Explaining the Predictions of Any Classifier", PROCEEDINGS OF THE 2017 ACM CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, CIKM '17, ACM PRESS, NEW YORK, USA, August 13, 2016, pages 1135-1144, XP058631176 reveals a new explanatory technique that explains the predictions of a classifier in an interpretable and credible way by learning an interpretable model locally around the prediction. Disclosure of the invention

[0010] Within the scope of the invention, a method for quality control of a mass-produced product was developed.

[0011] The method involves recording a multitude of parameters characterizing the manufacturing process during product production. In this respect, the manufacturing process is comparable to a "black box" function, of which it is only known that it takes at least the recorded parameters as arguments and maps these arguments to a measure of product quality.

[0012] Parameters that characterize the manufacturing process include, in particular, the settings of at least one machine that processes the product during manufacturing, such as a welding current, a force applied to the product, or a sputtering current from a coating system. It is not always possible to simply keep these parameters constant during series production in the interest of good product reproducibility. Rather, these parameters often need to be adjusted depending on the situation, for example, to regulate certain quantities of interest to target values.

[0013] The time elapsed since the last maintenance and / or adjustment work on at least one machine that processes the product during the manufacturing process can also have a positive or negative impact on product quality. For example, a machine being properly adjusted after maintenance can have a positive effect. Conversely, a vacuum chamber not being as clean immediately after being pumped out following maintenance as it would be after many months of uninterrupted vacuum can have a negative effect.

[0014] Similarly, a measure of the wear condition of at least one tool that comes into contact with the product during the manufacturing process can correlate with product quality. Many machining tools are wear parts specified for a certain service life, and their condition changes gradually throughout this lifespan. This measure can be obtained, for example, through a physical measurement of the tool, but alternatively or in combination, for example, with a tool life counter that counts the tool's operating time and / or the number of products already machined with the tool. The physical measurement can relate to the tool's dimensions, but also, for example, to its hardness or toughness.

[0015] Furthermore, any measured values ​​from the measurement of at least one physical quantity, particularly electrical quantities, on the product under construction, on a precursor used in its manufacture, and / or in the environment where the product is manufactured can be used as parameters. For example, precursors can be routinely tested for their physical properties or material composition (such as their purity) to determine whether changes in these properties or composition affect the product's quality. Properties can refer to dimensions, hardness, toughness, crystal structure, or magnetic properties. Climatic conditions (such as temperature and / or humidity) in the environment where the product is manufactured can also be recorded.For example, the final strength of an adhesive can depend on the temperature and humidity prevailing during processing.

[0016] Finally, at least one timestamp from at least one point in time when at least one processing step was performed on the product can also be used as a potential parameter that affects the product's quality. For example, if a new, previously unknown quality problem occurs, it can at least be narrowed down with an approximate knowledge of the time period in which it occurred.

[0017] The product undergoes inspection during or at the end of the manufacturing process, such as an end-of-line (EoL) inspection. This inspection includes at least a physical observation of the product and / or at least one physical functional test, as well as a comparison of the results obtained from the observation or functional test with a predefined reference. For example, a sensor, such as a pressure sensor or an air mass meter, can be exposed to one or more test stimuli, and it can then be checked whether the sensor's response to these stimuli falls within a predefined tolerance range. This comparison provides a quality assessment of the product.

[0018] The quality assessment can, for example, include a classification into the categories "Super", "OK", "not OK = NOK", and any intermediate levels. For instance, a product can be classified as "not OK" if a specific defect or damage, such as a crack or missing material, has been identified.

[0019] In response to the quality assessment fulfilling a predefined criterion, i.e., leading to classification into a corresponding category, and indicating, for example, that the product's quality does not meet requirements (category "NOK"), root cause analysis measures are initiated. For this purpose, the parameters recorded during the product's manufacture are fed into a trained machine learning model, which then maps them to a quality prediction for the product. Conversely, a completely analogous approach can be used to investigate, for example, why a particular instance of the product turned out exceptionally well and was assigned to the "Super" category. If this can be logically explained, the manufacturing process can potentially be optimized. The quality prediction is, in each case, a prediction of the quality assessment.

[0020] A machine learning model is defined as a model that embodies a function parameterized with adjustable parameters and possesses strong generalization power. During training, the parameters of a machine learning model can be adjusted to ensure that, when training input data is fed into the model, the corresponding pre-existing training output data is reproduced as accurately as possible. The machine learning model can, in particular, include an artificial neural network (ANN), and / or it can be an ANN itself. However, other models such as gradient-boosted trees or support vector machines can also be used.

[0021] The process now checks whether the quality prediction is consistent with the quality assessment obtained through physical inspection. This is the case, for example, if the physical inspection of the product and the quality prediction generated by the machine learning model assign the same or similar ratings or classes to the product's quality. "Similar" in this context can mean, for instance, that the quality prediction deviates from the quality assessment by no more than a predetermined number of classes or by no more than a predetermined value of the rating.

[0022] In this case, it is assumed that the machine learning model adequately explains the quality status evident from the physical inspection of the product. To then investigate the underlying cause of a quality problem, the recorded parameters are fed into an explainer for the machine learning model. This explainer assigns quantitative contributions to the quality prediction to individual recorded parameters and / or combinations thereof. The quantitative contribution of a parameter, or combination of parameters, can be understood, in particular, as the weight with which this parameter, or combination of parameters, was incorporated into the quality prediction.

[0023] The quantitative contributions are used to evaluate a probable, most probable and / or likely cause for the quality assessment obtained during the inspection.

[0024] For example, the explainer can provide the result that the quality prediction "not OK" delivered by the machine learning model is highly or extremely likely due to the fact that the diameter of a bore is at the lower tolerance limit, the bolt to be inserted into this bore comes from a specific supplier, and an attempt was made to insert this bolt into the bore at a borderline temperature, whereupon the bolt became jammed in the bore and could not be inserted as far as intended.

[0025] In particular, it is possible, for example, to distinguish whether a specific parameter has a significant influence on the quality forecast on its own or only in combination with other parameters.

[0026] For example, if the quantitative contribution of the parameter "diameter of the bore" exceeds a first predefined threshold, the explainer can provide the result that the diameter of the bore, even taken alone, leads to a quality prediction of "not OK".

[0027] However, if the quantitative contributions of the parameters "diameter of the bore", "supplier of the bolt" and "temperature" each only exceed a second predefined threshold that is significantly lower than the first predefined threshold, the explainer can deliver the result that the combination of these three parameters is decisive for the quality prediction "not OK".

[0028] The physical inspection of the product on the one hand and the root cause analysis using the machine learning model on the other hand work synergistically in several respects.

[0029] By using the machine learning model only in cases that are exceptional from the perspective of physical inspection according to the predefined criteria and therefore require closer examination, no computing time is used for the machine learning model in "uninteresting" normal cases. In these normal cases, the machine learning model cannot provide any new insights, because the fact that the manufacturing process is proceeding exactly as expected is already established after physical inspection. It is therefore advantageous to concentrate the total computing time used for the machine learning model on the significantly fewer exceptional cases. On average, more computing time can then be devoted to each such case without root cause analysis becoming a limiting factor for manufacturing throughput.

[0030] Physical inspection not only provides the impetus for using the machine learning model, but also serves to validate its predictions. If a contradiction arises, for example, if the machine learning model predicts that a product identified as defective during physical inspection should actually be fine, then the explanation of the quality prediction provided by the explainer is inherently unreliable. Therefore, the computational effort for the explainer can be entirely eliminated. At the same time, such a contradiction suggests that the error causes learned by the machine learning model may be incorrect and that the true cause lies elsewhere. This, too, significantly narrows down the possible causes of the error, saving valuable time.

[0031] Ultimately, this allows for faster and more accurate identification of the cause, particularly for why a product fails physical quality control. The root cause can then be eliminated more quickly, resulting in less product scrap and reduced production downtime.

[0032] In a particularly advantageous embodiment, between 50 and 10,000 parameters, preferably between 100 and 2,000, are recorded during the product's manufacture. This is the range in which, on the one hand, simpler root cause analysis methods that do not use a machine learning model reach their limits, and on the other hand, the computational effort for evaluating the machine learning model and, in particular, the associated explainer, which increases with the number of parameters, can still be processed quickly enough.

[0033] For analogous reasons, it is advantageous to record the parameters while the product undergoes between 10 and 200, preferably between 50 and 150, manufacturing steps.

[0034] In a particularly advantageous embodiment, an explainer is chosen that includes an approximation of the machine learning model. This approximation is selected such that it replicates the behavior of the machine learning model, at least locally, while simultaneously exhibiting lower complexity than the machine learning model itself, i.e., it is easier to interpret. The approximation can, for example, include a less complex and / or faster computational approximation of the model. Here, the term "complexity" can refer specifically to the memory or clock cycle requirements, as well as, alternatively or in combination, to the number of parameters and / or operations required to represent the approximation.In particular, a smaller number of parameters and / or links between parameters can make an approximation easier to interpret compared to the original machine learning model.

[0035] The underlying principle is that quality problems arising from relatively minor changes in manufacturing process parameters can be detected relatively quickly using such an approximation. At the same time, such quality problems have been difficult to diagnose until now, as illustrated by the example of the bolt jammed in a bore mentioned earlier. In contrast, severe failures affecting one or more parameters that exceed the validity range of the local simulation typically have causes that are obvious even without machine learning, such as a complete breakdown of the machining center.

[0036] In particular, an explainer can be chosen that provides at least one Local Interpretable Model-Agnostic Explanation (LIME) for predicting quality. Such an explainer approximates the behavior of the machine learning model in a locally multilinear manner.

[0037] In a further, particularly advantageous embodiment, an explainer is chosen who is trained to determine, starting from sequences of parameters in which a specific parameter or combination of parameters does not occur, the marginal contribution to the quality prediction that would be achieved by adding the specific parameter or combination of parameters. In particular, for example, by evaluating many sequences and averaging over these sequences, an average marginal contribution of the specific parameter or combination of parameters under investigation can be determined. Thus, situations with the parameter or combination of parameters are compared with situations without the parameter or combination of parameters.This is somewhat analogous to the fitting of a visual aid at an optician's, where the customer is repeatedly asked which of two currently available configurations allows him to better recognize a series of numbers or letters.

[0038] In this way, the influence of multiple parameters or combinations of parameters on quality prediction can be investigated. These parameters can also have opposing effects on the quality prediction. For example, certain aspects of the manufacturing process may have been very successful for a specific product, while other aspects may have failed completely. Some unsuccessful aspects may then be "cured" by the successful ones, while others may not. The calculation of the marginal contribution is model-agnostic. This means that the model can be used as a "black box" as is, without any simplifying assumptions.

[0039] In particular, an explainer can be chosen who is trained to determine Shapley values ​​for the specific parameter, or for the specific combination of parameters, with respect to the machine learning model or a conditional expectation of the machine learning model. For the Shapley values, there are mathematical guarantees that the quantitative contributions of all parameters add up to the difference between the average quality prediction and the quality prediction for the specific product under investigation. Furthermore, Shapley values ​​can be determined for any permutations of the sequences in which parameters are added to said sequences and averaged over these sequences.

[0040] Shapley values ​​determined with respect to a conditional expectation of the machine learning model are also called SHAP values ​​(Shapley Additive Explanations). In particular, SHAP values ​​can be determined, for example, as Shapley values ​​of a conditional expectation function of the machine learning model, as described, for instance, in (SM Lundberg, S.-I. Lee, "A Unified Approach to Interpreting Model Predictions", 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, https: / / papers.nips.cc / paper / 7062-a-unified-approach-to-interpreting-modelpredictions). Specifically, simplified inputs to the machine learning model can be defined using the conditional expectation function.

[0041] The price for these mathematical guarantees is that calculating the Shapley values ​​becomes exponentially more complex with an increasing number of parameters. In this context, it is advantageous that the explainer does not need to be used for every single product during mass production, but only for those products that exhibit positive or negative deviations based on physical inspection. A machine learning model can be used to complement this, as it is particularly easy for the explainer to interpret. For example, if a machine learning model based on gradient-boosted trees is used, SHAP with a "tree explainer" can be employed as the explainer, as it operates particularly quickly and reliably for the underlying structure of the model.

[0042] Shapley values, or SHAP values, provide marginal contributions from parameters or combinations of parameters, thus facilitating root cause analysis for a specific quality prediction. For example, it can be determined whether the material composition of a workpiece, the processing temperature, or the quality of a machining operation was the cause of poor final workpiece quality.

[0043] This is somewhat analogous to efforts to identify the main causes of new infections with the SARS-CoV-2 coronavirus at the most local level possible, in order to eliminate these causes while simultaneously allowing as much public life as possible. One strategy here, for example, could be to assign each measure from a predefined catalog, such as contact restrictions, mandatory mask-wearing, or border closures, a contribution to reducing the reproduction number R (i.e., the average number of people infected by one infected person). Since the costs of each measure are known or at least estimable, the measures can then be optimized, for example, to achieve a reproduction number R below 1 with the lowest possible overall costs.In a further advantageous embodiment, at least one additional proposal for a change to the manufacturing process is evaluated based on the quantitative contributions and / or the previously evaluated probable, most probable, and / or expected cause for the quality assessment. After implementation of this change, the products manufactured are expected to be closer to the specified reference during inspection than the products manufactured before the change was implemented. This proposal can be communicated to the operating personnel of a production line for the product in any desired format. For example, if the proposal is to adjust the welding current on a welding machine, this proposal can be displayed directly on that welding machine.The suggestion could also be, for example, to lower the temperature in a processing room, leaving it up to the operating personnel to decide exactly how this is implemented.

[0044] The preferred embodiment of the invention also includes a method for training a machine learning model for predicting the quality of a mass-produced product. The purpose of this training is to make the machine learning model usable for the method described above.

[0045] This procedure provides a multitude of parameters that characterize the manufacturing process of the product and were recorded during the production of a large number of copies of the product.

[0046] These parameters are pre-processed before being used to train the machine learning model. For this purpose, the parameters are scanned for unusual changes based on at least one predefined criterion. This criterion might include, for example, that the parameters exhibit outliers and / or apparent jumps over time. If at least one unusual change is detected, it is removed, and / or an explanation for the unusual change is requested from an operator and added to the parameters. The operator's explanation might include, for example, a statement indicating that maintenance was performed or that a tool or consumable was replaced.Removing the conspicuous change can, in particular, involve, for example, adjusting or completely removing values ​​that embody the conspicuous change from the set of available parameters. For each instance of the product, the set of available parameters is thus consistent and complete in such a way that neither the quality forecast nor the parameters used to determine this quality forecast are recognizably dependent on any other variables not included in said set of parameters.

[0047] This preprocessing is crucial for successful training. The data sources that provide parameters from the industrial manufacturing process are often not designed to ensure that data collected over long periods or even on different machines remains comparable and therefore cannot be meaningfully pooled to train a machine learning model. For example, measurements recorded before and after machine maintenance are sometimes only partially comparable because the measuring instrument or the measurement process can be affected by the maintenance. Such an influence can lead to noticeable changes in the measurement data. Without preprocessing, contradictions could arise in the training data, severely hindering the training process or even distorting the learned model.

[0048] For each instance of the product for which parameters have been provided, a quality measure is provided, based on at least one physical observation of that instance and / or at least one physical functional test of that instance during or after manufacturing. The previously provided parameters are then mapped by the machine learning model to quality predictions for the respective instances of the product. These quality predictions are compared with the corresponding quality measures.

[0049] Model parameters that characterize the behavior of the machine learning model are optimized with the goal of ensuring that, as the machine learning model further processes these parameters, the quality predictions more closely match the quality measures. These model parameters can, for example, be weights in a neural network. Any optimization algorithm can be used for this purpose. For instance, the deviation of the quality predictions from the quality measures can be evaluated using a predefined cost function. The value of the cost function can then be propagated back to the model parameters via gradients. Thus, the model parameters can be specifically modified in each training step to offer a reasonable prospect of improving the value of the cost function.

[0050] The quality measure can, for example, include a classification into the categories "Super," "OK," "Not OK" (NOK), and any intermediate levels. For instance, a product might be classified as "Not OK" if a specific defect or damage, such as a crack or missing material, has been identified. The quality measure can also be a quality score determined according to any predefined testing scheme. Even the product's lifespan, determined only after its eventual failure, can be used as a quality measure. For example, the lifespan of a light bulb before it fails can be used as a measure of its quality.

[0051] In a particularly advantageous embodiment, parameters are selected that were at least partially captured during a prototype phase of the product prior to the start of series production. This could, for example, be the so-called "C-prototype" phase, which is already carried out on the production line intended for later series production. The way in which the machine learning model is used in the previously described method to explain anomalies in products is comparatively robust against changes that the transition from C-prototype to series production imposes on the training data. Conversely, the training benefits from the fact that the parameters exhibit significantly greater variability in C-prototype than in later series production.In particular, some parameters, such as target values ​​for settings on certain processing machines, are developed or fine-tuned during the C-prototype construction.

[0052] In particular, training can begin even before series production starts, using training data that is already available at that point. This allows the machine learning model to be incrementally trained as new training data becomes available. Because the machine learning model only needs to be trained relatively infrequently, and because training can begin even before series production starts, it is readily available if problems suddenly arise during series production and a quick explanation is needed.

[0053] In a further advantageous embodiment, the trained machine learning model is used to evaluate causal relationships between different parameters with regard to the quality of the product, and / or at least one tolerance range for at least one parameter that is not critical with regard to the quality of the product.

[0054] For example, the fact that a value of a first parameter lies within a first range can, in itself, have a first effect. The fact that a value of a second parameter lies within a second range can, in itself, have a second effect. A causal relationship between the first and second parameters can then include, for example, the simultaneous occurrence of a value of the first parameter in the first range and a value of the second parameter in the second range having an effect that differs from the sum of the first and second effects. As mentioned earlier, for example, a borderline diameter of a bore into which a bolt is to be inserted, inaccurate manufacturing of this bolt, and a borderline temperature can interact in such a way that the bolt jams when inserted into the bore.

[0055] A tolerance range for a parameter can be understood, for example, as a range within which the parameter's value can vary without the product's quality deviating from a nominal state. For instance, if a hole is supposed to have a nominal diameter of 0.2 mm, the product may still be acceptable even if the hole actually has a diameter of 0.19 or 0.21 mm.

[0056] The relationships and tolerance ranges can be determined, for example, using the explainer, as described above. However, the relationships and tolerance ranges can also be determined directly from the quality predictions provided by the machine learning model. For instance, a quality prediction can be generated for a multitude of parameter combinations. If, for example, a combination of changes to several parameters leads to a change in the quality prediction based on such a combination, but this change does not occur if one of these changes is omitted, then a relationship between the parameters can be identified.

[0057] If, given a set of parameters, one parameter can be changed within a certain range without affecting the quality prediction, this range can be identified as the tolerance range for that parameter. Conversely, this approach can reveal that even a small change in a specific parameter leads to a change in the quality prediction. This parameter can then be identified as one on which the product quality is particularly critically dependent.

[0058] The trained machine learning model can therefore serve as a "digital twin" of the manufacturing process, allowing the effects of parameter changes to be studied much faster than on the manufacturing process itself.

[0059] The understanding gained by the machine learning model can therefore be used directly to improve the reliability and cost-effectiveness of the manufacturing process.

[0060] The trained machine learning model provides, for example, a well-motivated method for defining tolerances. In particular, form and position tolerances can be defined as tightly as possible. This simplifies manufacturing and reduces production costs, which increase disproportionately with more demanding tolerances.

[0061] Further knowledge can be extracted from the trained model and thus detached from the specific production line. For example, the realization that the product's quality depends particularly critically on certain parameters can stimulate further development of the product and / or the manufacturing process in such a way that this dependency is less critical. The product can then be manufactured with less effort and / or a lower reject rate.

[0062] Furthermore, the extracted knowledge also facilitates the duplication of a production line at the same or a different location. Particularly heavy and / or bulky parts in the automotive supply business, such as vehicle batteries, are often not shipped centrally from one location worldwide, but rather manufactured decentrally. In this case, it is crucial that the product behaves exactly the same regardless of which plant it originates from.

[0063] Advantageously, an XGBoost model, a Support Vector Machine, and / or an Explainable Boosting model are chosen as the machine learning model. These types of models work particularly well with the previously described explainers based on LIME or Shapley values.

[0064] The methods can be wholly or partially computer-implemented. Therefore, the invention also relates to a computer program with machine-readable instructions which, when executed on one or more computers as part of a quality assurance system, cause the quality assurance system and the computer(s) to execute one of the described methods. In this sense, vehicle control units and embedded systems for technical devices, which are also capable of executing machine-readable instructions, are also to be considered computers.

[0065] The invention also relates to a machine-readable data carrier and / or a downloadable product containing the computer program. A downloadable product is a digital product that can be transmitted over a data network, i.e., downloaded by a user of the data network, and which can, for example, be offered for immediate download in an online shop.

[0066] Furthermore, a computer can be equipped with the computer program, the machine-readable data carrier, or the download product as part of a quality assurance system.

[0067] Further measures improving the invention are described in more detail below, together with a description of preferred embodiments of the invention, with reference to figures. Examples of implementation

[0068] They show: Figure 1 Exemplary embodiment of method 100 for quality control of a mass-produced product 1; Figure 2Exemplary implementation of method 200 for training the machine learning model 2.

[0069] Figure 1 is a schematic flowchart of an embodiment of method 100 for quality control of a mass-produced product 1.

[0070] In step 110, a large number of parameters 11, which characterize the manufacturing process, are recorded during the production of product 1. According to block 111, between 50 and 10,000, preferably between 100 and 2,000, parameters 11 can be recorded. According to block 112, the parameters 11 can be recorded while product 1 undergoes between 10 and 200, preferably between 50 and 150, manufacturing steps.

[0071] In step 120, product 1 undergoes a physical inspection. This physical inspection results in a quality assessment 12.

[0072] In step 130, it is checked whether this quality assessment 12 meets a predefined criterion, such as "quality grade worse than satisfactory". If this is the case (truth value 1), in step 140 the parameters 11 recorded during the production of product 1 are fed to a trained machine learning model 2 and mapped by this trained machine learning model 2 to a quality prediction 13 for product 1.

[0073] In step 150, it is then checked whether the quality prediction 13 is consistent with the quality assessment 12. If this is the case (truth value 1), the recorded parameters 11 are fed to an explainer 21 for the machine learning model 2 in step 160. This explainer 21 assigns quantitative contributions 14 to individual recorded parameters 11, and / or combinations of the recorded parameters 11, to the quality prediction 13. If, on the other hand, the quality prediction 13 is not consistent with the quality assessment 12 (truth value 0 in step 150), the model can be retrained in an improved form and / or with the inclusion of further parameters.

[0074] According to Block 105, the Machine Learning Model 2 can be, for example, an XGBoost model, a Support Vector Machine, and / or an Explainable Boosting model.

[0075] According to Block 161, for example, an explainer 21 can be chosen that includes an approximation of the machine learning model 2. This approximation replicates the behavior of the machine learning model 2 at least locally and is easier to interpret than the machine learning model 2 itself. In particular, according to Block 161a, for example, an explainer 21 can be chosen that provides at least one Local Interpretable Model-Agnostic Explanation (LIME) of the quality prediction 13.

[0076] According to Block 162, for example, an explainer 21 can be chosen who is trained to determine, starting from sequences of parameters 11 in which a specific parameter 11' or a specific combination of parameters 11' does not occur, the marginal contribution to the quality prediction achieved by adding the specific parameter 11' or the specific combination of parameters 11'. In particular, according to Block 162a, for example, an explainer 21 can be chosen who is trained to determine Shapley values ​​for the specific parameter 11' or for the specific combination of parameters 11' with respect to the machine learning model 2 or a conditional expected value thereof.

[0077] In step 170, a probable, most probable, and / or likely cause 15 for the quality assessment 12 obtained during inspection 120 is evaluated from the quantitative contributions 14. Optionally, in step 180, a proposal 16 for a change to the process control of the manufacturing process for product 1 can also be evaluated from the quantitative contributions 14 and / or from the previously evaluated probable, most probable, and / or likely cause 15. This change is designed such that, after its implementation in the manufacturing process, products 1 manufactured during physical inspection 120 are closer to the specified reference than products 1 manufactured before the implementation of this change.

[0078] Figure 2Figure 1 is a schematic flowchart of an embodiment of method 200 for training the machine learning model 2 for the quality control of a mass-produced product 1. This machine learning model 2 can be used in particular in the previously described method 100.

[0079] In step 210, a multitude of parameters 11 are provided that characterize the manufacturing process of product 1 and were recorded during the production of numerous units of product 1. According to block 211, parameters can be selected, for example, that were at least partially recorded during a prototype construction phase of product 1 prior to the start of series production.

[0080] In step 220, the parameters 11 are preprocessed and searched for unusual changes based on at least one predefined criterion. If such changes are detected (truth value 1), they are removed in step 230, and / or an explanation for each unusual change is requested from an operator in step 240 and added to the parameters 11.

[0081] The pre-processed parameters 11 are mapped in step 260 by the machine learning model 2 to quality predictions 13 for the respective examples of product 1. For each of these examples of product 1, a quality measure 13# is further provided in step 250, which is recorded on the basis of at least one physical observation of this example of product 1, and / or on the basis of at least one physical functional test of this example of product 1, during or after production.

[0082] According to Block 205, the Machine Learning Model 2 can be, in particular, an XGBoost model, a Support Vector Machine, and / or an Explainable Boosting model.

[0083] In step 270, the quality predictions 13 are compared with the corresponding quality measures 13#. Based on the result of this comparison 270, in step 280, model parameters 2a, which characterize the behavior of the machine learning model 2, are optimized with the aim that, with further processing of parameters 11 by the machine learning model 2, the quality predictions 13 will more closely approximate the quality measures 13#. The training can continue until any termination criterion is met. Such a termination criterion could, for example, include the fulfillment of a certain accuracy of the quality predictions 13, measured using test or validation data, or the difference between the quality prediction and the quality measure falling below a predefined threshold. The fully trained state of the model parameters 2a is designated with the reference symbol 2a*.

[0084] Optionally, in step 290, additionally, causal relationships 11* between different parameters 11, and / or at least one tolerance range 11** for at least one parameter 11, can be evaluated from the trained machine learning model.

Claims

1. Method (100, 200) for quality control of a mass-produced product (1), comprising the following steps: • a multiplicity of parameters (11) that characterize the manufacturing process are recorded (110) during manufacture of the product (1); • the product (1) is subjected to a control (120), this control being at least one physical observation of the product (1) and / or at least one physical function test on the product (1), and also a comparison of the result obtained during the observation and / or the function test with a specified reference, and this comparison being used to determine a quality assessment (12) of the product (1); • in response to this quality assessment (12) meeting a specified criterion (130), the parameters (11) recorded during manufacture of the product (1) are fed to a trained machine learning model (2) and mapped (140) from that trained machine learning model (2) to a quality forecast (13) for the product (1); • a check (150) is performed to ascertain whether the quality forecast (13) is in line with the quality assessment (12); • if this is the case, the recorded parameters (11) are fed (160) to an explainer (21) for the machine learning model (2), this explainer (21) assigning individual recorded parameters (11) and / or combinations of the recorded parameters (11) quantitative contributions (14) to the quality forecast (13); • the quantitative contributions (14) are used to evaluate (170) a probable and / or most probable cause (15) of the quality assessment (12) obtained during the control (120), wherein the machine learning model (2) is trained to forecast the quality of the mass-produced product (1) using the following steps: • a multiplicity of parameters (11) are provided (210) that characterize the manufacturing process of the product (1) and have been recorded during manufacture of each of a multiplicity of copies of the product (1); • the parameters (11) are searched (220) for conspicuous changes on the basis of at least one specified criterion; • in response to at least one conspicuous change being found, that conspicuous change is removed (230) and / or an explanation for the conspicuous change is requested by an operator and added (240) to the parameters (11); • for each copy of the product (1) for which parameters (11) have been provided, a quality measure (13#) acquired on the basis of at least one physical observation of that copy of the product (1) and / or on the basis of at least one physical function test on that copy of the product (1), during or after manufacture, is provided (250); • the provided parameters (11) are mapped (260) from the machine learning model (2) to quality forecasts (13) for the respective copies of the product (1); • the quality forecasts (13) are compared (270) with the corresponding quality measures (13#); • model parameters (2a) that characterize the behaviour of the machine learning model (2) are optimized (280) with the aim of the quality forecasts (13) coming closer to the quality measures (13#) with further processing of parameters (11) by the machine learning model (2).

2. Method (100, 200) according to Claim 1, wherein the parameters (11) that characterize the manufacturing process comprise • settings of at least one machine that processes the product (1) during the manufacturing process, and / or • a period of time that has elapsed since maintenance and / or adjustment was last carried out on at least one machine that processes the product (1) during the manufacturing process, and / or • a measure of a wear condition of at least one tool that comes into contact with the product (1) during the manufacturing process, and / or • measured values from the measurement of at least one physical measured variable on the product (1) being manufactured, on a precursor used for the manufacture and / or in the environment in which the product (1) is manufactured, and / or • at least one time stamp of at least one time at which at least one processing step has been carried out on the product (1).

3. Method (100, 200) according to either of Claims 1 to 2, wherein between 50 and 10 000, preferably between 100 and 2000, parameters (11) are recorded (111) during manufacture of the product (1).

4. Method (100, 200) according to one of Claims 1 to 3, wherein the parameters (11) are recorded (112), while the product (1) passes through between 10 and 200, preferably between 50 and 150, manufacturing steps.

5. Method (100, 200) according to one of Claims 1 to 4, wherein an explainer (21) is chosen (161) that comprises an approximation of the machine learning model (2) that at least locally simulates the behaviour of the machine learning model (2) and has a lower complexity than the machine learning model (2).

6. Method (100, 200) according to Claim 5, wherein an explainer (21) is chosen (161a) that provides at least one local interpretable model-agnostic explanation, LIME, of the quality forecast (13).

7. Method (100, 200) according to one of Claims 1 to 6, wherein an explainer (21) is chosen (162) that is configured to determine, from sequences of the parameters (11) in which a specific parameter (11') or a specific combination of parameters (11') does not occur, the particular marginal contribution to the quality forecast that is achieved by adding the specific parameter (11'), or the specific combination of parameters (11').

8. Method (100, 200) according to Claim 7, wherein an explainer (21) is chosen (162a) that is configured to determine Shapley values for the specific parameter (11'), or for the specific combination of parameters (11'), with respect to the machine learning model (2) or a conditional expectation value of the machine learning model.

9. Method (100, 200) according to one of Claims 1 to 8, wherein the quantitative contributions (14) and / or the evaluated probable and / or most probable cause (15) is / are used to additionally evaluate (180) at least one proposal (16) for a change in the process control of the manufacturing process after the implementation of which in the manufacturing process the products (1) that are then manufactured are expected to be closer to the specified reference during the control (120) than the products (1) that were manufactured before this change was implemented.

10. Method (100, 200) according to one of the preceding claims, wherein at least some parameters are chosen (211) that have been acquired in a prototyping phase of the product (1) before the start of mass production of the product (1).

11. Method (100, 200) according to one of the preceding claims, wherein the trained machine learning model (2) is used to evaluate (290) correlations (11*) between different parameters (11) and / or at least one tolerance range (11**) for at least one parameter (11).

12. Method (100, 200) according to one of the preceding claims, wherein an XGBoost model, a support vector machine and / or an explainable boosting model is chosen (105, 205) as the machine learning model (2).

13. Computer program containing machine-readable instructions that, when executed on one or more computers as part of a system for quality control, cause the system for quality control and the computer or computers to perform a method (100, 200) according to one of the preceding claims.

14. Machine-readable data carrier comprising the computer program according to Claim 13.

15. System for quality control, comprising one or more computers, equipped with the computer program according to Claim 13 and / or with the machine-readable data carrier according to Claim 14.