Methods and systems for evaluating printed circuit boards

Abstract

A computer-implemented method for evaluating at least one printed circuit board (PCB 1, PCB 2, PCB 3) includes: - receiving input data (ID), wherein the input data (ID) is based on test data (TD) of at least one printed circuit board (PCB 1, PCB 2, PCB 3), wherein the test data is test data from in-circuit testing and includes measurement data (MDB) of a plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3); - applying a trained classification function (TCF) to the input data (ID), wherein output data is generated; - providing the output data, wherein the output data includes an assignment (ASG) of at least one to one of at least two different categories of the plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3).

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for evaluating at least one printed circuit board, wherein the at least one printed circuit board is assigned to one of at least two different categories by means of a trained classification function.

[0002] Furthermore, the present invention also relates to a machine-readable data medium comprising at least one such allocation.

[0003] Furthermore, the present invention relates to a computer-implemented method for providing a trained classification function, which can be used in the aforementioned method for evaluating printed circuit boards.

[0004] Furthermore, the present invention also relates to a computer-implemented method for providing a trained recommendation function for printed circuit board testing, which can be used in the methods described above for evaluating printed circuit boards.

[0005] Furthermore, the present invention also relates to a computer program that includes instructions which, when executed by a system, cause the system to perform at least one of the computer-implemented methods described above.

[0006] Furthermore, the present invention also relates to a system for evaluating at least one printed circuit board. Background Technology

[0007] In the field of electronic device manufacturing, the manufacturing and assembly of printed circuit boards (PCBs) have a relatively high level of process maturity. Methods and systems for PCB testing are known in the prior art. For example, US Patent 6,738,450 B1 discloses a method and system for detecting PCB solder joints based on image processing. US 7,133,797 B2 relates to PCB functional testing based on predetermined behaviors of input / output signals.

[0008] However, it is still necessary to verify that the correct components are in the correct positions and that they are correctly electrically connected to the circuit board without short circuits or open circuits. Therefore, electrical testing is necessary. An example of such testing is measuring the electrical values ​​of each component either by direct contact or through test pads. Due to poor connections (e.g., contamination of the test pad or pins), the resulting measurements deviate from the expected range of component values, thus causing good products to be judged as defects (pseudo-errors). That is, the term "[electrical] poor connection" is the opposite of the term "ideal connection," which means that the connection itself has zero resistance and zero capacitance. In cases of poor electrical connections due to contamination (pins), dust (contacts), temperature (PCB), etc., the connection itself has non-zero capacitance and / or non-zero resistance. Therefore, poor electrical connections can lead to incorrect measurements of the electrical parameters of PCB electronic components (e.g., incorrect resistance and / or capacitance values), as well as pseudo-errors. Pseudo-errors may lead to retesting or additional quality control work, and thus reduce the overall capability and effectiveness of production equipment. False errors often occur randomly, or their underlying causes are different and constantly changing. Therefore, process optimization is either costly or unsustainable, while retesting is an acceptable approach.

[0009] Therefore, there is a need for methods and systems that can handle the large amounts of data generated during printed circuit board testing and are more reliable than current methods and systems. Summary of the Invention

[0010] To achieve the above objectives, the present invention provides a computer implementation method for evaluating at least one printed circuit board, the method comprising:

[0011] - Receive input data, wherein the input data is based on test data of at least one, more specifically, at least one printed circuit board, wherein the test data is in-circuit test data and includes measurement data of multiple electronic components of the at least one printed circuit board;

[0012] - The trained classification function is applied to the input data, where the output data is generated;

[0013] - Provide the output data, wherein the output data includes the allocation of at least one of a plurality of electronic components of the at least one printed circuit board, preferably each to one of at least two different categories.

[0014] In an embodiment, the output data includes multiple assignments, each of which is an assignment of at least one of a plurality of electronic components of the at least one printed circuit board, and different assignments involve different electronic components.

[0015] An evaluation of the at least one printed circuit board is performed based on the output data, which is based on the allocation of at least one of a plurality of electronic components of the at least one printed circuit board. Specifically, the output data includes a plurality of allocations, each of which relates to a specific electronic component, and different electronic components have different allocations.

[0016] Using a trained classification function, higher accuracy can be achieved when assigning specific features to a tested printed circuit board.

[0017] In an embodiment, the output data can be implemented as input data enhanced by a corresponding assignment, i.e., labeled or categorized input data.

[0018] In some embodiments, the trained classification function may be based on a multivariate classification algorithm, more specifically, the multivariate classification algorithm may be a regular linear regression algorithm, a random forest algorithm, a gradient boosting algorithm, a LASSO algorithm, specifically an adaptive LASSO algorithm, more specifically a regularized linear regression algorithm, or a logistic regression algorithm, specifically a binary logistic regression algorithm.

[0019] In some embodiments, one of the at least two categories corresponds to the type of spurious fault printed circuit board. A spurious fault printed circuit board is a printed circuit board with a spurious error. That is, at least one of the multiple electronic components of the at least one printed circuit board has a spurious error (“false defect”). A spurious fault printed circuit board refers to a normally functioning printed circuit board that fails to pass a test due to one or another external condition (e.g., temperature of the printed circuit board, test pad contamination, etc.). Using a trained classification function can improve the detection accuracy of spurious fault printed circuit boards.

[0020] In other words, the trained classification function can be adjusted or trained to detect false errors in electronic components. The following section will further discuss how to train the classification function, for example, based on historical ICT measurement data and corresponding historical assignment data (e.g., assignment data from human experts). The trained classification function can subtract “noise” from the measurement data, which is introduced during ICT measurements due to one or more of the aforementioned reasons (e.g., probe contamination, printed circuit board “heating”, etc.).

[0021] In some embodiments, the at least one printed circuit board, more specifically each printed circuit board, is assigned to a group consisting of three categories, more specifically, to a group consisting of: a first category (“pass”), wherein the first category corresponds to the type of printed circuit boards that are operating normally; a second category (“fail”), wherein the second category corresponds to the type of printed circuit boards that are faulty; and a third category (“pseudo-fault”), wherein the third category corresponds to the type of printed circuit boards that are pseudo-faulty.

[0022] Retesting is costly in ICT scenarios, so detecting false errors can significantly reduce production costs in such situations.

[0023] In some embodiments, the measurement data of the plurality of electronic components of the at least one printed circuit board includes measurement data of at least one electrical parameter of each of the plurality of electronic components of the at least one printed circuit board. The electrical parameter may be an analog and / or digital parameter. For example, the test may include measurements of resistance, inductance, etc., i.e., the measurement data may include (different) electrical parameters of individual electronic components of the at least one printed circuit board. As mentioned above, measurements of individual electronic components (e.g., resistors) may result in different electrical parameters (resistance, inductance) being non-zero, for example, due to poor connections.

[0024] In an embodiment, the input data and the output data may include layout information of the at least one printed circuit board, and the output data includes measurement data of at least one electrical parameter of each of a plurality of electronic components of the at least one printed circuit board, wherein the visualization is based on or performed by means of the layout information.

[0025] In some embodiments, multiple different types of printed circuit boards can be tested, and the input data and the output data may include layout information for at least one type of printed circuit board. In this embodiment, the input data and the output data may include layout information for each type of printed circuit board.

[0026] In some embodiments, the method may include optimizing at least one statistical parameter, wherein the statistical parameter determines a predetermined allowable range of values ​​for at least one electrical parameter of each of a plurality of electronic components of the at least one printed circuit board, and providing output data may include recommending at least one optimized statistical parameter for use in further testing.

[0027] In some embodiments, this optimization can be based on recently acquired test data, i.e., recent historical data. For example, this recent historical data could be data collected during the current production cycle or / and data collected within the last two, three, or four weeks. In this way, the allowable range of the value can be adjusted according to the laboratory conditions of the test.

[0028] The statistical parameter can be a location parameter, such as the mean, and / or a permutation parameter, such as the standard deviation, variance, range, interquartile range, absolute deviation, etc. These parameters can characterize (measure) the degree of statistical dispersion.

[0029] For example, one could assess the average of a specific capacity over a period of time. For instance, one could assess the average of measurements of that capacity over the past four weeks and suggest an adjusted confidence interval based on that.

[0030] Recommendations can be made in visualizations; for example, test data enhanced by one or more assignments can be visualized to display the allowed and recommended values ​​on, for example, a screen or HMI.

[0031] In some embodiments, recommendations can be performed using a trained recommendation function, more specifically, a trained recommendation function based on data-driven optimization, particularly based on distributed robust optimization and / or online linear programming algorithms and / or non-convex regularized least squares and / or multi-block alternating direction multiplier methods. In this way, the test data evaluation process can be fully automated.

[0032] According to one aspect of the invention, a machine-readable data medium comprising the above-described allocation is provided.

[0033] According to one aspect of the present invention, a computer-implemented method is provided for providing a trained classification function. The trained classification function can be used in the aforementioned method for evaluating printed circuit boards. The training method includes:

[0034] A0) Receives training input data and training output data, wherein the training input data represents test data for online testing, wherein the ICT test data includes (historical) measurement data of a plurality of electronic components of the at least one printed circuit board, and the training output data represents an allocation (historical allocation data) of at least one of the plurality of electronic components of the printed circuit board, wherein each allocation is an allocation of at least one of the plurality of electronic components to one of at least two different categories.

[0035] B0) Perform a classification function on the training input data to generate predicted training output data;

[0036] C0) compares the training output data with the predicted training output data to determine the error;

[0037] D0) Update the classification function based on the determined error, and

[0038] E0) performs steps A0) to D0) iteratively to reduce the error.

[0039] Different allocations correspond to different electronic components.

[0040] The training input and output data include historical data, namely data obtained from previous ICT measurements (training measurement data) and data obtained from the corresponding allocation of electronic components on the printed circuit board (training allocation data).

[0041] The training input data and training output data constitute what is called the training dataset. This training dataset can be a part of the entire test data.

[0042] In some embodiments, the training dataset may include approximately or less than 20% of all test data. In this way, the probability of overfitting the classification function can be reduced.

[0043] In some embodiments, the training dataset in each iteration of steps A0 to D0 may be selected to have no overlap or only partial overlap with the training dataset previously selected from all training data.

[0044] In some embodiments, the training input data and the training output data may be preprocessed before the classification function is executed.

[0045] In some embodiments, the method may include cross-validation.

[0046] In some embodiments, the classification function may be based on a multivariate classification algorithm, more specifically, the multivariate classification algorithm is a common linear regression algorithm, a random forest algorithm, a gradient boosting algorithm, a LASSO algorithm, specifically an adaptive LASSO algorithm, more specifically a regularized linear regression algorithm, or a logistic regression algorithm, specifically a binary logistic regression algorithm.

[0047] In this embodiment, one of the at least two categories corresponds to the pseudo-fault category.

[0048] In some embodiments, the at least one printed circuit board may be assigned to a group consisting of three categories, more specifically, to a group consisting of: a first category, wherein the first category corresponds to the type of normally functioning printed circuit boards; a second category, wherein the second category corresponds to the type of faulty printed circuit boards; and a third category, wherein the third category corresponds to the type of pseudo-faulty printed circuit boards. In this case, the feature vector has three entries, corresponding to the categories of "pass," "fail," and "pseudo-error," respectively.

[0049] According to one aspect of the present invention, a computer-implemented method is provided for providing a trained recommendation function for printed circuit board testing, wherein the method includes:

[0050] A1) Receive training input data and training output data, wherein the training input data represents a recommendation for at least one optimized statistical parameter, wherein the (not yet optimized) statistical parameter determines a predetermined range of values ​​for at least one electrical parameter of each of a plurality of electronic components of the at least one printed circuit board, wherein the predetermined range of values ​​includes values ​​allowed during printed circuit board testing, and the training output data represents acceptance of the recommendation.

[0051] B1) Execute the recommendation function on the training input data to generate the predicted training output data;

[0052] C1) Compare the training output data with the predicted training output data to determine the error;

[0053] D1) Update the recommendation function based on the determined error, and

[0054] E1) Perform steps A1) to D1) iteratively to reduce the error.

[0055] The (unoptimized) statistical parameters can be provided by human experts.

[0056] This (not yet optimized) statistical parameter does not depend on and / or take into account the conditions of the current test procedure.

[0057] In some embodiments, the recommendation function may be based on data-driven optimization, particularly distributed robust optimization and / or online linear programming algorithms and / or non-convex regularized least squares and / or multi-block alternating direction multiplier method (ADMM).

[0058] According to one aspect of the present invention, a computer program is provided, wherein the computer program includes instructions that, when executed by a system, cause the system to perform the computer-implemented method described above.

[0059] According to one aspect of the present invention, a system is provided, the system comprising:

[0060] - A first interface is configured to receive input data, wherein the input data is based on test data of the at least one printed circuit board, wherein the test data is in-circuit test data and includes measurement data of multiple electronic components of the at least one printed circuit board;

[0061] - A computing device is configured to apply a trained classification function to the input data to generate output data;

[0062] - A second interface is configured to provide the output data, wherein the output data includes the allocation of at least one of a plurality of electronic components of the at least one printed circuit board to one of at least two different categories.

[0063] In some embodiments, the computing device or the second interface may be configured to:

[0064] - Optimize at least one statistical parameter, wherein the statistical parameter determines a predetermined range of permissible values ​​for at least one electrical parameter of each of a plurality of electronic components of the at least one printed circuit board, wherein

[0065] The computing device or the second interface is configured to:

[0066] - Along with providing this output data, we recommend, more specifically, at least one optimized statistical parameter for use in further testing by applying a trained recommendation function.

[0067] According to one aspect of the invention, a training system for providing a trained classification function is provided, wherein the training system includes a first training interface configured to receive training input data and training output data, wherein the training input data represents test data from an online testing (ICT) test. The ICT test data includes measurement data of a plurality of electronic components of the at least one printed circuit board, and the training output data represents an assignment of at least one of the plurality of electronic components, wherein each assignment is an assignment of the printed circuit board to one of at least two different categories; a training computing device is configured to:

[0068] *Execute a classification function on the training input data to generate predicted training output data;

[0069] * Compare the training output data with the predicted training output data to determine the error;

[0070] * The classification function is updated based on the determined error; and a second training interface is configured to provide the trained classification function.

[0071] According to one aspect of the present invention, a training system is provided for providing a trained recommendation function for printed circuit board testing, wherein the training system comprises:

[0072] - A first training interface is configured to receive training input data and training output data, wherein the training input data represents a recommendation of at least one optimized statistical parameter, wherein the statistical parameter determines a predetermined range of values ​​for at least one electrical parameter of each of a plurality of electronic components of the at least one printed circuit board, wherein the predetermined range of values ​​includes values ​​allowed during printed circuit board testing, and the training output data represents acceptance of the recommendation;

[0073] - The training computing device is configured to:

[0074] *Execute the recommendation function on the training input data to generate the predicted training output data;

[0075] * Compare the training output data with the predicted training output data to determine the error;

[0076] * Update the recommendation function based on the determined error, and

[0077] - The second training interface is configured to provide the trained recommendation function. Attached Figure Description

[0078] The above and other objects and advantages of the present invention will become apparent from the following specific embodiments taken in conjunction with the accompanying drawings, in which like reference numerals refer to like parts throughout:

[0079] Figure 1 This is an example of an environment where test data from multiple printed circuit boards can be evaluated;

[0080] Figure 2 This is a flowchart illustrating an example of a computer-implemented method for evaluating at least one printed circuit board;

[0081] Figure 3 This is an example of visualizing PCB layout information;

[0082] Figure 4 This is a flowchart illustrating an example of a computer-implemented method for evaluating at least one printed circuit board and for visualizing the results;

[0083] Figure 5 It is an example of an environment that can evaluate test data from multiple printed circuit boards and suggest visualizations and optimization parameters for testing;

[0084] Figure 6 and Figure 7 This is a flowchart of a method for producing machine-readable media;

[0085] Figure 8 These are examples of methods for providing computer-based implementations of trained classification functions;

[0086] Figure 9 This is an example of data preprocessing;

[0087] Figure 10 These are examples of computer-implemented methods for providing trained recommendation functions;

[0088] Figure 11 These are examples of training systems used to provide trained functions, such as trained classification functions and / or trained recommendation functions. Detailed Implementation

[0089] Turning Figure 1 and Figure 2 An example of an environment in which test data of several printed circuit boards PCB 1, PCB 2, and PCB 3 can be evaluated is shown, along with an example of a computer-implemented method that can be performed in this environment.

[0090] In particular, Figure 1An industrial plant (PL) or manufacturing facility for manufacturing printed circuit boards (PCBs) is shown. Those skilled in the art will understand that the number of PCBs 1, 2, and 3 may be more than three, but only three are shown. Typically, there are hundreds or sometimes thousands of PCBs in a PCB manufacturing facility (PL). Each PCB 1, 2, and 3 undergoes a quality testing procedure before leaving the manufacturing plant (PL). During this testing, test data is generated. The testing performed on PCBs 1, 2, and 3 is in-circuit testing (ICT). Therefore, the test data for PCBs 1, 2, and 3 are ICT measurement data. During ICT, measurement data is obtained for multiple electronic components of each PCB 1, 2, and 3, and in particular for each individual electronic component. For example, during ICT measurement, one or more electrical parameters of each of these multiple electronic components are measured.

[0091] In-circuit testing of each printed circuit board PCB 1, PCB 2, PCB 3 can be performed by one or more test machines TM 1, TM 2, TM 3. Figure 1 As shown, each printed circuit board (PCB 1, PCB 2, PCB 3) can be associated with a specific test machine (TM 1, TM 2, TM 3) that is only responsible for testing that particular PCB. However, those skilled in the art will understand that all PCBs can be tested, for example, by a single test machine. For this purpose, the PCBs can be positioned, for example, on a conveyor belt that can transport the PCBs to the ICT machine for testing and remove the tested PCBs from the machine.

[0092] All printed circuit boards (PCBs 1, PCB 2, and PCB 3) can be of the same type. Many different types of printed circuit boards can also be tested.

[0093] After testing, the ICT test data TD (including measurement data MDB of multiple electronic components on printed circuit boards PCB 1, PCB 2, and PCB 3) can be forwarded to the system, where the ASG of printed circuit boards PCB 1, PCB 2, and PCB 3 is assigned. The test data can be preprocessed in some way (e.g., via a preprocessor PP) before arriving at the system as input data ID, or it can be forwarded as raw data.

[0094] This input data ID is based on the ICT test data TD of printed circuit boards PCB 1, PCB 2, and PCB 3.

[0095] The system may include a first interface IF 1, configured to receive the input data ID. The system may also include a computing unit CU, configured to apply a trained classification function TCF to the input data ID to generate output data. Figure 1 As shown, the trained classification function TCF can be stored on the computing device CU. In other embodiments, the trained classification function TCF can be stored elsewhere, such as in the cloud or in any other repository that can be remotely accessed from the computing device.

[0096] In some embodiments, the first interface IF 1 may be part of the gateway component GW. Figure 1 The trained classification function TCF can be stored on the gateway component GW. Figure 1 The gateway component (GW) can be part of the computing unit (CU). Figure 1 ).

[0097] The system may also include a second interface IF 2, configured to provide the output data. This output data includes the assignment ASG of individual electronic components of the tested printed circuit boards PCB 1, PCB 2, and PCB 3 to at least one of two different categories.

[0098] This system can be implemented as an edge device.

[0099] Turn Figure 2 This shows the method used for evaluation Figure 1 An example of a computer-implemented method for testing printed circuit boards PCB 1, PCB 2, and PCB 3. The method may include:

[0100] Step S1: Receive input data ID, wherein the input data ID is based on test data TD of printed circuit boards PCB 1, PCB 2, and PCB 3, wherein the test data is test data of online testing and includes measurement data MDB of multiple electronic components of at least one printed circuit board PCB 1, PCB 2, and PCB 3;

[0101] Step S2: Apply the trained classification function TCF to the input data ID to generate output data;

[0102] Step S3: Provide the output data, wherein the output data includes the assignment ASG of the electronic components of each printed circuit board PCB 1, PCB 2, PCB 3 to one of at least two different categories. In some embodiments, the output data may be implemented as input data ID enhanced by the corresponding assignment ASG, such as tag or categorized input data.

[0103] Steps S1 to S3 can be performed Figure 1This method is executed in the environment shown. In particular, it can be executed by reference. Figure 1 The system execution described.

[0104] In some embodiments, one of the at least two categories corresponds to a type of spurious fault printed circuit board. That is, at least one of the plurality of electronic components of the at least one printed circuit board has a spurious error (“false defect”).

[0105] The category of "pseudo-fault printed circuit boards" includes printed circuit boards with only pseudo-faults. This means that these are normally functioning printed circuit boards that failed testing, for example, due to an overall deviation in measurement statistics caused by an increase in the temperature of the printed circuit board during testing. The term "pseudo-fault" is widely known in the field of printed circuit board testing.

[0106] In some embodiments, each printed circuit board PCB 1, PCB 2, and PCB 3 is assigned to a group consisting of three categories, more specifically, to a group consisting of: a first category, corresponding to the type of normally functioning printed circuit boards; a second category, corresponding to the type of faulty printed circuit boards; and a third category, corresponding to the type of pseudo-faulty printed circuit boards. In this case, the feature vector has three entries, for example, "pass" for normally functioning printed circuit boards, "fail" for faulty printed circuit boards, and "pseudo-error" for normally functioning printed circuit boards with at least one pseudo-error.

[0107] In some embodiments, the trained classification function TCF may be based on a multivariate classification algorithm, more specifically, the multivariate classification algorithm is a common linear regression algorithm, a random forest algorithm, a gradient boosting algorithm, a LASSO algorithm, especially an adaptive LASSO algorithm, more specifically a regularized linear regression algorithm, or a logistic regression algorithm, especially a binary logistic regression algorithm.

[0108] Various parameters of printed circuit boards PCB 1, PCB 2, and PCB 3 can be measured, allowing ICT test data TD to include measurement data for multiple electrical parameters. For example, for multiple electronic components on each of PCB 1, PCB 2, and PCB 3, and especially for each electronic component, at least one electrical parameter can be measured. As mentioned above, in the presence of contamination leading to poor connections, the measured resistance (electronic component) value will have a non-zero resistance value (first electrical parameter) and a non-zero capacitance (second electrical parameter), which will impair the measured value.

[0109] In some embodiments, analog and / or digital electrical parameters of printed circuit boards PCB 1, PCB 2, and PCB 3 can be measured.

[0110] In some embodiments, the measurement may include measurements of all electronic components of printed circuit boards PCB 1, PCB 2, and PCB 3, such as resistance, inductance, and capacitance.

[0111] In some embodiments, the input data ID and output data include layout information for the printed circuit boards PCB 1, PCB 2, and PCB 3 under test. For example, if PCB 1, PCB 2, and PCB 3 of the same type are being tested, they have the same layout. It would be advantageous to visualize the measurement data MDB, and this visualization would utilize this layout information. This visualization can be performed simultaneously with the assignment ASG for a specific PCB 1, PCB 2, or PCB 3.

[0112] In some embodiments, this visualization can be performed, for example, via a human-machine interface (HMI). Such an HMI may be part of a second interface IF 2 of a computing device (CU) and connected to an industrial equipment PL (…). Figure 1 (A cloud, not shown, is located in the same location.) An example of this type of visualization is... Figure 3 As shown, hollow squares ES represent the locations on the printed circuit board that have passed the test, meaning the corresponding electronic components have passed the test, while solid squares FS represent the locations on the printed circuit board that have failed the test, meaning the corresponding electronic components have failed the test.

[0113] In this context, when measuring various parameters of each printed circuit board, the assigned ASG for a particular printed circuit board can be based on multiple individual assignments, where each individual assignment is an assignment of a specific parameter of that printed circuit board to one of at least two, more specifically, three different categories. For example, if all the individual assignments are “pass”, then the overall assigned ASG is also “pass”.

[0114] It should be understood that, in order to pass the test, the measured value of each parameter should be within a predetermined allowable range. This predetermined range can be determined based on statistical values / parameters. For example, this predetermined range can be obtained from the parameter's nominal value (e.g., the average nominal value) and confidence intervals (e.g., confidence intervals of σ, 2σ, 3σ, 4σ, or 5σ), where σ is the standard deviation.

[0115] Due to various interfering factors, such as different temperatures of printed circuit boards PCB 1, PCB 2, and PCB 3 during testing, and residual contaminants after production, the predetermined range of values ​​may not accurately describe the actual performance of the parameter. For example, the nominal value of a resistor can be determined as R0 at room temperature (i.e., approximately 25°C). However, during measurement, the temperature of the printed circuit board under test may be higher (e.g., 50°C to 60°C or higher) or lower than room temperature. Because of this, the predetermined nominal value of the resistor may not match the nominal value in the actual production environment. This can lead to a shift in the probability distribution describing the parameter measurement relative to the predetermined nominal probability distribution. The parameter measurements of printed circuit boards PCB 1, PCB 2, and PCB 3 can also be described using statistical parameters, such as positional parameters and / or arrangement parameters. Examples of such statistical parameters are standard deviation, variance, range, interquartile range, absolute deviation, etc.

[0116] Therefore, the test data TD of printed circuit boards PCB 1, PCB 2, and PCB 3 may include a predetermined allowable range of values ​​for the electrical parameters of the electronic components on the printed circuit boards, as well as the measured values ​​of these electrical parameters, the distribution of which can be compared with the predetermined allowable range of values.

[0117] In state-of-the-art testing, this range of values ​​is specified by human experts and test engineers, not based on recent measurements, but rather selected according to design rules or experience, and adjusted based on process data during the introduction of new products.

[0118] It should be understood that "recently performed measurement" refers to measurements taken, for example, in the most recent production cycle. For instance, the most recent four-week measurement could be a measurement taken within the last four weeks.

[0119] Turn Figure 4 The document describes an embodiment in which the method further includes step S2A: optimizing at least one predetermined statistical parameter that defines the allowable range of values ​​of parameters of the at least one electronic component of the printed circuit board to obtain an optimized value of the at least one predetermined statistical parameter, such as mean, deviation, variance, range, interquartile range, absolute deviation, etc., and providing output data. Step 3 includes sub-step S3-1: recommending at least one optimized predetermined statistical parameter for use in further testing.

[0120] The above optimizations can be performed based on recent historical test data.

[0121] In some embodiments, optimization can be performed based on a heuristic mathematical model. For example, an optimized nominal value for a parameter (e.g., resistance) can be calculated based on parameter values ​​measured over the past four weeks on a normally functioning printed circuit board (i.e., a tested printed circuit board). This optimized nominal value could be, for example, an average value. Upper and lower bounds of the confidence interval can be calculated, such as 5σ.

[0122] In some embodiments, the recommendation can be performed using a trained recommendation function (TRF), more specifically, the trained recommendation function is based on data-driven optimization, particularly distributed robust optimization and / or online linear programming algorithms and / or non-convex regularized least squares and / or multi-block alternating direction multiplier method (ADMM).

[0123] For example, on the second interface, the recommended optimized values ​​of predefined statistical parameters can be visualized.

[0124] Turning Figure 5 This example demonstrates an environment that can evaluate test data from multiple printed circuit boards and can suggest visualizations and optimization parameters for testing.

[0125] and Figure 1 Compared to the computing device CU, Figure 5 The computing device CU' can be further configured to optimize statistical parameters to obtain optimized statistical parameters OPT (values). For this purpose, the computing device CU' may include other hardware and / or software components. Figure 5 An embodiment of a computing device CU' including, for example, other software components ASC that may be located in a public network PN (e.g., in the cloud) is shown.

[0126] The other hardware and / or software components may include a Model of Determination (MOD) for optimizing one or more nominal values ​​of parameters in testing based on the latest data (e.g., data from the same production cycle). For example, the model could be a heuristic mathematical model that evaluates the average value of each measurement parameter based on measurement data from a normally functioning printed circuit board. The model may also optimize other statistical parameters of the measurement parameters, such as (allowable) deviation, variance, range, interquartile range, absolute deviation, etc.

[0127] The second interface IF2' can be configured to recommend the statistical parameter OPT for this optimization. For example, the second interface IF2' can suggest using the average value calculated above, instead of the nominal value and / or optimization allowable value of the deviation, variance, range, interquartile range, absolute deviation, for use in further testing.

[0128] For example, the optimized statistical parameter OPT can be calculated in the cloud PN and forwarded to the industrial equipment PL ( Figure 5 (As indicated by the middle arrow). It should be understood that the optimized statistical parameter OPT can be achieved through the gateway component GW or through another gateway component. For example, the second interface IF2' may include other gateway components for receiving data directly from the cloud PN.

[0129] For example, the second interface IF2' may include a visualization device, such as a human-machine interface (HMI). The HMI' may be configured to visualize the optimized statistical parameters OPT (and the allocation ASG). The HMI' may also be configured to receive the optimized statistical parameters OPT directly from the cloud PN.

[0130] Other software components, such as the ASC, can also be located in the same place as the industrial equipment PL and stored on, for example, the gateway component GW.

[0131] The trained recommendation function TRF, which is used for the statistical parameters OPT for recommendation / suggestion optimization, can be stored on the second interface IF2'. More specifically, the trained recommendation function TRF can be stored on the human-machine interface HMI'. Figure 5 )superior.

[0132] It should be understood that the trained classification function TCF and / or the predetermined statistical parameters can be remotely applied to obtain optimized statistical parameters OPT and / or the trained recommendation function TRF can be applied. All of these operations can be performed in the cloud, thereby enabling the final product of this method (i.e., the assignment ASG, which can be optionally enhanced by recommending optimized statistical parameters OPT) to be produced off-site, rather than in industrial equipment PL.

[0133] In particular, Figure 6 A method for generating a machine-readable data medium is shown, the method comprising:

[0134] Step P1: Provide a machine-readable data medium;

[0135] Step P2: As described above, assign at least one of the multiple electronic components of the at least one printed circuit board PCB 1, PCB 2, PCB 3 to one of at least two categories, and

[0136] Step P3: Store the allocation ASG on a machine-readable data medium.

[0137] More specifically, the step of providing the allocation of ASG may also include providing at least one optimized statistical parameter OPT for further testing— Figure 7 Step P2' in the process, and the step of storing the allocated ASG on a machine-readable data medium, may include storing at least one optimized statistical parameter OPT for further testing on the machine-readable data medium. Figure 7 Step P3' in the text.

[0138] In view of the above, Figure 8An example of a computer-implemented method for providing a trained classification function TCF is shown. The method may include step A0: receiving training input data and training output data, wherein the training input data represents test data TD for online testing, wherein the ICT test data TD includes measurement data MDB of multiple electronic components of printed circuit boards PCB 1, PCB 2, and PCB 3, and the training output data represents an assignment ASG of at least one of the multiple electronic components, wherein each assignment ASG is an assignment of at least one of the multiple electronic components to one of at least two different categories. The method may further include step B0: performing the classification function on the training input data to generate predicted training output data; step C0: comparing the training output data with the predicted training output data to determine an error; and step D0: updating the classification function based on the determined error. For example, if the determined error is unsatisfactory, steps A0 to D0 may be repeated iteratively to reduce the error (model fitting); otherwise, the method may be terminated.

[0139] The comparison can be performed using a confusion matrix—step C0.

[0140] The classification function can be based on a multivariate classification algorithm, such as a regular linear regression algorithm, a random forest algorithm, a gradient boosting algorithm, a LASSO algorithm, especially an adaptive LASSO algorithm, more specifically a regularized linear regression algorithm, or a logistic regression algorithm, especially a binary logistic regression algorithm.

[0141] This method can be performed using different classification functions, with the final results compared after fitting is complete. In this way, the training method can include performing steps A0 to D0 (e.g., in parallel) for different classification functions, and selecting the best-performing model / algorithm after training.

[0142] In some embodiments, the method may further include a cross-validation step.

[0143] In some embodiments, the method may include data preprocessing that can be performed prior to step A0. Preprocessing of the training data may also be performed, for example, by a preprocessor PP on the test data TD.

[0144] The sub-step of cross-validation—the step of splitting the entire dataset into training and validation datasets—can be part of this type of preprocessing. The training dataset consists of training input data and training output data. Such a training dataset can be test data (i.e., training input data) from a subset of multiple printed circuit boards (e.g., only PCB 1), augmented by corresponding assignments (i.e., training output data).

[0145] Therefore, the training dataset can constitute a portion of the entire dataset, thus reducing the probability of overfitting the model. Particularly good results can be obtained if the training dataset comprises approximately 20% of the entire dataset. Further improvements can be achieved by performing cross-validation and always selecting a new training dataset from the entire dataset. Even better results can be obtained if the new training dataset does not overlap with any older training datasets. In this way, the training data is always "new".

[0146] Go to Figure 9 The example shown is a data preprocessing technique. This data preprocessing is applicable to both training and test data (TD).

[0147] First, data is received from one or more data sources. Figure 9 An embodiment with three data sources is shown: the printed circuit board measurement data MDB (raw data), such as data from ICT measurements; the allocation data ADB, which is based on the allocation of electronic components on the printed circuit board to different categories (e.g., two or three different categories) (e.g., performed by human experts or machine learning algorithms); and the printed circuit board layout information LDB (e.g., for further visualization of the results).

[0148] Measurement data MDB can be provided, for example, as a .csv file and can contain millions of entries, such as approximately 4 to 5 million entries, corresponding to the measured values ​​of different electrical parameters of the printed circuit board. Assignment data ADB can be provided, for example, as a .xls file and can include thousands of entries, such as approximately 6,000 to 7,000 entries. Measurement data MDB can also be provided, for example, as a .pcf file.

[0149] These data can be preprocessed using DPP as follows. Preprocessing can be performed automatically. In an embodiment, a violation history (VH) can be created (for the printed circuit board). The violation history may include the time to pass a measurement cycle, the first / last pass, the time to fail a measurement cycle, the first / last fail, and a threshold number of violations, where the threshold determines the allowable range of values ​​for the (measurement) parameters of the printed circuit board.

[0150] In an embodiment, the LL measurement data MDB can be labeled based on the assigned data ADB. This labeling can include two or more categories, for example, a first pass labeled "Pass," a pseudo-error labeled "Pseudo-Error," and a pure failure labeled "Fail." In this way, supervised learning can be enabled in a simple manner.

[0151] To better handle violation information and expert information (regarding thresholds), logarithmic operations (OL) can be performed on outliers outside the threshold (i.e., outside the allowed range).

[0152] To reduce the impact of principal component analysis (PCA), correlation analysis and / or coefficient of variation can be performed to consider FR.

[0153] Then, if cross-validation is to be performed, the preprocessed data PPD or a portion thereof can be provided as the training dataset to the classification function CF, which can be located in the cloud PN.

[0154] It should be understood that measurement data (MDB), allocation data (ADB), etc., can be provided in the cloud or in the form of a data lake. The data lake can be located in the same location as the industrial equipment (PL) or in the cloud (PN), or even distributed between the industrial equipment (PL) and the cloud (PN). The data lake may include a manufacturing data lake. These databases can be updated as a result of implementing the methods described above, for example, by allocation (ASG) and / or by optimizing the statistical parameters (OPT).

[0155] Turning Figure 10 This illustrates an example of a method for providing a computer-based implementation of a trained recommendation function (TRF) for printed circuit board testing, the method comprising:

[0156] Step A1: Receive training input data and training output data, wherein the training input data represents a recommendation for at least one optimized statistical parameter OPT, wherein the statistical parameter determines a predetermined range of values ​​for at least one electrical parameter of each of a plurality of electronic components of the at least one printed circuit board, wherein the predetermined range of values ​​includes values ​​allowed during printed circuit board testing, and the training output data represents acceptance of the recommendation;

[0157] Step B1: Execute the recommendation function on the training input data to generate the predicted training output data;

[0158] Step C1: Compare the training output data with the predicted training output data to determine the error;

[0159] Step D1: Update the recommendation function based on the determined error, and, for example, if the error is still unsatisfactory, perform steps B1 to D1 iteratively to (further) reduce the error. Otherwise, terminate the method T'.

[0160] In some embodiments, the recommendation function may be based on data-driven optimization, specifically based on distributed robust optimization and / or online linear programming algorithms and / or non-convex regularized least squares and / or multi-block alternating direction multiplier method (ADMM).

[0161] Figure 11An example of a training system for providing the trained classification function TCF (or trained recommendation function TRF, as described below) is shown. The training system may include software and / or hardware components. The training system may reside on a public network PN, such as the cloud (e.g., ...). Figure 11 (As shown). It can also be located in the same location as the industrial equipment PL (not shown). The training system can be distributed between the industrial equipment PL and the public network PN. For example, it may be advantageous to use cloud resources to process large amounts of test data DT and / or to execute the classification or recommendation function.

[0162] The training system may include a first training interface TIF1 configured to receive training input data and training output data, for example, in the form of preprocessed training data PPD, wherein the training input data represents ICT test data TD of printed circuit boards PCB 1, PCB 2, and PCB 3, wherein the ICT test data includes measurement data MDB of multiple electronic components of the at least one printed circuit board, and the training output data represents an assignment ASG of at least one, particularly each, of the multiple electronic components of printed circuit boards PCB 1, PCB 2, and PCB 3, wherein each assignment is an assignment of the electronic component of the printed circuit board to one of at least two, more specifically three, different categories, and different assignments correspond to different electronic components. The training system may also include a training computing device TCA configured to execute a classification function CF on the training input data to generate predicted training output data; compare the training output data with the predicted training output data to determine an error; and update the classification function based on the determined error. Furthermore, the training system may include a second training interface TIF2 configured to provide the trained classification function.

[0163] Figure 11 The training system shown can also be used to train the trained recommendation function TRF. If the training input data indicates that the recommendation is accepted by at least one optimized statistical parameter OPT, the training output data indicates that the recommendation is accepted. The training computing device TCA is configured to execute the recommendation function.

[0164] The embodiments of the present invention described above are presented for illustrative purposes and not for limitation. In particular, the embodiments described with reference to the accompanying drawings are merely a few examples of the embodiments described in the introductory section.

[0165] Reference numerals in the claims are for clarity only and should not be considered as limiting elements of the claims.

[0166] The features, advantages, or alternative embodiments described herein can be assigned to other claims and vice versa. In other words, features described or claimed in the context of this method can be used to improve the claims for providing the system. In this case, the functional features of the method are embodied by the target unit of the providing system.

[0167] Furthermore, in the following text, the solution according to the invention is described by way of methods and systems for using the trained function and by way of methods and systems for providing the trained function. Features, advantages, or alternative embodiments described herein may be assigned to other claims, and vice versa. In other words, the claims for methods and systems for providing the trained function can be modified by features described or claimed in the context of methods and systems for using the trained function, and vice versa.

[0168] In particular, the trained (machine learning) function or algorithm of the method and system can be adjusted by the method and system providing the trained function. Furthermore, the input data may include advantageous features and embodiments of the training input data, and vice versa. Additionally, the output data may include advantageous features and embodiments of the output training data, and vice versa.

[0169] Typically, trained functions mimic the cognitive functions by which humans connect with other human brains. In particular, through training based on training data, these trained functions can adapt to new environments and detect and infer patterns.

[0170] Typically, the parameters of a trained function can be tuned through training. Specifically, for training a recommendation function, supervised training, semi-supervised training, unsupervised training, reinforcement learning, and / or active learning can be used. Additionally, representation learning (an alternative term is "feature learning") can be employed. In particular, the parameters of a trained function can be tuned through several training iterations.

[0171] In particular, the trained function may include neural networks, support vector machines, decision trees, and / or Bayesian networks, and / or the trained function may be based on k-means clustering, Q-learning, genetic algorithms, and / or association rules. Specifically, the neural network may be a deep neural network, a convolutional neural network, or a convolutional deep neural network. Furthermore, the neural network may be an adversarial network, a deep adversarial network, and / or a generative adversarial network.

Claims

1. A computer-implemented method for evaluating at least one printed circuit board (PCB 1, PCB 2, PCB 3), comprising: - receiving input data (ID), wherein the input data (ID) is based on test data (TD) of at least one printed circuit board (PCB 1, PCB 2, PCB 3), wherein the test data is test data of an online test and comprises measurement data (MDB) of a plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3); - applying a trained classification function (TCF) to the input data (ID), wherein output data is generated; - providing the output data, wherein the output data comprises an assignment (ASG) of at least one of the plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3) to one of at least two different classes, wherein one of the at least two different classes corresponds to a kind of a pseudo-faulty printed circuit board, and wherein the measurement data (MDB) of the plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3) comprises measurement data of at least one electrical parameter of each of the plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3).

2. The method of claim 1, wherein, The trained classification function (TCF) is based on a multivariate classification algorithm.

3. The method of claim 1, wherein, The at least one printed circuit board is assigned to a group consisting of three classes.

4. The method of claim 1, wherein, The input data and the output data comprise layout information of the at least one printed circuit board, and providing the output data comprises visualizing the measurement data (MDB) of the at least one electrical parameter of each of the plurality of electronic components of the at least one printed circuit board based on the layout information.

5. The method according to any one of claims 1-4, further comprising: - optimizing at least one statistical parameter, wherein the statistical parameter determines a predetermined allowed range of values of the at least one electrical parameter of each of the plurality of electronic components of the at least one printed circuit board, wherein, providing the output data comprises: - recommending at least one optimized statistical parameter (OPT) for use in a further test.

6. The method of claim 5, wherein, The recommendation is performed by means of a trained recommendation function (TRF).

7. The method of any one of claims 1 to 4, wherein, Providing a trained classification function (TCF) comprises: A0) receiving training input data and training output data, wherein the training input data represents test data (TD) of an online test, wherein the test data of the online test comprises measurement data of a plurality of electronic components of at least one printed circuit board (PCB 1, PCB 2, PCB 3), and the training output data represents an assignment (ASG) of at least one of the plurality of electronic components of the printed circuit board (PCB 1, PCB 2, PCB 3), wherein each assignment is an assignment of at least one of the plurality of electronic components into one of at least two different classes; B0) performing a classification function (CF) on the training input data to generate predicted training output data; C0) comparing the training output data with the predicted training output data to determine an error; D0) updating the classification function in dependence on the determined error, and E0) performing steps A0) to D0) in an iterative manner to reduce the error.

8. The method of claim 6, wherein, providing a trained recommendation function (TRF) for printed circuit board testing comprises: A1) receiving training input data and training output data, wherein the training input data represents a recommendation of at least one optimized statistical parameter (OPT), wherein the statistical parameter determines a predetermined range of values of at least one electrical parameter of each electronic component of a plurality of electronic components of at least one printed circuit board, wherein the predetermined range of values comprises values allowed during printed circuit board testing, and the training output data represents an acceptance of the recommendation; B1) performing a recommendation function on the training input data to generate predicted training output data; C1) comparing the training output data with the predicted training output data to determine an error; D1) updating the recommendation function in dependence on the determined error, and E1) performing steps A1) to D1) in an iterative manner to reduce the error.

9. The method of claim 2, wherein, The multivariate classification algorithm is a ordinary linear regression algorithm, a random forest algorithm, a gradient boosting algorithm, a LASSO algorithm or a logistic regression algorithm.

10. The method of claim 9, wherein, The LASSO algorithm is an adaptive LASSO algorithm.

11. The method of claim 10, wherein, The adaptive LASSO algorithm is a regularized linear regression algorithm.

12. The method of claim 9, wherein, The logistic regression algorithm is a binary logistic regression algorithm.

13. The method of claim 3, wherein, The at least one printed circuit board is assigned to a group consisting of a first class, wherein the first class corresponds to a kind of normally working printed circuit boards, a second class, wherein the second class corresponds to a kind of faulty printed circuit boards, and a third class, wherein the third class corresponds to a kind of pseudo-faulty printed circuit boards.

14. The method of claim 6, wherein, The trained recommendation function is based on data-driven optimization.

15. The method of claim 14, wherein, The trained recommendation function is based on a distributed robust optimization and / or an online linear programming algorithm and / or a non-convex regularized least squares method and / or a multi-block alternating direction method of multipliers.

16. A machine-readable data medium comprising output data, wherein, The output data is provided according to the method of any one of claims 1 to 15.

17. A computer program comprising instructions which, when the program is executed by a system, cause the system to carry out the method according to any one of claims 1 to 15.

18. A system for evaluating at least one printed circuit board (PCB 1, PCB 2, PCB 3), comprising: - a first interface (IF 1) configured for receiving input data (ID), wherein the input data (ID) is based on test data of at least one printed circuit board, wherein the test data is test data of an online test and comprises measurement data (MDB) of a plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3); - a computing device configured to apply a trained classification function (TCF) to the input data (ID) to generate output data; - a second interface configured for providing the output data, wherein the output data comprises an assignment (ASG) of at least one of the plurality of electronic components of the at least one printed circuit board to one of at least two different classes, wherein one of the at least two different classes corresponds to a kind of pseudo-faulty printed circuit boards, and wherein the measurement data (MDB) of the plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3) comprises measurement data of at least one electrical parameter of each of the plurality of electronic components of the at least one printed circuit board (PCB 1, PCB 2, PCB 3).

19. The system of claim 18, wherein, The computing device or the second interface is configured for: - optimizing at least one statistical parameter, wherein the statistical parameter determines a predetermined range of allowed values of at least one electrical parameter of each of the plurality of electronic components of the at least one printed circuit board, wherein The computing device or the second interface is configured for:

20. The system of claim 19, wherein, - recommending at least one optimized statistical parameter (OPT) for use in further testing while providing the output data. The computing device or the second interface is configured for: - recommending at least one optimized statistical parameter (OPT) for use in further testing while providing the output data by applying a trained recommendation function (TRF).

Images