Method and apparatus for product quality detection

Abstract

A method and device for product quality detection are proposed to find the number of unknown manufacturing parameters that affect product quality. A method (200) for product quality detection comprises: obtaining (S201) for each product in a set of products: an image, a value of each known manufacturing parameter that affects the quality of the set of products, and a quality assessment result; training (S202) a neural network, wherein a layer M of the neural network comprises at least one first neuron and at least one second neuron, and each first neuron represents a known manufacturing parameter that affects the quality of the set of products, and each second neuron represents an unknown manufacturing parameter that affects the quality of the set of products, and the image of the set of products is input to the neural network, the quality assessment result is an output of the neural network, and a value of each first neuron is set to the value of the known manufacturing parameter that the first neuron represents.

Description

Technical Field

[0001] This invention relates to product quality testing technology, and more specifically, to a method, apparatus, and computer-readable storage medium for product quality testing. Background Technology

[0002] Product manufacturing processes, such as industrial processes, typically employ some form of quality inspection to ensure product quality. Automated vision inspection systems are frequently used for this purpose, employing various computer algorithms to examine captured product images for any defects. Once a defect is detected, the defective product is separated from the high-quality product.

[0003] However, the aforementioned detection systems only help manufacturers identify defective products, but cannot help them understand how to improve product quality. Summary of the Invention

[0004] To improve product quality, the first step is to identify the manufacturing parameters that affect it. Typically, manufacturers may know these parameters from domain knowledge. However, domain knowledge is sometimes insufficient, as other possible manufacturing parameters may also influence product quality. The solution of this invention can determine whether there are unknown manufacturing parameters affecting product quality.

[0005] According to a first aspect of this disclosure, a method for performing product quality testing on a set of products is presented, comprising the following steps:

[0006] - Obtain for each product in the set of products: an image, the value of each known manufacturing parameter affecting the quality of the set of products, and the quality assessment results;

[0007] - Training a neural network, wherein layer M of the neural network includes at least one first neuron and at least one second neuron, and each first neuron represents a known manufacturing parameter affecting the quality of the set of products, and each second neuron represents an unknown manufacturing parameter affecting the quality of the set of products, and the image of the set of products is input into the neural network, the quality assessment result is the output of the neural network, and the value of each first neuron is set to the value of the known manufacturing parameter represented by the first neuron.

[0008] According to a second aspect of this disclosure, an apparatus for performing product quality testing on a set of products is provided, comprising:

[0009] - A data acquisition module, configured to acquire for each product in the set of products: an image, the value of each known manufacturing parameter affecting the quality of the set of products, and the quality assessment result;

[0010] - A training module configured to train a neural network, wherein layer M of the neural network includes at least one first neuron and at least one second neuron, and each first neuron represents a known manufacturing parameter affecting the quality of the set of products, and each second neuron represents an unknown manufacturing parameter affecting the quality of the set of products, and the image of the set of products is input into the neural network, the quality assessment result is the output of the neural network, and the value of each first neuron is set to the value of the known manufacturing parameter represented by the first neuron.

[0011] According to a third aspect of this disclosure, an apparatus for product quality testing is presented, comprising:

[0012] - At least one memory configured to store instructions;

[0013] - At least one processor coupled to the at least one memory and configured to perform the method according to the first aspect of this disclosure when executing executable instructions.

[0014] According to a fourth aspect of this disclosure, a computer-readable medium is provided that stores executable instructions, which, when executed by a processor, enable the processor to perform the method according to a first aspect of this disclosure.

[0015] In this disclosure, a product image reflecting quality is used as input to a neural network, and the quality assessment result is used as the output of the neural network to find the relationship between the product image and quality. Furthermore, at least one first neuron and at least one second neuron in layer M of the neural network represent manufacturing parameters affecting product quality. By training the neural network to a stable state, the manufacturing parameters affecting product quality can be obtained. The provided solution introduces unknown manufacturing parameters as neurons in the same layer as known manufacturing parameters. By training the neural network, the precise influence of unknown manufacturing parameters on product quality can be easily obtained compared to known parameters.

[0016] In embodiments of this disclosure, the impact of changes in the value of each second neuron on quality changes can be calculated based on the trained neural network. By comparing the calculated impacts, the number of unknown manufacturing parameters affecting the quality of a set of products can be determined. Based on the trained model, the impact of added unknown manufacturing parameters on quality can be obtained.

[0017] In embodiments of this disclosure, when training a neural network, the following steps may be repeated until a predefined condition is met:

[0018] - Add a second neuron to layer M of the neural network;

[0019] - The neural network is trained in such a way that the value of each neuron in layer M, except for the newly added second neuron, is set to the value of the manufacturing parameter represented by that neuron;

[0020] - Based on the trained neural network, calculate the value of the newly added second neuron as the value of the unknown manufacturing parameter represented by the newly added second neuron.

[0021] Considering that some manufacturing parameters may be related to each other, preferably, only one second neuron is added at a time and the neural network is trained with only one new neuron to obtain the pure effect of each unknown manufacturing parameter on product quality.

[0022] In embodiments of this disclosure, quality assessment results can be obtained through the following steps:

[0023] - Select high-quality product images from a set of product images;

[0024] - Train a model for object recognition using the selected images;

[0025] - Based on a trained model for object recognition, identify products from each image in a set of products;

[0026] - Use the identification confidence value as a quality assessment result of the product identified by the image.

[0027] The provided solution allows for accurate evaluation of product quality. It offers an easy-to-implement method for evaluating any type of product by comparing its image with images of high-quality products. Attached Figure Description

[0028] The above-mentioned attributes, other features and advantages of the present invention, as well as its implementation, will become more apparent and the present invention itself will be better understood by referring to the following description of embodiments of the present invention taken in conjunction with the accompanying drawings, in which:

[0029] Figure 1 An image of an anisotropic conductive film (ACF) containing particle tracks is shown, where each particle track represents a product of this disclosure.

[0030] Figure 2 A flowchart for product quality inspection according to an exemplary embodiment of the present disclosure is depicted.

[0031] Figure 3 The mass of the particle trajectory, affected by manufacturing parameters, is depicted.

[0032] Figure 4 A flowchart for obtaining quality assessment results according to embodiments of the present disclosure is depicted.

[0033] Figure 5A An image of an ACF containing particle trajectories is shown.

[0034] Figure 5B The identification confidence values ​​for each particle trajectory according to exemplary embodiments of this disclosure are shown.

[0035] Figure 6 A flowchart for training a neural network according to an exemplary embodiment of the present disclosure is depicted.

[0036] Figure 7 The process of training a neural network according to exemplary embodiments of the present disclosure is described.

[0037] Figure 8 Depicting Figure 7 The influence of each neuron in layer M of the neural network shown on the quality.

[0038] Figure 9 and Figure 10 A block diagram depicting an exemplary embodiment of the product quality testing equipment of this disclosure is provided.

[0039] Figure label:

[0040] 10, Particle trajectories on ACF

[0041] 20, ACF

[0042] 200, A method for product quality inspection according to exemplary embodiments of the present disclosure

[0043] S201~S204, S2011~S2014, S2021~S2023, step 30 of method 200, a set of product images

[0044] 40. Neural Networks

[0045] 401, layer M of a neural network with 40 layers.

[0046] 401a, presents neurons that represent known manufacturing parameters that affect the quality of a set of products.

[0047] 401b, presents neurons that represent unknown manufacturing parameters affecting the quality of a set of products.

[0048] 50. Quality assessment results of a group of products.

[0049] 300, an apparatus for product quality inspection according to an exemplary embodiment of the present disclosure.

[0050] 301, Data Acquisition Module

[0051] 302, Training Module

[0052] 303, Calculation Module

[0053] 304, Comparison Module

[0054] 305, at least one processor

[0055] 306, at least one memory

[0056] 307, I / O Interface Detailed Implementation

[0057] The above-described features and other features of the present invention are described in detail below. Various embodiments are described with reference to the drawings, wherein the same reference numerals are used throughout to refer to the same elements. In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of one or more embodiments. It should be noted that the embodiments shown are intended to explain rather than limit the invention. It will be apparent that such embodiments may be practiced without these specific details.

[0058] When describing elements of various embodiments of this disclosure, the articles “a” and “described” are intended to indicate the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to the listed elements.

[0059] Using the disclosed solution, unknown manufacturing parameters affecting product quality can help manufacturers understand how product quality relates to the manufacturing process during product preparation. By controlling and / or manipulating both known and unknown manufacturing parameters during manufacturing, product quality can be improved and quality fluctuations can be significantly reduced.

[0060] Product Quality -> Known Manufacturing Parameters + Unknown Manufacturing Parameters

[0061] The disclosed solution can be used when product images are available. In one embodiment of this disclosure, a possible number of unknown manufacturing parameters affecting product quality can be identified. Through possible further processing, such manufacturing parameters can be identified and processed to improve product quality.

[0062] We will use the following use case to illustrate this: finding the number of unknown manufacturing parameters affecting the particle trajectory on the conductive mass of an ACF. It should be noted that the particle trajectory on the ACF is merely an example of the product disclosed herein; considering manufacturing parameters, the solution can be applied to other types of products to improve quality. Furthermore, manufacturing parameters can encompass all types of parameters related to the manufacturing of the product, such as temperature, humidity, etc.

[0063] Now we will refer to Figures 1 to 10 The present invention is described in detail below.

[0064] Figure 1 ACF 20 is shown, a flexible film that can be used in display manufacturing processes, such as adhering chips to glass, bonding glass to displays, and mounting ICs onto boards. These processes rely on ACF as a medium to bond components, including IC driver modules, panels, displays, etc., together and allow them to be electrically interconnected. Particle trajectories 10 on ACF 20 are examples of products in this disclosure.

[0065] Figure 2 A flowchart for product quality inspection according to an exemplary embodiment of this disclosure is depicted. Method 200 may include the following steps:

[0066] S201: Retrieve each product in a set of products:

[0067] -image,

[0068] - The value of each known manufacturing parameter that affects the quality of a set of products, and

[0069] -Quality assessment results;

[0070] Taking the particle trajectory as an example of a set of products, the following information items can be obtained in step S201:

[0071]

[0072] Optionally, product quality can be assessed by comparing the product image with other images containing similar high-quality products.

[0073] refer to Figure 4 First, in sub-step S2011, images of high-quality products can be selected from a set of product images. The quality of a product can be determined by an engineer or domain expert. Then, in sub-step S2012, a model for object recognition can be trained using the selected images obtained in sub-step S2011. Next, based on each image of the set of products, the product can be recognized using the trained model for object recognition. Therefore, in sub-step S2014, we can use the recognition confidence value as the quality assessment result of the product recognized from the image. Here, we use the recognition confidence value as the quality assessment result based on the principle that the higher the product quality, the higher the similarity to high-quality products. Product quality is often difficult to measure, and domain experts subjectively classify it as "good" or "poor." Here, we use product images as input to the model for object recognition and assign numerical values ​​to the quality. With these values, we can continue to search for unknown manufacturing parameters that affect product quality.

[0074] Figure 3The particle trajectory mass distribution is shown under a set of manufacturing parameters including pressure and temperature. The values ​​on the y-axis indicate the normalized frequency based on temperature and pressure during manufacturing, and low values ​​on the x-axis indicate low particle trajectory mass.

[0075] Typically, in substep S2012, a model for object recognition, such as a neural network, will be trained to recognize high-quality particle trajectory images. After training, the model receives... Figure 5A The image shown (the trajectory particles are in one image, which can be considered as a separate image for each trajectory particle) then, in sub-step S2013, the model searches these images for objects that can be identified as particle trajectories. Once found, in sub-step S2014, the model outputs a confidence value indicating that the object is a particle trajectory. This confidence value can be considered as a quality assessment result of the particle trajectory, such as... Figure 5B As shown.

[0076] S202: Training the neural network.

[0077] refer to Figure 7 The layer M401 of neural network 40 includes:

[0078] - At least one first neuron 401a, representing known manufacturing parameters affecting the quality of a set of products, and

[0079] - At least one second neuron 401b, which represents an unknown manufacturing parameter that affects the quality of a set of products.

[0080] A set of product images 30 are input into a neural network 40, the quality assessment result 50 is the output of the neural network 40, and the value of each first neuron 401a is set to the value of the known manufacturing parameter represented by the first neuron 401a.

[0081] The following sub-steps can be repeated within a predetermined time or before predetermined conditions are met, and for each repetition, the neural network will be trained for 40 seconds until convergence:

[0082] -S2021: Add the second neuron 401b to layer M 401 of neural network 40.

[0083] -S2022: Train neural network 40, in which the value of each neuron in layer M, except for the newly added second neuron 401b, is set to the value of the manufacturing parameter represented by neuron 401b.

[0084] -S2023: Based on the trained neural network, calculate the value of the newly added second neuron 401b as the value of the unknown manufacturing parameter represented by the newly added second neuron 401b.

[0085] For example, given two known manufacturing parameters, there are two first neurons in layer M 401. Initially, layer M 401 contains only two first neurons 401a, then one second neuron 401b is added, therefore the total number of neurons in layer M 401 is:

[0086] Total number of neurons = NF + NA + NE = 2 + 0 + 1 = 3 neurons

[0087] Wherein, NF indicates the number of known manufacturing parameters, i.e., the number of first neurons; NA indicates the number of unknown manufacturing parameters other than the newly added second neurons; and NE indicates the number of newly added second neurons. To minimize the influence of one manufacturing parameter on another, preferably, only one new second neuron is added for each repetition.

[0088] Using a set of product images 30 as input, the neural network 40 is trained by setting the value of at least one first neuron to the corresponding known manufacturing parameter value to output the product quality assessment result. The loss function used for a single input to image 30 during training is:

[0089] Loss = (Q) 真实 -Q 预测 ) 2 +∑(F 真实 -F 预测 ) 2 ,

[0090] Here, the summation is the sum of the manufacturing parameter F and the quality assessment result Q. In the equation, "true" refers to the true known value, and "predicted" refers to the predicted value during the training of the neural network 40. Therefore, F 真实 F represents the actual value of the manufacturing parameters. 预测 Q represents the predicted value of the generated parameters during training. 真实 Q represents the true value of the quality assessment result. 预测 This indicates the predicted quality assessment results during training.

[0091] Next, we add another new second neuron, 401b. The second neuron in the NA will use the value calculated from the NE during the previous repetition as its true value.

[0092] Now, the total number of neurons in layer M is:

[0093] Total number of neurons = NF + NA + NE = 2 + 1 + 1 = 4 neurons

[0094] The loss function is:

[0095] Loss = ∑(F 真实 -F 预测 )2 +(NA, 真实 -NA, 预测 ) 2 +(Q 真实 -Q 预测 ) 2 ,

[0096] Where NA, truth It is the true value obtained from the NE as described above.

[0097] During each repetition, the value of the newly added second neuron 401b can be calculated as the value of the unknown manufacturing parameter represented by the newly added second neuron 401b.

[0098] For example, three new second neurons 401b are added, each representing an unknown manufacturing parameter that affects the quality of a set of products.

[0099]

[0100] After step S202, a neural network 40 having at least a second neuron 401b in layer M 401 can be trained, and the following calculations regarding the importance of manufacturing parameters can be performed based on the neural network.

[0101] In step S203, we can calculate the impact of changes in the value of each second neuron 401b on the quality change based on the trained neural network 40. And in step S204, we can compare the calculated impacts to determine the number of unknown manufacturing parameters affecting the quality of a set of products.

[0102] For each second neuron 401b in layer M 401, we change its value by a certain percentage and calculate the change in particle trajectory mass ΔQ. The results are plotted by drawing the change in ΔQ relative to each second neuron 401b (see reference). Figure 8 This can be achieved by setting a minimum threshold of 80 to determine the number of potentially influential but unknown manufacturing parameters. It should be noted that for... Figure 8 We repeated the results 9 times. Using a predefined minimum threshold of 80, the algorithm found 4 neurons above the threshold (4th from the left), so 2 influential but unknown manufacturing parameters may still affect the particle trajectory quality (because we started with 2 known manufacturing parameters).

[0103] Figure 9 and Figure 10 A block diagram illustrating an exemplary embodiment of the product quality inspection apparatus of this disclosure is provided. (Reference) Figure 9 Product quality testing equipment 300 may include:

[0104] - Data acquisition module 301, configured to acquire data for each product in a set of products:

[0105] -image,

[0106] - The value of each known manufacturing parameter that affects the quality of a set of products, and

[0107] -Quality assessment results;

[0108] - Training module 302, which is configured to train a neural network, wherein

[0109] - Layer M of the neural network includes at least one first neuron and at least one second neuron, where each first neuron represents a known manufacturing parameter affecting the quality of a set of products, and each second neuron represents an unknown manufacturing parameter affecting the quality of a set of products.

[0110] A set of product images are input into a neural network, the quality assessment result is the output of the neural network, and the value of each first neuron is set to the value of the known manufacturing parameter represented by the first neuron.

[0111] Optionally, the equipment 300 may further include:

[0112] - Computation module 303 is configured to calculate, for each second neuron, the effect of changes in the value of the second neuron on the quality change based on the trained neural network with at least one neuron added;

[0113] - Comparison module 304, which is configured to compare the calculated effects to determine the number of unknown manufacturing parameters affecting the quality of a set of products.

[0114] Optionally, the training module 302 is further configured to repeat the following steps during the training of the neural network until a predefined condition is met, said steps including:

[0115] - Add a second neuron to layer M of the neural network;

[0116] - Train a neural network in which the value of each neuron in layer M, except for the newly added second neuron, is set to the value of the manufacturing parameter represented by the neuron;

[0117] - Based on the trained neural network, calculate the value of the newly added second neuron as the value of the unknown manufacturing parameter represented by the newly added second neuron.

[0118] Optionally, when acquiring the quality assessment results for each product in a set of products, the data acquisition module 301 is further configured to:

[0119] - Select high-quality product images from a set of product images;

[0120] - Train a model for object recognition using the selected images;

[0121] - Based on a trained model for object recognition, identify products from each image in a set of products;

[0122] - Use the identification confidence value as a quality assessment result for products identified by image recognition.

[0123] Figure 10 Another block diagram depicting the product quality inspection equipment 300. (Reference) Figure 10 Product quality testing equipment 300 may include:

[0124] - At least one memory 305, which is configured to store instructions;

[0125] - At least one processor 306, coupled to at least one memory 305, and configured to perform the steps of method 200 when executing executable instructions.

[0126] The modules 301 to 304 described above may be software modules containing instructions stored in at least one memory 305, which execute method 200 when executed by at least one processor 306.

[0127] Optionally, the product quality inspection device 300 may also include an I / O interface 307 configured to receive inputs into and send outputs from the device 300. At least one processor 306, at least one memory 305, and the I / O interface may be connected via a bus or directly to each other.

[0128] This disclosure also provides a computer-readable medium storing executable instructions that, when executed by a computer, enable the computer to perform any of the methods presented in this disclosure.

[0129] The computer program is being executed by at least one processor and is performing any of the methods presented in this disclosure.

[0130] While the invention has been described in detail with reference to certain embodiments, it should be understood that the invention is not limited to those precise embodiments. In fact, given the scope of this disclosure describing exemplary modes for practicing the invention, many modifications and variations can be made by those skilled in the art without departing from the scope and spirit of the invention. Therefore, the scope of the invention is indicated by the appended claims, and not by the foregoing description. All changes, modifications, and variations within the equivalent meaning and scope of the claims are considered to be within the scope of the claims.

Claims

1. A method (200) for performing product quality inspection on a group of products, comprising: - Obtain each product from the set of products mentioned in (S201): -image, - The value of each known manufacturing parameter that affects the quality of the set of products, and -Quality assessment results; -Train the (S202) neural network, where The neural network layer M includes at least one first neuron and at least one second neuron, where each first neuron represents a known manufacturing parameter affecting the quality of the set of products, and each second neuron represents an unknown manufacturing parameter affecting the quality of the set of products. - The images of the set of products are input into the neural network, the quality assessment result is the output of the neural network, and the value of each first neuron is set to the value of the known manufacturing parameter represented by the first neuron; It further includes: - For each second neuron, calculate (S203) the impact of the change in the value of the second neuron on the quality change based on the trained neural network; - Compare (S204) the calculated impact with a predefined minimum threshold to determine the number of unknown manufacturing parameters affecting the quality of the set of products; The training (S202) neural network includes: Repeat the following steps until the predefined conditions are met: - Add a second neuron (S2021) to layer M of the neural network; - Train (S2022) the neural network, wherein the value of each neuron in layer M, except for the newly added second neuron, is set to the value of the manufacturing parameter represented by the neuron, wherein the value of the newly added second neuron in the previous repetition is set to the value of the unknown manufacturing parameter represented by the neuron calculated in the previous repetition; -Based on the trained neural network, calculate (S2023) the value of the newly added second neuron as the value of the unknown manufacturing parameter represented by the newly added second neuron; The quality assessment results for each product in the set of products described in (S201) include: - Select (S2011) high-quality product images from the set of product images; - Train (S2012) a model for object recognition using the selected images; -Based on the trained model for object recognition, identify each product according to each image of the set of products (S2013); - The identification confidence value (S2014) is used as the quality assessment result of the product identified from the image, the identification confidence value indicating the degree of similarity between the product in the image and the high-quality product; The loss function used for a single input image during training is: Loss = (Q 真实 -Q 预测 ) 2 +∑(F 真实 -F 预测 ) 2 , Wherein, the summation is the sum of the manufacturing parameter F and the quality assessment result Q, F 真实 F represents the actual value of the manufacturing parameters. 预测 Q represents the predicted value of the generated parameters during training. 真实 Q represents the true value of the quality assessment result. 预测 This indicates the predicted quality assessment results during training.

2. An apparatus (300) for performing product quality inspection on a group of products, comprising: - Data acquisition module (301), configured to acquire each of the set of products: -image, - The value of each known manufacturing parameter that affects the quality of the set of products, and -Quality assessment results; - Training module (302), which is configured to train a neural network, wherein The neural network layer M includes at least one first neuron and at least one second neuron, where each first neuron represents a known manufacturing parameter affecting the quality of the set of products, and each second neuron represents an unknown manufacturing parameter affecting the quality of the set of products. - The images of the set of products are input into the neural network, the quality assessment result is the output of the neural network, and the value of each first neuron is set to the value of the known manufacturing parameter represented by the first neuron; It further includes: - A calculation module (303) is configured to calculate, for each second neuron, the effect of the change in the value of the second neuron on the quality change based on the trained neural network with at least one neuron added; - Comparison module (304), configured to compare the calculated impact with a predefined minimum threshold to determine the number of unknown manufacturing parameters affecting the quality of the set of products; The training module (302) is further configured to repeat the following steps during the training of the neural network until a predefined condition is met, the steps including: - Add a second neuron to layer M of the neural network; - Train the neural network, wherein the value of each neuron in layer M, except for the newly added second neuron, is set to the value of the manufacturing parameter represented by the neuron, wherein the value of the newly added second neuron in the previous repetition is set to the value of the unknown manufacturing parameter represented by the neuron calculated in the previous repetition; -Based on the trained neural network, calculate the value of the newly added second neuron as the value of the unknown manufacturing parameter represented by the newly added second neuron; When obtaining the quality assessment results for each product in the set of products, the data acquisition module (301) is further configured to: - Select high-quality product images from the set of product images; - Train a model for object recognition using the selected images; - Based on the trained model for object recognition, identify the product according to each image of the set of products; - Use the identification confidence value as a quality assessment result of the product identified from the image, the identification confidence value indicating the degree of similarity between the product in the image and the high-quality product; The loss function used for a single input image during training is: Loss = (Q 真实 -Q 预测 ) 2 +∑(F 真实 -F 预测 ) 2 , Wherein, the summation is the sum of the manufacturing parameter F and the quality assessment result Q, F 真实 F represents the actual value of the manufacturing parameters. 预测 Q represents the predicted value of the generated parameters during training. 真实 Q represents the true value of the quality assessment result. 预测 This indicates the predicted quality assessment results during training.

3. An apparatus (300) for performing product quality inspection on a group of products, comprising: - At least one processor (306); - At least one memory (307) coupled to the at least one processor (306) is configured to perform the method according to claim 1.

4. A non-transitory computer-readable medium for product quality inspection, encoded with computer-executable instructions, wherein the computer-executable instructions, when executed, cause at least one processor to perform the method according to claim 1.

Images