Buffing process defect automatic analysis system based on deep learning

Abstract

The invention discloses a Buffing process defect automatic analysis system based on deep learning, relates to the technical field of defect automatic detection, and solves the problems that spatial association, causal relationships and propagation paths among various defects cannot be systematically analyzed, comprehensive analysis means based on historical data and real-time detection are lacked, and the detection efficiency is low. According to the method, a single defect and multiple types of defects are distinguished, a co-occurrence mode is analyzed through a defect association network, missing detection of a single algorithm on a complex defect combination is avoided, the defects are abstracted into graph nodes, and the degree centrality and the betweenness centrality are calculated, so that the defect source can be quickly positioned. The method comprises the following steps of: quantifying'activeness' and'bridge action 'of defects in an association network, combining historical defect records and real-time detection data, automatically positioning root defects through frequency proportion sorting, a reason-defect mapping table and threshold judgment, mining potential association reasons for non-corresponding defect signals through space or process intersection analysis, and avoiding omission of an indirect causal chain.

Description

Technical Field

[0001] The present invention relates to the technical field of automated defect detection, and specifically to an automated analysis system for Bumping process defects based on deep learning. Background Art

[0002] In the process of modern automatic production, machine vision systems have been widely used in monitoring, finished product inspection and quality control. The characteristics of machine vision systems are that they can improve the flexibility and automation of production.

[0003] According to the patent application with publication number CN117350947A, a method and system for detecting chip appearance defects in the semiconductor packaging test link are disclosed, including: importing the image to be detected and the template image; performing super-tolerance operation on the image to be detected through the template image to obtain a binary defect image; performing connected domain analysis on the binary defect image to obtain defect marks, and respectively cropping the image to be detected and the template image according to the defect marks; superimposing the defect detection cropped image and the defect template cropped image on a preset channel to form a two-channel image; inputting the two-channel image into a pre-built deep learning binary classification model for classification prediction to obtain a real defect image.

[0004] However, some existing automated defect analysis systems are unable to systematically analyze the spatial correlation, causal relationship, and propagation path among multiple defects. They lack comprehensive analysis methods based on historical data and real-time detection, making it difficult to quickly locate the source of defects. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides an automated analysis system for Bumping process defects based on deep learning, which solves the problems of being unable to systematically analyze the spatial correlation, causal relationship and propagation path between multiple defects, lacking comprehensive analysis methods based on historical data and real-time detection, and difficulty in quickly locating the source of defects.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a deep learning-based automated analysis system for bumping process defects, comprising:

[0007] An image defect analysis unit is used to preprocess the surface image transmitted by the semiconductor image acquisition unit to obtain a preprocessed image, extract image features and match them with defect features, generate a single defect signal or multi-type defect signals, and transmit the multi-type defect signals to the multi-type defect analysis unit;

[0008] For a single defect signal, the defects in the pre-processed image are segmented, and the defect type is determined by geometric changes to generate defect type information, which is then transmitted to the defect information output unit.

[0009] The multi-type defect analysis unit is used to analyze multi-type defect signals, record defects as nodes, calculate the degree centrality and betweenness centrality corresponding to the nodes, conduct comprehensive correlation analysis, generate defect correlation information, and transmit it to the comprehensive analysis processing unit and the defect information output unit;

[0010] The comprehensive analysis and processing unit is used to analyze defect-related information, classify defect records in historical data according to the same type, calculate the corresponding frequency ratio to generate ratio ranking information, determine the analysis object based on the frequency ratio, obtain the corresponding defect cause, make a judgment based on the defect-related information, and generate a defect analysis signal and a non-corresponding analysis signal;

[0011] Analyze the defect analysis signal, calculate the proportion of the number of associated defects, and generate defect cause information by comparing with the preset value. Analyze the non-corresponding analysis signal, use the defect cause of the associated defect as the standard to judge the relationship with the current defect, generate defect cause information, and transmit it to the defect information output unit at the same time.

[0012] As a further solution of the present invention, it also includes a semiconductor image acquisition unit and a defect information output unit;

[0013] A semiconductor image acquisition unit is used to acquire surface images of semiconductors through an industrial camera and transmit the images to an image defect analysis unit;

[0014] The defect information output unit is used to display the acquired defect association information and defect cause information to the corresponding operator.

[0015] As a further solution of the present invention, the specific manner in which the image defect analysis unit generates a single defect signal or multiple types of defect signals is as follows:

[0016] Acquire the surface image and perform denoising, enhancement and ROI extraction to obtain a preprocessed image. At the same time, use the Canny detection algorithm to extract the image features of the preprocessed image and match them with the defect features to generate defect information, determine the type of defect information, and generate a single defect signal or multiple types of defect signals.

[0017] As a further solution of the present invention, the specific manner in which the image defect analysis unit generates defect type information is as follows:

[0018] The defect information and defect mark are obtained at the same time, and then the preprocessed image is segmented according to the defect mark to obtain a defect segmentation image. Then, the obtained defect segmentation image is geometrically transformed, and the defect type is determined based on the different defect segmentation images obtained after the geometric transformation to generate defect type information.

[0019] As a further solution of the present invention, the specific manner in which the multi-type defect analysis unit generates defect association information is as follows:

[0020] Obtain multiple types of defects in the surface image and label them as i, where i = 1, 2, ..., a, where a represents the number of defects. Defect i is treated as a node, and the degree centrality and betweenness centrality of defect i are calculated.

[0021] The correlation of defect i is analyzed according to the calculated centrality, and a centrality calculation result table is obtained. The calculated centrality is matched with it, and corresponding defect correlation information is generated.

[0022] As a further solution of the present invention, the specific method of calculating the degree centrality and betweenness centrality corresponding to the node by the multi-type defect analysis unit is:

[0023] Calculate the degree centrality of node i and obtain the degree of node i in the undirected graph as d(i), which specifically represents the number of edges of node i. Then perform normalized degree centrality processing on the obtained d(i). Where a is the total number of nodes in the network, which also represents the number of defects;

[0024] Calculate the betweenness centrality of node i according to the formula The degree centrality of node i is calculated, where s and t represent nodes in the network, specifically s represents the source node, t represents the target node, and δ st Refers to the total number of shortest paths from node s to t, δ st (i) represents the number of nodes i in the shortest path from node s to t. If δ st The larger (i) is, the more likely it is that node i is on the shortest path from s to t.

[0025] As a further solution of the present invention, the specific method of the comprehensive analysis processing unit to generate the proportion ranking information is:

[0026] The defect record number obtained from the historical data is denoted as o, and o = 1, 2, ..., p, where p represents the number of defect records. The defect records are classified by defect type to generate defect information of the same type. The number of occurrences Ho of the same type of defect information is counted, and the proportion of the number of times the same type of defect information occurs is calculated. Then, the defect records are sorted from large to small according to the proportion of the number of times to generate proportion sorting information.

[0027] As a further solution of the present invention, the specific manner in which the comprehensive analysis processing unit generates the defect analysis signal and the non-corresponding analysis signal is as follows:

[0028] The defect type ranked first in terms of percentage is selected as the analysis object, and its associated defects and the defect types corresponding to the causes are obtained. If the associated defects exist in the defect types corresponding to the causes, a defect analysis signal is generated; otherwise, a non-corresponding analysis signal is generated.

[0029] As a further solution of the present invention, the specific method of generating defect cause information by the comprehensive analysis processing unit is:

[0030] Analyze the defect analysis signal, calculate the ratio of the number of related defects, and compare it with the preset value set by the operator. If the ratio is greater than the preset value, the corresponding cause is determined to be the source and the cause information is generated. Otherwise, no processing is done;

[0031] The non-corresponding analysis signal is analyzed, and the cause to be identified and the abnormal defect are obtained by correlating the defects. If the abnormal defect and the current defect have an intersection, the cause information is generated, otherwise the cause of the current and associated defects is transmitted to the output unit.

[0032] The present invention provides an automated analysis system for bumping process defects based on deep learning. Compared with the existing technology, it has the following advantages:

[0033] The present invention distinguishes between single defects and multiple types of defects, analyzes co-occurrence patterns through defect association networks, and avoids missed detection of complex defect combinations by a single algorithm. By abstracting defects into graph nodes, calculating degree centrality and betweenness centrality, and quantifying the "activity" and "bridge role" of defects in the association network, the present invention combines historical defect records with real-time detection data, automatically locates the root defect through frequency percentage sorting, cause-defect mapping table and threshold judgment, and for non-corresponding defect signals, explores potential related causes through spatial or process intersection analysis to avoid missing indirect causal chains. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a block diagram of the system principle of the present invention;

[0035] Figure 2 This is the centrality calculation result diagram of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] Example 1

[0038] See also Figure 1 and Figure 2This application provides a deep learning-based Bumping process defect automatic analysis system, including a semiconductor image acquisition unit, an image defect analysis unit, a multi-type defect analysis unit and a defect information output unit, combined with Figure 1 It can be known that the functional units are electrically connected in a unidirectional manner.

[0039] The semiconductor image acquisition unit is used to acquire the surface image of the semiconductor and transmit it to the image defect analysis unit. Here, the semiconductor surface is illuminated by an industrial camera to obtain its surface image.

[0040] Image defect analysis unit, which is used to preprocess the acquired surface image. The preprocessing operations include image denoising, image enhancement, and image ROI extraction to obtain a preprocessed image. Image denoising is performed by removing salt and pepper noise through median filtering (3×3 window), while image enhancement is performed by processing local uneven illumination through CLAHE (Contrast-Limited Adaptive Histogram Equalization). For image ROI extraction, template matching (such as NCC normalized cross correlation) is used to locate the chip area and eliminate background interference.

[0041] Then, the image features of the preprocessed image are extracted by using the Canny detection algorithm, and the extracted image features are matched with the defect features. The defect features here are obtained based on the various types of defects existing in the historical data to obtain the corresponding defect information. At the same time, the defect information is classified and analyzed. If there is only one type of defect information, a single defect signal is generated. Conversely, if there are multiple defect features in the defect information, multiple types of defect signals are generated and transmitted to the multi-type defect analysis unit at the same time.

[0042] For the single defect signal generated, the defect information and the defect mark are obtained, and then the preprocessed image is segmented according to the defect mark to obtain a defect segmentation image. Then, the obtained defect segmentation image is geometrically transformed, and the defect type is determined based on the different defect segmentation images obtained after the geometric transformation. The geometric transformation here includes rotation, scaling and translation, and the specific defect type is transmitted to the defect information output unit.

[0043] A multi-type defect analysis unit is used to analyze the acquired multi-type defect signals, obtain the multi-type defects in the surface image, and label them as i, where i = 1, 2, ..., a, where a represents the number of defects. At the same time, defect i is regarded as a node, and the degree centrality and betweenness centrality of defect i are calculated;

[0044] Calculate the degree centrality of node i and obtain the degree of node i in the undirected graph as d(i), which specifically represents the number of edges of node i. Then perform normalized degree centrality processing on the obtained d(i). Where a is the total number of nodes in the network, which also represents the number of defects;

[0045] Calculate the betweenness centrality of node i according to the formula The degree centrality of node i is calculated, where s and t represent nodes in the network, specifically s represents the source node, t represents the target node, and δ st Refers to the total number of shortest paths from node s to t, δ st (i) represents the number of nodes i in the shortest path from node s to t. If δ st The larger the value of (i), the more likely it is that node i is on the shortest path from s to t.

[0046] Then, the correlation of defect i is analyzed based on the calculated centrality to obtain the centrality calculation result table, such as Figure 2 As shown, the calculated centrality is matched with it and the corresponding defect association information is generated, which is then transmitted to the defect information output unit.

[0047] The following is analysis and inference

[0048] A(hole):

[0049] The degree centrality is the highest (directly related to B, C, and D), indicating that it is a core defect in the casting process, which may be caused by incomplete degassing of the molten metal.

[0050] The betweenness centrality is medium, indicating that it mainly affects the local area and does not become a bridge across clusters.

[0051] B (crack):

[0052] It has the highest betweenness centrality (the shortest path connecting A, D, and E). Although it has fewer direct connections than A, it is the only bridge between the "casting defect cluster (A, C)" and the "environmental aging cluster (D, E)".

[0053] Inference: The crack was initiated by stress concentration at the edge of the hole (A→B). The subsequent crack propagation exposed the metal, causing oxidation (B→D) and corrosion (B→E). The root cause included dual problems of the casting process (hole) and stress design (crack propagation path).

[0054] E(Corrosion):

[0055] The betweenness centrality is the second highest. Although there are few direct connections (only D and B), it is on the critical path of "oxidation → corrosion", verifying the chemical chain reaction of "oxidation → corrosion".

[0056] The defect information output unit is used to display the acquired defect-related information to the corresponding operator.

[0057] Example 2

[0058] As the second embodiment of the present invention, it is implemented on the basis of the first embodiment, and differs from the first embodiment in the following aspects:

[0059] The multi-type defect analysis unit transmits the generated defect association information to the comprehensive analysis processing unit, which is used to analyze the acquired defect association information and extract historical defect records D from the database, where D = d1, d2, ..., dp. Each record do (and o = 1, 2, ..., p) contains the defect type To, the occurrence time to, and the associated process parameters Po. The defect types are grouped by T to generate a set of defects of the same type G. T , calculate the number of occurrences of each group of defects H T , and frequency ratio R T =H T / p×100%;

[0060] According to the frequency ratio R T Generate a list ranked by percentage in descending order. Take Tmax, the top-ranked item, as the core analysis object. Obtain Tmax's associated defect set A = {a1, a2, ..., am} from the multi-type defect analysis unit. Each associated defect a is accompanied by an association strength Wi (such as co-occurrence frequency and confidence level). Extract the top three most frequent causes C = {c1, c2, c3} from historical records for the analysis object Tmax. Construct a cause-defect mapping table M, recording the known defect type T(cj) corresponding to each cause c.

[0061] Match the obtained associated defect set A with the defect type T(cj). If A∩T(cj)≠0, it is determined that c and the associated defect have a causal correspondence. For example: if Tmax = solder joint defect, associated defect A = {circuit short circuit, component dropout}, cause c1 = insufficient welding pressure, corresponding T(c1) = {solder joint defect, component dropout}, then c1 matches "component dropout" in A, generating a defect analysis signal;

[0062] If the cause c2 = high ambient humidity corresponds to T(c2) = {solder spot oxidation}, which has no intersection with A, then a non-corresponding analysis signal is generated;

[0063] Analyze the defect analysis signal to obtain the corresponding matching cause cj, count the number of times it causes the associated defect in the historical records, calculate the number of times ratio, and obtain the corresponding preset value. If the number of times ratio is greater than the preset value, then the matching cause cj is determined to be the root cause of the associated defect and the defect cause information is output. Otherwise, it is skipped;

[0064] The non-corresponding analysis signals are processed to obtain unmatched associated defects, and the associated abnormal process parameters and defect causes are extracted from the real-time data. If the associated defect has a spatial or process intersection with the current defect, the defect cause information is generated based on the defect cause. Otherwise, the defect cause of the associated defect is used as the standard to generate defect cause information.

[0065] The defect information output unit is used to display the acquired defect cause information to the corresponding operator.

[0066] Example 3

[0067] As the third embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments.

[0068] Some of the data in the above formulas are calculated based on their numerical values and are not substituted into parameter units for calculation. At the same time, the contents not described in detail in this specification belong to the existing technology known to those skilled in the art.

[0069] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. The deep learning-based automated analysis system for Bumping process defects is characterized by: include: An image defect analysis unit is used to preprocess the surface image transmitted by the semiconductor image acquisition unit to obtain a preprocessed image, extract image features and match them with defect features, generate a single defect signal or multi-type defect signals, and transmit the multi-type defect signals to the multi-type defect analysis unit; For a single defect signal, the defects in the pre-processed image are segmented, and the defect type is determined by geometric changes to generate defect type information, which is then transmitted to the defect information output unit. The multi-type defect analysis unit is used to analyze multi-type defect signals, record defects as nodes, calculate the degree centrality and betweenness centrality corresponding to the nodes, conduct comprehensive correlation analysis, generate defect correlation information, and transmit it to the comprehensive analysis processing unit and the defect information output unit; The comprehensive analysis and processing unit is used to analyze defect-related information, classify defect records in historical data according to the same type, calculate the corresponding frequency ratio to generate ratio ranking information, determine the analysis object based on the frequency ratio, obtain the corresponding defect cause, make a judgment based on the defect-related information, and generate a defect analysis signal and a non-corresponding analysis signal; Analyze the defect analysis signal, calculate the proportion of the number of associated defects, and generate defect cause information by comparing with the preset value. Analyze the non-corresponding analysis signal, use the defect cause of the associated defect as the standard to judge the relationship with the current defect, generate defect cause information, and transmit it to the defect information output unit at the same time.

2. The deep learning-based automated analysis system for Bumping process defects according to claim 1, characterized in that: It also includes a semiconductor image acquisition unit and a defect information output unit; A semiconductor image acquisition unit is used to acquire surface images of semiconductors through an industrial camera and transmit the images to an image defect analysis unit; The defect information output unit is used to display the acquired defect association information and defect cause information to the corresponding operator.

3. The deep learning-based automated analysis system for Bumping process defects according to claim 1, characterized in that: The specific method of the image defect analysis unit generating a single defect signal or multiple types of defect signals is as follows: Acquire the surface image and perform denoising, enhancement and ROI extraction to obtain a preprocessed image. At the same time, use the Canny detection algorithm to extract the image features of the preprocessed image and match them with the defect features to generate defect information, determine the type of defect information, and generate a single defect signal or multiple types of defect signals.

4. The deep learning-based automated analysis system for Bumping process defects according to claim 1, characterized in that: The specific method of the image defect analysis unit generating defect type information is as follows: The defect information and defect mark are obtained at the same time, and then the preprocessed image is segmented according to the defect mark to obtain a defect segmentation image. Then, the obtained defect segmentation image is geometrically transformed, and the defect type is determined based on the different defect segmentation images obtained after the geometric transformation to generate defect type information.

5. The deep learning-based automatic analysis system for Bumping process defects according to claim 1, characterized in that: The specific method for the multi-type defect analysis unit to generate defect association information is: Obtain multiple types of defects in the surface image and label them as i, where i = 1, 2, ..., a, where a represents the number of defects. Defect i is treated as a node, and the degree centrality and betweenness centrality of defect i are calculated. The correlation of defect i is analyzed according to the calculated centrality, and a centrality calculation result table is obtained. The calculated centrality is matched with it, and corresponding defect correlation information is generated.

6. The deep learning-based automated analysis system for Bumping process defects according to claim 5, characterized in that: The specific method for the multi-type defect analysis unit to calculate the degree centrality and betweenness centrality corresponding to the node is: Calculate the degree centrality of node i and obtain the degree of node i in the undirected graph as d(i), which specifically represents the number of edges of node i. Then perform normalized degree centrality processing on the obtained d(i). Where a is the total number of nodes in the network, which also represents the number of defects; Calculate the betweenness centrality of node i according to the formula The degree centrality of node i is calculated, where s and t represent nodes in the network, specifically s represents the source node, t represents the target node, and δ st Refers to the total number of shortest paths from node s to t, δ st (i) represents the number of nodes i in the shortest path from node s to t. If δ st The larger (i) is, the more likely it is that node i is on the shortest path from s to t.

7. The deep learning-based automated analysis system for Bumping process defects according to claim 1, characterized in that: The specific method for the comprehensive analysis processing unit to generate the proportion ranking information is: The defect record number obtained from the historical data is denoted as o, and o = 1, 2, ..., p, where p represents the number of defect records. The defect records are classified by defect type to generate defect information of the same type. The number of occurrences Ho of the same type of defect information is counted, and the proportion of the number of times the same type of defect information occurs is calculated. Then, the defect records are sorted from large to small according to the proportion of the number of times to generate proportion sorting information.

8. The deep learning-based automated analysis system for Bumping process defects according to claim 1, characterized in that: The specific method of the comprehensive analysis processing unit generating the defect analysis signal and the non-corresponding analysis signal is as follows: The defect type ranked first in terms of percentage is selected as the analysis object, and its associated defects and the defect types corresponding to the causes are obtained. If the associated defects exist in the defect types corresponding to the causes, a defect analysis signal is generated; otherwise, a non-corresponding analysis signal is generated.

9. The deep learning-based automated analysis system for Bumping process defects according to claim 1, characterized in that: The specific method for the comprehensive analysis processing unit to generate defect cause information is as follows: Analyze the defect analysis signal, calculate the ratio of the number of related defects, and compare it with the preset value set by the operator. If the ratio is greater than the preset value, the corresponding cause is determined to be the source and the cause information is generated. Otherwise, no processing is done; The non-corresponding analysis signal is analyzed, and the cause to be identified and the abnormal defect are obtained by correlating the defects. If the abnormal defect and the current defect have an intersection, the cause information is generated, otherwise the cause of the current and associated defects is transmitted to the output unit.

Images