A method for detecting defects in IC substrate production

Abstract

The present application relates to the technical field of defect tracing, and specifically discloses a defect tracing method for bad IC substrate production, which collects the surface state image of the IC substrate to be detected by a camera, extracts a set of IC substrate sample library images marked with defect types from the sample library, and uses machine vision-based image recognition and analysis technology to perform feature extraction and gradient significance of the surface state image of the IC substrate to be detected and the sample library image, so as to intelligently judge the defect type of the IC substrate to be detected based on the maximum similarity between the surface state feature of the IC substrate to be detected and the image feature of the IC substrate sample library. In this way, various defects on the IC substrate can be identified more accurately, human errors are reduced, and detection accuracy is improved. In addition, by analyzing the defect type, the specific link where the problem occurs in the production process can be traced back to improve the degree of automation of defect tracing for bad IC substrate production.

Description

Technical Field

[0001] The present application relates to the technical field of defect tracing, and more specifically, to a method for tracing defects of bad IC substrates produced. Background Art

[0002] With the rapid development of the electronics industry, the complexity and integration of integrated circuits (ICs) continue to increase, and the quality and reliability requirements of IC substrates are becoming higher and higher. As a key component of integrated circuits, any tiny defects in the production process of IC substrates may cause failure or performance degradation of the final product.

[0003] The production process of IC substrates is also long and involves many steps, usually including substrate production, drilling, electroplating, photolithography, etching, lamination, surface treatment and other key steps. In these processes, each link may produce surface or internal defects. Therefore, it is very important to effectively detect and trace defects in the production process of IC substrates.

[0004] However, the defect detection and traceability methods in traditional IC substrate production mainly rely on manual inspection and manual recording by production line workers, which has obvious limitations. Specifically, manual inspection is slow and prone to visual fatigue, which not only increases the risk of missed inspections, but may also lead to misjudgments and affect product quality. In addition, manual data recording is prone to errors and it is difficult to ensure the accuracy and completeness of the data, which is crucial for subsequent data analysis and quality improvement.

[0005] Therefore, a method for tracing defects of bad IC substrates in production is desired. Summary of the invention

[0006] The present application provides a method for tracing defects of bad IC substrate production, which can more accurately identify various defects on IC substrates, reduce human errors, and improve detection accuracy. In addition, by analyzing the defect type, it is possible to trace back to the specific link where the problem occurs in the production process, which is convenient for targeted improvement, thereby improving the automation of tracing defects of bad IC substrate production.

[0007] In a first aspect, a defect tracing method for bad IC substrate production is provided, comprising: obtaining a surface state image of an IC substrate to be detected collected by a camera; extracting a set of IC substrate sample library images labeled with defect types from a sample library; inputting each IC substrate sample library image labeled with defect types in the set of IC substrate sample library images labeled with defect types into an image feature extractor to obtain a set of IC substrate defect type feature maps; inputting each IC substrate defect type feature map in the set of IC substrate defect type feature maps into a feature distribution gradient mask salient device to obtain a set of IC substrate defect type significant feature maps; inputting the surface state image of the IC substrate to be detected into the image feature extractor and the feature distribution gradient mask salient device in sequence to obtain a surface state feature map of the IC substrate to be detected; calculating the similarity between the surface state feature map of the IC substrate to be detected and each IC substrate defect type significant feature map in the set of IC substrate defect type significant feature maps to obtain a set of similarities; and obtaining a defect type detection result of the IC substrate to be detected based on the set of similarities.

[0008] In one possible implementation, each IC substrate sample library image labeled with a defect type in the set of IC substrate sample library images labeled with defect types is input into an image feature extractor to obtain a set of IC substrate defect type feature maps, including: each IC substrate sample library image labeled with a defect type in the set of IC substrate sample library images labeled with defect types is input into an image feature extractor based on a deep neural network model to obtain a set of IC substrate defect type feature maps.

[0009] In a possible implementation, the image feature extractor based on the deep neural network model is an image feature extractor based on a hole convolutional neural network model.

[0010] In a possible implementation, each IC substrate defect type feature map in the set of IC substrate defect type feature maps is input into a feature distribution gradient mask salient device to obtain a set of IC substrate defect type salient feature maps, including: calculating the multi-directional gradient value distribution of each position in the IC substrate defect type feature map, and determining the gradient amplitude value of each position in the IC substrate defect type feature map based on the multi-directional gradient value distribution of each position to obtain an IC substrate defect type feature gradient amplitude distribution feature map; calculating the gradient amplitude local description operator of each position in the IC substrate defect type feature gradient amplitude distribution feature map to obtain an IC substrate defect type feature gradient amplitude local significant distribution feature map; inputting the IC substrate defect type feature gradient amplitude local significant distribution feature map into a gated masker based on a GELU function to obtain an IC substrate defect type feature gradient amplitude local significant gated mask feature map; calculating the position point multiplication between the IC substrate defect type feature gradient amplitude local significant gated mask feature map and the IC substrate defect type feature map to obtain the IC substrate defect type salient feature map.

[0011] In a possible implementation, the multi-directional gradient value distribution of each position in the IC substrate defect type feature map is calculated, and the gradient amplitude value of each position in the IC substrate defect type feature map is determined based on the multi-directional gradient value distribution of each position to obtain the IC substrate defect type feature gradient amplitude distribution feature map, including: calculating the gradient value of each characteristic value in the IC substrate defect type feature map along the width direction to obtain the IC substrate defect type width direction feature gradient value feature map; calculating the gradient value of each characteristic value in the IC substrate defect type feature map along the height direction to obtain the IC substrate defect type height direction feature gradient value feature map; calculating the gradient value of each characteristic value in the IC substrate defect type feature map along the channel direction to obtain the IC substrate defect type channel direction feature gradient value feature map; calculating the square root of the sum of the squares of the characteristic values ​​at corresponding positions in the IC substrate defect type width direction feature gradient value feature map, the IC substrate defect type height direction feature gradient value feature map and the IC substrate defect type channel direction feature gradient value feature map to obtain the IC substrate defect type feature gradient amplitude distribution feature map.

[0012] In one possible implementation, the local description operator of the gradient amplitude of each position in the IC substrate defect type characteristic gradient amplitude distribution feature map is calculated to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution feature map, including: determining the local neighborhood range of the gradient amplitude, and calculating a sequence of IC substrate defect type characteristic gradient amplitude neighborhood distribution feature maps based on the gradient amplitude local neighborhood range and the IC substrate defect type characteristic gradient amplitude distribution feature map; counting the number of feature maps in the sequence of the IC substrate defect type characteristic gradient amplitude neighborhood distribution feature maps to obtain the statistical value of the number of IC substrate defect type characteristic gradient amplitude neighborhoods; calculating the local neighborhood range of the gradient amplitude and the sequence of the IC substrate defect type characteristic gradient amplitude neighborhood distribution feature maps. After positionally differencing the degree amplitude neighborhood distribution feature maps, the difference is divided by position point with the IC substrate defect type characteristic gradient amplitude distribution feature map to obtain a sequence of IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature maps; after positionally adding each IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature map in the sequence of IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature maps, the difference is divided by position point with the IC substrate defect type characteristic gradient amplitude neighborhood number statistics to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution feature map.

[0013] In a possible implementation, the surface state image of the IC carrier to be detected is sequentially input into the image feature extractor and the feature distribution gradient mask salient device to obtain a surface state feature map of the IC carrier to be detected, including: inputting the surface state image of the IC carrier to be detected into the image feature extractor based on the void convolutional neural network model and the feature distribution gradient mask salient device to obtain the surface state feature map of the IC carrier to be detected.

[0014] In one possible implementation, the similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps is calculated to obtain a set of similarities, including: calculating the hash similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps to obtain a set of hash similarities as the set of similarities.

[0015] In one possible implementation, based on the set of similarities, a defect type detection result of the IC carrier to be inspected is obtained, including: taking the defect type of the IC carrier sample library image marked with the defect type corresponding to the maximum hash similarity in the set of hash similarities as the defect type detection result of the IC carrier to be inspected.

[0016] The present application provides a method for tracing defects of bad IC substrates in production, which uses a camera to collect the surface state image of the IC substrate to be detected, and extracts a collection of IC substrate sample library images marked with defect types from the sample library, and uses image recognition and analysis technology based on machine vision to perform feature extraction and gradient significance of the surface state image of the IC substrate to be detected and the sample library image, so as to intelligently judge the defect type of the IC substrate to be detected based on the maximum similarity between the surface state feature of the IC substrate to be detected and the image feature of the IC substrate sample library. In this way, various defects on the IC substrate can be identified more accurately, human errors are reduced, and detection accuracy is improved. In addition, by analyzing the defect type, it is possible to trace back to the specific link where the problem occurs in the production process, which is convenient for targeted improvement, thereby improving the degree of automation of defect tracing of bad IC substrates in production. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic flowchart of a defect tracing method for bad IC substrate production according to an embodiment of the present application.

[0018] Figure 2 Schematic diagram of data flow of the IC substrate production defect tracing method according to an embodiment of the present application.

[0019] Figure 3 In the defect tracing method for bad IC substrate production in an embodiment of the present application, each IC substrate defect type feature map in the set of IC substrate defect type feature maps is input into a feature distribution gradient mask salient device to obtain a schematic flowchart of a set of IC substrate defect type salient feature maps.

[0020] Figure 4 In the defect tracing method for bad IC substrate production in an embodiment of the present application, the multi-directional gradient value distribution of each position in the IC substrate defect type characteristic map is calculated, and the gradient amplitude value of each position in the IC substrate defect type characteristic map is determined based on the multi-directional gradient value distribution of each position to obtain a schematic flow chart of the IC substrate defect type characteristic gradient amplitude distribution characteristic map.

[0021] Figure 5 In the defect tracing method for bad IC substrate production in an embodiment of the present application, a schematic flowchart of calculating the gradient amplitude local description operator of each position in the IC substrate defect type characteristic gradient amplitude distribution feature map to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution feature map. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0023] It should be understood that once the defect type on the product (IC substrate) is determined by a certain detection method during the production process, this information can be used to track and identify the specific link in the production process where the problem occurs. This is because different defects are usually caused by different reasons, and these reasons are often associated with specific production links. By understanding the defect type, it is possible to infer the link where there may be problems, and then conduct targeted investigations and improvements. For example, if a certain defect is found to be mainly caused by the soldering link of the printed circuit board, then the focus can be on checking factors such as soldering equipment, solder temperature, and the operator's operational norms. In this way, by continuously optimizing the production process and strengthening quality monitoring, similar defects can be effectively prevented from occurring again, and product quality and production efficiency can be improved.

[0024] Based on this, the technical concept of the present application is to collect the surface state image of the IC carrier to be detected through a camera, and extract a set of IC carrier sample library images marked with defect types from the sample library, and use machine vision-based image recognition and analysis technology to perform feature extraction and gradient significance of the surface state image of the IC carrier to be detected and the sample library image, so as to intelligently judge the defect type of the IC carrier to be detected based on the maximum similarity between the surface state feature of the IC carrier to be detected and the image feature of the IC carrier sample library. In this way, various defects on the IC carrier can be identified more accurately, human errors are reduced, and detection accuracy is improved. In addition, by analyzing the defect type, it is possible to trace back to the specific link where the problem occurs in the production process, which is convenient for targeted improvements, thereby improving the degree of automation in tracing defects in bad IC carrier production.

[0025] Figure 1 This is a schematic flow chart of a method for tracing defects in IC substrate production according to an embodiment of the present application. Figure 1As shown, the IC substrate production defect tracing method comprises: S1, obtaining a surface state image of the IC substrate to be detected collected by a camera; S2, extracting a set of IC substrate sample library images marked with defect types from a sample library; S3, inputting each IC substrate sample library image marked with defect types in the set of IC substrate sample library images marked with defect types into an image feature extractor to obtain a set of IC substrate defect type feature maps; S4, inputting each IC substrate defect type feature map in the set of IC substrate defect type feature maps into a feature distribution Gradient mask salient device to obtain a set of significant feature maps of IC carrier defect types; S5, inputting the surface state image of the IC carrier to be detected into the image feature extractor and the feature distribution gradient mask salient device in sequence to obtain the surface state feature map of the IC carrier to be detected; S6, calculating the similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps to obtain a set of similarities; S7, based on the set of similarities, obtaining the defect type detection result of the IC carrier to be detected.

[0026] In the above IC substrate production defect tracing method, S1, obtains the surface state image of the IC substrate to be inspected captured by the camera. It should be understood that the captured surface state image of the IC substrate to be inspected can be used for subsequent image processing and analysis to identify different types of defects.

[0027] Optionally, in one embodiment of the present application, a high-resolution industrial camera is installed on the production line to ensure that it can cover the entire inspection area of ​​the IC carrier. Appropriate lighting conditions are adjusted so that the camera can clearly capture the image of the surface of the IC carrier. An evenly distributed light source is usually used to reduce shadows and reflections. The camera is started for image acquisition. The camera captures the image of the surface of the IC carrier in real time and converts it into a digital signal. The collected original image may need to be pre-processed, such as denoising, contrast enhancement, etc., to improve the image quality for subsequent analysis. The collected image data is stored in a database for subsequent analysis and defect tracing.

[0028] In the above-mentioned defect tracing method for bad IC carrier production, the S2 extracts a set of IC carrier sample library images marked with defect types from the sample library. It should be understood that by extracting images from the sample library, features can be extracted and compared with the surface state image of the IC carrier to be detected to identify similar defect patterns. Using sample images of known defects can help improve the accuracy of defect detection because they provide accurate defect feature information. The sample library image collection allows the system to learn the features of different defect types, thereby more accurately identifying the defect type during the detection process.

[0029] In the above-mentioned defect tracing method for IC substrate production bad boards, the S3, respectively inputs each IC substrate sample library image with a defect type in the set of IC substrate sample library images with defect types into an image feature extractor to obtain a set of IC substrate defect type feature maps. It should be understood that, considering that each IC substrate sample library image with a defect type in the set of IC substrate sample library images with defect types contains defect feature information of different scales about the sample image, and the hole convolution can increase the receptive field, so that the model can capture a wider range of features in the image, which is helpful to identify defects of different scales. Based on this, in the technical solution of the present application, each IC substrate sample library image with a defect type in the set of IC substrate sample library images with defect types is respectively input into an image feature extractor based on a hole convolution neural network model to refine and mine defect features of different scales, and obtain a set of IC substrate defect type feature maps. The image feature extractor based on the hole convolution neural network model here is essentially a hole convolution neural network model.

[0030] Optionally, in one embodiment of the present application, each IC substrate sample library image labeled with a defect type in the set of IC substrate sample library images labeled with defect types is respectively input into an image feature extractor to obtain a set of IC substrate defect type feature maps, including: each IC substrate sample library image labeled with a defect type in the set of IC substrate sample library images labeled with defect types is respectively input into an image feature extractor based on a deep neural network model to obtain the set of IC substrate defect type feature maps.

[0031] Optionally, in one embodiment of the present application, the image feature extractor based on the deep neural network model is an image feature extractor based on a hole convolutional neural network model.

[0032] In the above-mentioned defect tracing method for bad IC substrate production, the S4, inputs each IC substrate defect type feature map in the set of IC substrate defect type feature maps into the feature distribution gradient mask salient device to obtain a set of IC substrate defect type salient feature maps. It should be understood that, considering that the feature importance of each IC substrate defect type feature map in the set of IC substrate defect type feature maps is different at different positions, therefore, in order to more selectively highlight and salient key feature information, in the technical solution of the present application, each IC substrate defect type feature map in the set of IC substrate defect type feature maps is input into the feature distribution gradient mask salient device to obtain a set of IC substrate defect type salient feature maps. It is worth mentioning that the feature distribution gradient mask salient device calculates the gradient amplitude of each IC substrate defect type feature map, and makes the key position more significant based on the gradient amplitude and mask mechanism in the surrounding neighborhood, thereby improving the salient expression ability of the entire feature. In detail, first, the gradient amplitude value of the IC substrate defect type feature map at each position is calculated to obtain the IC substrate defect type feature gradient amplitude distribution feature map representing the intensity change of the local area of ​​the image. Next, the local description operator of each position in the IC substrate defect type feature gradient amplitude distribution feature map is calculated to strengthen the key significant features in the gradient amplitude, especially those subtle or difficult to directly observe features, so that they are more obvious in defect detection, and the IC substrate defect type feature gradient amplitude local significant distribution feature map is obtained. Then, the nonlinear characteristics of the GELU (Gaussian Error Linear Unit) function are used to input the IC substrate defect type feature gradient amplitude local significant distribution feature map into the gated masker for adaptive weighting of the gradient amplitude significant area, and generate the IC substrate defect type feature gradient amplitude local significant gated mask feature map with significant feature distinction. Finally, the IC substrate defect type feature gradient amplitude local significant gated mask feature map is multiplied by position by position with the IC substrate defect type feature map, retaining the features assigned with higher weights, and suppressing or removing other features. That is, the IC substrate defect type significant feature map improves the model's recognition ability of key significant features by combining the IC substrate defect type and gradient significance information, thereby providing a more refined and powerful feature representation for the final defect identification.

[0033] Optionally, in one embodiment of the present application, Figure 3 This is a schematic flow chart of the method for tracing the defect of IC substrate production bad boards in the embodiment of the present application, in which each IC substrate defect type feature map in the set of IC substrate defect type feature maps is input into the feature distribution gradient mask salient device to obtain a set of IC substrate defect type salient feature maps. Figure 3As shown, each IC substrate defect type feature map in the set of IC substrate defect type feature maps is input into a feature distribution gradient mask salient device to obtain a set of IC substrate defect type salient feature maps, including: S41, calculating the multi-directional gradient value distribution of each position in the IC substrate defect type feature map, and determining the gradient amplitude value of each position in the IC substrate defect type feature map based on the multi-directional gradient value distribution of each position to obtain an IC substrate defect type feature gradient amplitude distribution feature map; S42, calculating the IC substrate defect type feature gradient amplitude distribution feature map. The gradient amplitude of each position in the feature map is locally described by an operator to obtain a local significant distribution feature map of the gradient amplitude of the IC substrate defect type feature; S43, the IC substrate defect type feature gradient amplitude local significant distribution feature map is input into a gated masker based on a GELU function to obtain a local significant gated mask feature map of the IC substrate defect type feature gradient amplitude; S44, the position point multiplication between the IC substrate defect type feature gradient amplitude local significant gated mask feature map and the IC substrate defect type feature map is calculated to obtain the IC substrate defect type significant feature map.

[0034] Optionally, in one embodiment of the present application, Figure 4 In the IC substrate production defect tracing method of the embodiment of the present application, the multi-directional gradient value distribution of each position in the IC substrate defect type characteristic map is calculated, and the gradient amplitude value of each position in the IC substrate defect type characteristic map is determined based on the multi-directional gradient value distribution of each position to obtain the IC substrate defect type characteristic gradient amplitude distribution characteristic map. Figure 4 As shown, the multi-directional gradient value distribution of each position in the IC substrate defect type characteristic map is calculated, and the gradient amplitude value of each position in the IC substrate defect type characteristic map is determined based on the multi-directional gradient value distribution of each position to obtain the IC substrate defect type characteristic gradient amplitude distribution characteristic map, including: S411, calculating the gradient value of each characteristic value along the width direction of the IC substrate defect type characteristic map to obtain the IC substrate defect type width direction characteristic gradient value characteristic map; S412, calculating the gradient value of each characteristic value along the height direction of the IC substrate defect type characteristic map to obtain the IC substrate defect type height direction characteristic gradient value characteristic map; S413, calculating the gradient value of each characteristic value along the channel direction of the IC substrate defect type characteristic map to obtain the IC substrate defect type channel direction characteristic gradient value characteristic map; S414, calculating the square root of the sum of the squares of the characteristic values ​​of the corresponding positions in the IC substrate defect type width direction characteristic gradient value characteristic map, the IC substrate defect type height direction characteristic gradient value characteristic map and the IC substrate defect type channel direction characteristic gradient value characteristic map to obtain the IC substrate defect type characteristic gradient amplitude distribution characteristic map.

[0035] Optionally, in one embodiment of the present application, Figure 5 This is a schematic flow chart of the defect tracing method for IC substrate production defective boards in the embodiment of the present application, which calculates the local description operator of the gradient amplitude of each position in the characteristic gradient amplitude distribution characteristic map of the defect type of the IC substrate to obtain the characteristic gradient amplitude of the local significant distribution characteristic map of the defect type of the IC substrate. Figure 5 As shown, the local description operator of the gradient amplitude of each position in the IC substrate defect type characteristic gradient amplitude distribution feature map is calculated to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution feature map, including: S421, determining the local neighborhood range of the gradient amplitude, and calculating the sequence of the IC substrate defect type characteristic gradient amplitude neighborhood distribution feature map based on the gradient amplitude local neighborhood range and the IC substrate defect type characteristic gradient amplitude distribution feature map; S422, counting the number of feature maps in the sequence of the IC substrate defect type characteristic gradient amplitude neighborhood distribution feature map to obtain the IC substrate defect type characteristic gradient amplitude neighborhood number statistics; S423, calculating the IC substrate defect type characteristic gradient amplitude distribution feature map S424, adding each IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature map in the sequence of the IC substrate defect type characteristic gradient amplitude neighborhood distribution feature map at position point, and dividing the difference by the position point with the IC substrate defect type characteristic gradient amplitude distribution feature map to obtain a sequence of IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature maps; S424, adding each IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature map in the sequence of the IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature map at position point, and dividing the difference by the position point with the IC substrate defect type characteristic gradient amplitude neighborhood number statistics to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution feature map.

[0036] Specifically, in one embodiment of the present application, the IC substrate defect type feature map is input into a feature distribution gradient mask salient device, and is processed using the following gradient mask salient formula to obtain the IC substrate defect type salient feature map; wherein the gradient mask salient formula is:

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] in, , , , , and They are respectively the first , , , , and The eigenvalues ​​at the positions, is the first defect type in the IC substrate along the width direction in the defect type feature map The gradient amplitude value at each position, is the first defect type in the IC substrate along the height direction in the defect type feature map The gradient amplitude value at each position, is the first The gradient amplitude value at each position, The defect type characteristic gradient amplitude distribution characteristic diagram of the IC substrate is The eigenvalues ​​at the positions, The defect type characteristic gradient amplitude distribution characteristic diagram of the IC substrate is The eigenvalues ​​at the positions, The range is Neighborhood of express The number of elements in , The first one represents the local significant distribution feature map of the characteristic gradient amplitude of the defect type of the IC substrate The eigenvalues ​​at the positions, represents the Gaussian error linear gating function, The local significant gated mask feature map of the characteristic gradient amplitude of the IC substrate defect type is shown in the first The eigenvalues ​​at the positions, is a local significant gated mask feature map of the characteristic gradient amplitude of the IC substrate defect type, is the IC substrate defect type characteristic diagram, It means point multiplication by position. This is a significant feature diagram of the defect type of the IC substrate.

[0045] In the above-mentioned IC carrier production defect tracing method, the S5, the surface state image of the IC carrier to be detected is sequentially input into the image feature extractor and the feature distribution gradient mask salient device to obtain the surface state feature map of the IC carrier to be detected. It should be understood that considering that the surface state image of the IC carrier to be detected also has feature information of different scales and a wide range of context features, therefore, in the technical solution of the present application, the surface state image of the IC carrier to be detected is input into the image feature extractor based on the hole convolution neural network model to capture and extract the features of the surface state to be detected. Then, in order to further enhance and highlight the key feature information in the surface state features of the IC carrier to be detected, and provide more accurate data support for the subsequent defect type comparison and judgment, in the technical solution of the present application, the features after hole convolution coding are input into the feature distribution gradient mask salient device to obtain the surface state feature map of the IC carrier to be detected. In this way, through the feature distribution gradient mask salient device, the key edge and texture information features in the image to be detected, especially those related to defects, can be enhanced, so that these features are more obvious in the subsequent analysis and recognition process.

[0046] Optionally, in one embodiment of the present application, the surface state image of the IC carrier to be detected is sequentially input into the image feature extractor and the feature distribution gradient mask salient device to obtain a surface state feature map of the IC carrier to be detected, including: inputting the surface state image of the IC carrier to be detected into the image feature extractor based on the void convolutional neural network model and the feature distribution gradient mask salient device to obtain the surface state feature map of the IC carrier to be detected.

[0047] In the above-mentioned defect tracing method for IC substrate production, the S6 calculates the similarity between the surface state feature map of the IC substrate to be detected and the significant feature maps of each IC substrate defect type in the set of significant feature maps of the IC substrate defect type to obtain a set of similarities. It should be understood that in order to identify the known defect type feature map that best matches the surface state feature of the IC substrate to be detected, thereby assisting in the rapid and accurate diagnosis of defects, in the technical solution of the present application, the hash similarity between the surface state feature map of the IC substrate to be detected and the significant feature maps of each IC substrate defect type in the set of significant feature maps of the IC substrate defect type is calculated to obtain a set of hash similarities. That is, by calculating the hash similarity, the defect type that is most similar to the surface state of the IC substrate to be detected can be effectively identified from a large number of samples, thereby achieving rapid and accurate defect detection and classification.

[0048] Optionally, in one embodiment of the present application, the similarity between the surface state characteristic map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps is calculated to obtain a set of similarities, including: calculating the hash similarity between the surface state characteristic map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps to obtain a set of hash similarities as the set of similarities.

[0049] Preferably, considering that the surface state feature map of the IC carrier to be detected represents the image semantic features of the surface state image of the IC carrier to be detected, and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps represents the image semantic features of each IC carrier sample library image marked with the defect type based on feature space distribution gradient enhancement, therefore, when the hash similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps is calculated to perform aggregation mapping toward a common hash similarity representation space, there will also be aggregation alignment conflicts caused by differences in the image semantic feature representation dimensions, thereby causing the loss of aggregation key information and affecting the calculation accuracy of each hash similarity.

[0050] Based on this, when calculating the hash similarity between the surface state feature map of the IC substrate to be detected and each significant feature map of the IC substrate defect type in the set of significant feature maps of the IC substrate defect type, the present application jointly optimizes the surface state feature map of the IC substrate to be detected and the significant feature maps of each IC substrate defect type, and the optimization process includes:

[0051] Cascading the surface state feature map of the IC substrate to be detected and the significant feature maps of the defect types of each IC substrate according to the channel into a joint feature map of the IC substrate to be detected-IC substrate defect type;

[0052] Clustering all feature values ​​of the joint feature map of the IC substrate to be detected and the IC substrate defect type based on the distance between feature values, such as the absolute value distance of the difference, and determining the ratio of the number of features of the clustered features to the number of features of the joint feature map of the IC substrate to be detected and the IC substrate defect type;

[0053] Arranging the cluster features into a joint cluster vector of IC substrate-IC substrate defect types to be detected, and dividing the binary norm of the joint cluster vector of IC substrate-IC substrate defect types to be detected by the binary norm of the joint feature vector of IC substrate-IC substrate defect types to be detected obtained after expanding the joint feature map of IC substrate-IC substrate defect types to be detected to obtain a first joint clustering related weight value of IC substrate-IC substrate defect types to be detected;

[0054] Divide a first power value of a norm of the IC substrate to be detected-IC substrate defect type joint clustering vector with the ratio value as an exponent by a second power value of a norm of the IC substrate to be detected-IC substrate defect type joint feature vector with the ratio value as an exponent to obtain a second IC substrate to be detected-IC substrate defect type joint clustering related weight value;

[0055] For each eigenvalue of the IC substrate-IC substrate defect type joint clustering vector to be detected, multiply it by the inverse of the difference between the first IC substrate-IC substrate defect type joint clustering related weight value to be detected and the second IC substrate-IC substrate defect type joint clustering related weight value to be detected to obtain an optimized eigenvalue of the IC substrate-IC substrate defect type joint clustering vector to be detected;

[0056] For each feature value outside the cluster in the IC substrate to be detected-IC substrate defect type joint feature map, multiply it by the reciprocal of the sum of the weight value associated with the first IC substrate to be detected-IC substrate defect type joint cluster and the weight value associated with the second IC substrate to be detected-IC substrate defect type joint cluster to obtain an optimized out-of-class feature value of the IC substrate to be detected-IC substrate defect type joint feature map;

[0057] Combining the optimized eigenvalues ​​of the IC substrate to be detected-IC substrate defect type joint clustering vector and the optimized out-of-class eigenvalues ​​of the IC substrate to be detected-IC substrate defect type joint feature map into an optimized IC substrate to be detected-IC substrate defect type joint feature map;

[0058] The optimized IC substrate to be detected-IC substrate defect type joint feature map is split into an optimized IC substrate to be detected surface state feature map and each optimized IC substrate defect type significant feature map.

[0059] The optimized representation of the joint feature map of the defect type of the IC substrate to be detected and the IC substrate is:

[0060]

[0061]

[0062] in, is each eigenvalue of the joint clustering vector of the IC substrate to be detected and the IC substrate defect type, is the joint feature vector of the IC substrate to be detected and the defect type of the IC substrate, is the joint clustering vector of the IC substrate to be detected and the defect type of the IC substrate, is the number of features of the joint feature vector of the IC substrate to be detected and the IC substrate defect type, is the number of features of the joint clustering vector of the IC substrate to be detected and the IC substrate defect type, is the ratio of the number of features of the joint clustering vector of the IC substrate to be detected and the IC substrate defect type to the number of features of the joint feature vector of the IC substrate to be detected and the IC substrate defect type, represents the clustering feature set corresponding to the joint clustering vector of the IC substrate to be detected-IC substrate defect type, and Represents the binary norm and the uninorm of the vector respectively. Power, is each feature value of the optimized IC substrate to be detected-IC substrate defect type joint feature map.

[0063] That is, in order to avoid the loss of key suffix semantic information relative to the original feature set as a whole due to aggregation conflict between the surface state feature map of the IC substrate to be detected and the significant feature map of each IC substrate defect type based on the aggregation features, the clustering ratio of the number of features of the joint clustering vector of the IC substrate to be detected-IC substrate defect type relative to the number of features of the joint feature map of the IC substrate to be detected-IC substrate defect type is used as a decision function to perform an adversarial decision on the set absolute representation of the joint clustering vector of the IC substrate to be detected-IC substrate defect type and the one-norm joint feature vector of the IC substrate to be detected-IC substrate defect type, and the clustering ratio is used with the IC substrate to be detected respectively. The clustering intrinsic conflict representation of the board-IC substrate defect type joint clustering vector and the two-norm joint feature vector of the IC substrate-IC substrate defect type to be detected is positively and negatively interacted to construct a solid alignment guardrail of the optimized surface state feature map of the IC substrate to be detected and each optimized IC substrate defect type significant feature map based on the aggregated features and the original feature set as a whole, so as to achieve the optimized surface state feature map of the IC substrate to be detected and each optimized IC substrate defect type significant feature map based on the aggregation risk transferability. The intentional mitigation of the loss of harmful information is improved, and the calculation accuracy of the hash similarity between the surface state feature map of the IC substrate to be detected and the significant feature map of each IC substrate defect type is improved. In this way, various defects on the IC substrate can be identified more accurately, human errors are reduced, and detection accuracy is improved. In addition, by analyzing the defect type, the specific link where the problem occurs in the production process can be traced back, which is convenient for targeted improvements, thereby improving the automation of defect tracing of bad boards in IC substrate production.

[0064] In the above-mentioned defect tracing method for bad IC carrier production, the S7 obtains the defect type detection result of the IC carrier to be detected based on the set of similarities. It should be understood that the defect type of the IC carrier sample library image marked with the defect type corresponding to the maximum hash similarity in the set of hash similarities is used as the defect type detection result of the IC carrier to be detected. That is, the maximum hash similarity in the set of hash similarities calculated by hashing the surface state feature map of the IC carrier to be detected and the set of significant feature maps of the IC carrier defect type is used for judgment, so as to intelligently obtain the defect type of the IC carrier to be detected. In this way, various defects on the IC carrier can be identified more accurately, human errors are reduced, and detection accuracy is improved. In addition, by analyzing the defect type, it is possible to trace back to the specific link where the problem occurs in the production process, which is convenient for targeted improvements, thereby improving the degree of automation in the defect tracing of bad IC carrier production.

[0065] Optionally, in one embodiment of the present application, based on the set of similarities, a defect type detection result of the IC carrier to be inspected is obtained, including: taking the defect type of the IC carrier sample library image marked with the defect type corresponding to the maximum hash similarity in the set of hash similarities as the defect type detection result of the IC carrier to be inspected.

[0066] In summary, the defect tracing method for bad IC carrier board production according to the embodiment of the present application is explained, which uses a camera to collect the surface state image of the IC carrier board to be detected, and extracts a set of IC carrier board sample library images marked with defect types from the sample library, and uses machine vision-based image recognition and analysis technology to perform feature extraction and gradient significance of the surface state image of the IC carrier board to be detected and the sample library image, so as to intelligently judge the defect type of the IC carrier board to be detected based on the maximum similarity between the surface state feature of the IC carrier board to be detected and the image feature of the IC carrier board sample library. In this way, various defects on the IC carrier board can be identified more accurately, human errors are reduced, and detection accuracy is improved. In addition, by analyzing the defect type, it is possible to trace back to the specific link where the problem occurs in the production process, which is convenient for targeted improvements, thereby improving the degree of automation in the defect tracing of bad IC carrier board production.

[0067] The IC substrate production defect tracing system 100 comprises: a surface state image acquisition module 110 for an IC substrate to be detected, which is used to acquire a surface state image of the IC substrate to be detected collected by a camera; a defective type IC substrate sample library image extraction module 120, which is used to extract a set of IC substrate sample library images marked with defective types from a sample library; a defective type image feature extraction module 130, which is used to input each IC substrate sample library image marked with defective types in the set of IC substrate sample library images marked with defective types into an image feature extractor to obtain a set of IC substrate defect type feature maps; a feature distribution gradient mask significant module 140, which is used to extract each IC substrate sample library image marked with defective types in the set of IC substrate defect type feature maps; The defect type feature map is input into the feature distribution gradient mask salient device to obtain a set of IC carrier defect type salient feature maps; an IC carrier surface state feature extraction module 150 is used to input the surface state image of the IC carrier to be detected into the image feature extractor and the feature distribution gradient mask salient device in sequence to obtain the surface state feature map of the IC carrier to be detected; a similarity calculation module 160 is used to calculate the similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type salient feature map in the set of IC carrier defect type salient feature maps to obtain a set of similarities; a defect type detection result determination module 170 is used to obtain the defect type detection result of the IC carrier to be detected based on the set of similarities.

[0068] Here, those skilled in the art can understand that the specific operations of each module in the above IC substrate production defect tracing system have been referenced above. Figures 1 to 5 The description of the defect tracing method for bad IC substrate production has been introduced in detail, and therefore, its repeated description will be omitted.

[0069] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0070] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0071] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0072] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0073] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the device claim can also be implemented by one unit through software or hardware.

[0074] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for detecting defects in IC substrate production, characterized in that: include: Acquire a surface state image of the IC substrate to be inspected captured by a camera; Extracting a set of IC substrate sample library images annotated with defect types from the sample library; Inputting each IC carrier board sample library image with a defect type in the set of IC carrier board sample library images with a defect type respectively into an image feature extractor to obtain a set of IC carrier board defect type feature maps; Inputting each IC carrier defect type feature map in the set of IC carrier defect type feature maps into a feature distribution gradient mask salient device to obtain a set of IC carrier defect type salient feature maps; The surface state image of the IC carrier to be detected is sequentially input into the image feature extractor and the feature distribution gradient mask highlighter to obtain a surface state feature map of the IC carrier to be detected; Calculating the similarity between the surface state feature map of the IC substrate to be detected and each significant feature map of the IC substrate defect type in the set of significant feature maps of the IC substrate defect type to obtain a set of similarities; Based on the set of similarities, a defect type detection result of the IC substrate to be detected is obtained.

2. The IC substrate production defect detection method according to claim 1, characterized in that: Inputting each IC carrier board sample library image marked with defect type in the set of IC carrier board sample library images marked with defect type into an image feature extractor to obtain a set of IC carrier board defect type feature maps, including: inputting each IC carrier board sample library image marked with defect type in the set of IC carrier board sample library images marked with defect type into an image feature extractor based on a deep neural network model to obtain a set of IC carrier board defect type feature maps.

3. The IC substrate production defect detection method according to claim 2, characterized in that: The image feature extractor based on the deep neural network model is an image feature extractor based on the hole convolutional neural network model.

4. The IC substrate production defect detection method according to claim 3, characterized in that: Inputting each IC substrate defect type feature map in the set of IC substrate defect type feature maps into a feature distribution gradient mask salient device to obtain a set of IC substrate defect type salient feature maps, including: Calculating the multi-directional gradient value distribution of each position in the IC substrate defect type characteristic map, and determining the gradient amplitude value of each position in the IC substrate defect type characteristic map based on the multi-directional gradient value distribution of each position to obtain an IC substrate defect type characteristic gradient amplitude distribution characteristic map; Calculating the local description operator of the gradient amplitude at each position in the IC substrate defect type characteristic gradient amplitude distribution characteristic map to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution characteristic map; Inputting the IC substrate defect type characteristic gradient amplitude local significant distribution feature map into a gated masker based on a GELU function to obtain an IC substrate defect type characteristic gradient amplitude local significant gated mask feature map; The IC substrate defect type significant feature map is obtained by calculating the position point multiplication between the local significant gated mask feature map of the IC substrate defect type characteristic gradient amplitude and the IC substrate defect type feature map.

5. The IC substrate production defect detection method according to claim 4, characterized in that: Calculating the multi-directional gradient value distribution of each position in the IC substrate defect type characteristic map, and determining the gradient amplitude value of each position in the IC substrate defect type characteristic map based on the multi-directional gradient value distribution of each position to obtain the IC substrate defect type characteristic gradient amplitude distribution characteristic map, including: Calculating the gradient value of each characteristic value in the IC substrate defect type characteristic map along the width direction to obtain the IC substrate defect type width direction characteristic gradient value characteristic map; Calculating the gradient value of each characteristic value in the IC substrate defect type characteristic map along the height direction to obtain the IC substrate defect type height direction characteristic gradient value characteristic map; Calculating the gradient value of each characteristic value in the IC substrate defect type characteristic map along the channel direction to obtain the IC substrate defect type channel direction characteristic gradient value characteristic map; The square root of the sum of the squares of the characteristic values ​​at corresponding positions in the width direction characteristic gradient value characteristic map of the IC carrier defect type, the height direction characteristic gradient value characteristic map of the IC carrier defect type, and the channel direction characteristic gradient value characteristic map of the IC carrier defect type is calculated to obtain the IC carrier defect type characteristic gradient amplitude distribution characteristic map.

6. The IC substrate production defect detection method according to claim 5, characterized in that: Calculating the local description operator of the gradient amplitude at each position in the IC substrate defect type characteristic gradient amplitude distribution characteristic map to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution characteristic map, including: Determine a local neighborhood range of the gradient amplitude, and calculate a sequence of IC substrate defect type characteristic gradient amplitude neighborhood distribution characteristic maps based on the local neighborhood range of the gradient amplitude and the IC substrate defect type characteristic gradient amplitude distribution characteristic map; Counting the number of feature graphs in the sequence of the IC substrate defect type feature gradient amplitude neighborhood distribution feature graphs to obtain a statistical value of the number of IC substrate defect type feature gradient amplitude neighborhoods; Calculating the position difference between each IC substrate defect type characteristic gradient amplitude neighborhood distribution feature map in the sequence of the IC substrate defect type characteristic gradient amplitude distribution feature map and the IC substrate defect type characteristic gradient amplitude neighborhood distribution feature map, and dividing the difference by position point with the IC substrate defect type characteristic gradient amplitude distribution feature map to obtain a sequence of IC substrate defect type characteristic gradient relative amplitude distribution neighborhood characteristic maps; Each IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature map in the sequence of the IC substrate defect type characteristic gradient relative amplitude distribution neighborhood feature map is added point by point and then divided by the IC substrate defect type characteristic gradient amplitude neighborhood statistical value by point to obtain the IC substrate defect type characteristic gradient amplitude local significant distribution feature map.

7. The IC substrate production defect detection method according to claim 6, characterized in that: The surface state image of the IC carrier to be detected is sequentially input into the image feature extractor and the feature distribution gradient mask salient device to obtain the surface state feature map of the IC carrier to be detected, including: inputting the surface state image of the IC carrier to be detected into the image feature extractor based on the void convolutional neural network model and the feature distribution gradient mask salient device to obtain the surface state feature map of the IC carrier to be detected.

8. The IC substrate production defect detection method according to claim 7, characterized in that: Calculating the similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps to obtain a set of similarities, including: calculating the hash similarity between the surface state feature map of the IC carrier to be detected and each IC carrier defect type significant feature map in the set of IC carrier defect type significant feature maps to obtain a set of hash similarities as the set of similarities.

9. The IC substrate production defect detection method according to claim 8, characterized in that: Based on the set of similarities, a defect type detection result of the IC carrier to be detected is obtained, including: taking the defect type of the IC carrier sample library image marked with the defect type corresponding to the maximum hash similarity in the set of hash similarities as the defect type detection result of the IC carrier to be detected.

Images