Image Processing-Based Surface Anomaly Detection and Processing System for Precision Semiconductor Parts
The Canny algorithm, which uses local gradient significance calculation and adaptive threshold optimization, solves the problem of insufficient accuracy in detecting surface anomalies of precision semiconductor parts, achieving higher detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHANGSHU ZHAOHENGZHONGLI PRECISION MASCH CO LTD
- Filing Date
- 2026-05-11
- Publication Date
- 2026-07-31
AI Technical Summary
The existing Canny algorithm is not accurate enough in detecting anomalies on the surface of precision semiconductor parts, and it is difficult to adapt to edge extraction under different materials and angles, resulting in frequent false detections or missed detections.
Significant pixels are selected by the local gradient magnitude saliency calculation module, target regions are formed by superpixel segmentation and clustering, the high and low thresholds of the Canny algorithm are adaptively adjusted for edge detection, and edge extraction is optimized by combining local density features.
It improves the accuracy and robustness of surface anomaly detection for precision semiconductor parts, overcomes the limitations of traditional global fixed threshold methods, and enhances the integrity and detection accuracy of edge extraction.
Smart Images
Figure CN122156215B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to an image processing-based system for detecting and processing surface anomalies in precision semiconductor parts. Background Technology
[0002] With the rapid development of the semiconductor industry, the quality requirements for precision semiconductor components are increasing. Especially in the production of microelectronic components, the detection and handling of surface defects has become a crucial step in ensuring product quality. Image processing and machine vision technologies can efficiently detect surface defects in precision semiconductor components by acquiring and analyzing images of their surfaces.
[0003] However, surface defects (such as cracks and chipped edges) in precision semiconductor components exhibit diverse morphologies and can be influenced by factors such as lighting, material properties, and surface reflection. Surface texture features at different locations can vary significantly, and the texture characteristics of defects also differ. This presents a challenge for commonly used image-based defect detection algorithms, such as the Canny algorithm, in extracting defect edges. Specifically, the Canny algorithm relies on global and fixed high and low thresholds to distinguish edges. However, these global and fixed thresholds are difficult to adapt to the edge features at different locations on the component surface, especially when the edge structure features of defects or devices are affected by angle or lighting. This can lead to inaccurate edge extraction for Canny algorithms when dealing with surface edges of different materials or angles, resulting in false positives or false negatives and reducing detection accuracy and robustness.
[0004] In other words, the Canny algorithm currently used for surface defect detection has a technical problem of insufficient detection accuracy when it comes to detecting surface anomalies in precision semiconductor parts. Summary of the Invention
[0005] In view of this, the present invention provides an image processing-based surface anomaly detection system for precision semiconductor parts, in order to solve the technical problem of insufficient accuracy of the current Canny algorithm in detecting surface anomalies of precision semiconductor parts.
[0006] The present invention provides an image processing-based surface anomaly detection and processing system for precision semiconductor parts, comprising: The local gradient magnitude significance calculation module is used to acquire grayscale images of the surface of semiconductor precision parts as images to be analyzed, and to determine the local gradient magnitude significance of any pixel based on the gradient vector of each pixel in the image to be analyzed. The pixel selection and determination module is used to arrange the local gradient magnitude saliency of each pixel in order to form a saliency sequence, determine the saliency segmentation value based on the sequence stability and value difference of the two subsequences obtained by segmenting the saliency sequence according to the local gradient magnitude saliency of each pixel in the saliency sequence, and select the pixels whose local gradient magnitude saliency is greater than the saliency segmentation value as the selected pixels. The target region determination module is used to segment the image to be analyzed into sub-blocks of a preset size, determine the clustering distance between any two sub-blocks based on the difference in the significance of the local gradient magnitude of each selected pixel in any two sub-blocks and the difference in the gradient magnitude, cluster all the sub-blocks based on the clustering distance, merge the sub-blocks that belong to the same cluster and are adjacent into a new sub-block, and take each sub-block as the target region. An anomaly detection module is used to determine the high and low thresholds of the Canny algorithm for any target area based on each of the selected pixels in the target area and to complete the edge detection of the target area. Based on the edge detection results of each target area, the module completes the surface anomaly detection of the semiconductor precision parts.
[0007] Furthermore, pixels with non-zero gradient magnitudes in the image to be analyzed are taken as pixels to be analyzed, and only the significance of the local gradient magnitude of each pixel to be analyzed is calculated.
[0008] Furthermore, determining the significance of the local gradient magnitude at any pixel point includes: A predetermined number of reference pixels are selected along the straight line containing the gradient vector of any pixel. Based on the gradient magnitude of any pixel and each of the reference pixels, the normalized gradient magnitude of each pixel and each of the reference pixels is determined. Based on the normalized gradient magnitude, the significance of the local gradient magnitude of any pixel is calculated.
[0009] Furthermore, the calculation of the local gradient magnitude significance of any pixel based on each of the normalized gradient magnitudes includes: The normalized gradient magnitude of any pixel and the normalized gradient magnitude of each of the reference pixels constitute the set of normalized gradient magnitudes corresponding to any pixel. The absolute value of the difference between the maximum value of the normalized gradient magnitude in the set of normalized gradient magnitudes corresponding to any pixel and the normalized gradient magnitude of any pixel is recorded as the local upper limit difference of the gradient magnitude of any pixel. The difference between the normalized gradient magnitude of any pixel and the mean of the normalized gradient magnitudes of each reference pixel is recorded as the local average difference of the gradient magnitude of any pixel. The local gradient magnitude significance of any pixel is determined based on the local upper limit difference of the gradient magnitude and the local average difference of the gradient magnitude of any pixel. The local gradient magnitude significance of any pixel is inversely proportional to the local upper limit difference of the gradient magnitude and directly proportional to the local average difference of the gradient magnitude.
[0010] Furthermore, determining the saliency segmentation value includes: The two subsequences obtained by dividing the saliency sequence by the saliency magnitude of any local gradient are denoted as the first subsequence and the second subsequence, respectively. The absolute value of the difference between the mean of the magnitude significance of each local gradient in the first subsequence and the mean of the magnitude significance of each local gradient in the second subsequence is denoted as the sequence value difference degree between the first subsequence and the second subsequence. The sum of the coefficients of variation of the first subsequence and the coefficients of variation of the second subsequence is denoted as the sequence instability of the first subsequence and the second subsequence. The confidence level of the segmentation value of any local gradient magnitude significance is determined based on the sequence value difference and the sequence instability. The confidence level of the segmentation value is directly proportional to the sequence value difference and inversely proportional to the sequence instability. The local gradient magnitude significance corresponding to the confidence level of the maximum segmentation value in the significance sequence is used as the significance segmentation value.
[0011] Further, determining the clustering distance between any two sub-blocks includes: The two sub-blocks are respectively designated as the first sub-block and the second sub-block. The normalized value of the absolute value of the difference between the mean local gradient magnitude significance of each selected pixel in the first sub-block and the mean local gradient magnitude significance of each selected pixel in the second sub-block is calculated and denoted as the first distance characterization value between the two sub-blocks. Calculate the normalized value of the absolute value of the difference between the mean gradient magnitude of each selected pixel in the first sub-block and the mean gradient magnitude of each selected pixel in the second sub-block, and record it as the second distance characterization value between any two sub-blocks; The clustering distance between any two sub-blocks is determined based on the first distance representation value and the second distance representation value, and the clustering distance between any two sub-blocks is proportional to both the first distance representation value and the second distance representation value.
[0012] Furthermore, determining the high and low thresholds for the Canny algorithm used for the target region includes: A local density feature vector of any selected pixel is constructed based on the local gradient magnitude significance of any selected pixel in any target region and the position of any selected pixel. The local density value of any selected pixel is determined based on the local density feature vectors of all selected pixels in any target region. Calculate the product of the local density value and the local gradient magnitude significance of any selected pixel, construct a first weight of any selected pixel that is proportional to the product, determine the first weighted average of the local gradient magnitude significance of all selected pixels in any target region based on the first weight corresponding to each selected pixel in any target region, and use the first weighted average as the high threshold of the Canny algorithm for any target region; Construct a second weight for any selected pixel that is inversely proportional to the product. Determine a second weighted average of the local gradient magnitude significance of all selected pixels in any target region based on the second weight corresponding to each selected pixel in any target region. Use the second weighted average as the low threshold of the Canny algorithm for any target region.
[0013] Furthermore, constructing the local density feature vector of any selected pixel includes: Calculate the normalized value of the local gradient magnitude significance, the normalized value of the x-coordinate, and the normalized value of the y-coordinate of any selected pixel. Use the normalized value of the local gradient magnitude significance, the normalized value of the x-coordinate, and the normalized value of the y-coordinate as the three dimensions of the local density feature vector of any selected pixel to complete the construction of the local density feature vector of any selected pixel.
[0014] Further, determining the local density value of any of the selected pixels includes: The cosine similarity between the local density feature vector of any selected pixel and the local density feature vector of any other selected pixel within any target region is calculated as the density distance between the selected pixel and the other selected pixel. The mean of the density distances between the selected pixel and all other selected pixels within any target region is recorded as the average density distance corresponding to the selected pixel. The local density value of the selected pixel is determined based on the average density distance. The local density value of the selected pixel is inversely proportional to the average density distance.
[0015] Furthermore, based on the edge detection results of each target region, surface anomaly detection of the semiconductor precision component is completed, including: The edge detection results of each target region are used to form the edge detection result map of the image to be analyzed. The edge detection result map corresponding to the surface image of the semiconductor precision part with no surface abnormalities is obtained according to the method of obtaining the edge detection result map of the image to be analyzed and recorded as the template image. The edge detection result map of the image to be analyzed is compared with the template image to complete the surface abnormality detection of the semiconductor precision part.
[0016] The advantages of this invention compared to the prior art are: This invention first identifies selectable pixels with significant local gradients in the image to be analyzed. Based on these selectable pixels, the image is divided into multiple sub-blocks using superpixel segmentation, and these sub-blocks are clustered into different clusters. Based on the clustering results, multiple target regions with different degrees of edge texture features are formed. Then, within each target region, a local density value reflecting the edge characteristics of each selectable pixel is constructed by combining the continuous positional distribution characteristics of the edge texture and the local saliency of the grayscale gradient magnitude. Based on the local density values of each selectable pixel, high and low thresholds for Canny edge detection are determined for the target regions. Based on the adaptive edge extraction results of each target region, the edge detection results of the image to be analyzed are obtained, and surface anomaly detection is completed. This invention can improve the accuracy of surface anomaly detection for precision semiconductor parts based on the obtained adaptive edge detection thresholds. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a structural block diagram of a semiconductor precision component surface anomaly detection and processing system based on image processing, provided in Embodiment 1 of the present invention. Detailed Implementation
[0019] The overall concept of this invention is as follows: The threshold determination strategy in the Canny edge detection algorithm is optimized so that the threshold can be adaptively adjusted according to the image characteristics of different regions. This better adapts to the different material surfaces and complex textures of semiconductor precision parts, improves the integrity and accuracy of edge extraction, overcomes the limitations of the traditional global fixed threshold method, and enhances the robustness and detection accuracy of anomaly detection on the surface of semiconductor precision parts.
[0020] To further illustrate the technical solution of the present invention, specific embodiments are described below.
[0021] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a particular feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, a particular feature, structure, or characteristic in one or more embodiments may be combined in any suitable form, and the terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized.
[0022] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0023] System Implementation Example: See Figure 1 This is a structural block diagram of a semiconductor precision component surface anomaly detection and processing system based on image processing, provided in Embodiment 1 of the present invention. Figure 1 As shown, the system includes a local gradient magnitude significance calculation module 11, a pixel point selection and determination module 12, a target region determination module 13, and an anomaly detection module 14. The following describes each module in detail: The local gradient magnitude significance calculation module 11 is used to acquire a grayscale image of the surface of a semiconductor precision part as the image to be analyzed, and to determine the local gradient magnitude significance of any pixel based on the gradient vector of each pixel in the image to be analyzed.
[0024] Semiconductor precision components are typically made from composite materials such as silicon (Si), gallium arsenide (GaAs), gallium nitride (GaN), ceramics, and metal alloys. During processing and assembly, various types of defects may occur on the component surface, including cracks, chipping, and surface scratches, with significant differences in morphology, size, and distribution. Furthermore, due to the surface texture characteristics of different materials and their varying reflectivity under different shooting angles and lighting conditions, the grayscale changes and contrast of the component surface edges are inconsistent. This leads traditional edge detection algorithms, such as the Canny edge detection algorithm, to easily misidentify incomplete normal edges as defects or fail to adequately extract true defect edges, resulting in false positives or false negatives.
[0025] The purpose of this invention is to optimize the threshold setting of the Canny edge detection algorithm in the defect detection and analysis process, so as to adapt to the changes in different positions, materials and complex textures on the surface of semiconductor precision parts, thereby accurately extracting normal edges and defect edges, achieving high-precision identification of defects such as cracks and chipping, overcoming the problem of false detection or missed detection in edge extraction by traditional fixed threshold methods, thereby improving the detection accuracy of surface anomalies of semiconductor precision parts and the reliability of the detection system.
[0026] To this end, this system first acquires surface images of the precision semiconductor parts to be inspected using a high-speed, high-definition industrial camera. Then, the acquired surface images are denoised and converted to grayscale to obtain the image to be analyzed. This avoids noise affecting image quality during acquisition and improves the reliability of subsequent inspections. Noise denoising can be performed using any feasible method, such as bilateral filtering. Next, the Sobel operator is used on the preprocessed image to obtain the gradient magnitude and direction of each pixel, resulting in the gradient vector for each pixel. Since the image to be analyzed is a grayscale image, the gradient magnitude of each pixel determined here is the grayscale gradient magnitude, and the gradient vector is the grayscale gradient vector.
[0027] Based on the gradient magnitude of each pixel in the image to be analyzed, if any pixel's gradient magnitude is more prominent or significant compared to other pixels in its local neighborhood, then it may be an edge pixel such as a texture or structure. Correspondingly, considering that the gradient magnitude of pixels representing real semiconductor structure edges, textures, and defect edges in the image to be analyzed must be non-zero or greater than zero, in a preferred embodiment, the objects for examining the significance of the gradient magnitude of such pixels compared to the gradient magnitudes of other pixels in their local neighborhood are initially selected. Pixels with non-zero gradient magnitudes in the image to be analyzed are taken as the pixels to be analyzed, and only the significance of the local gradient magnitude of each pixel to be analyzed is calculated.
[0028] Of course, in this embodiment, the preliminary selection of pixels can be omitted, and the significance of the local gradient magnitude of any pixel in the image to be analyzed can be determined directly based on the gradient vector of each pixel in the image to be analyzed, or the gradient magnitude in the gradient vector, including: A predetermined number of reference pixels are selected along the straight line containing the gradient vector of any pixel. Based on the gradient magnitude of any pixel and each of the reference pixels, the normalized gradient magnitude of each pixel and each of the reference pixels is determined. Based on the normalized gradient magnitude, the significance of the local gradient magnitude of any pixel is calculated.
[0029] The reason for preferentially selecting a predetermined number of reference pixels symmetrical about any given pixel along the gradient vector (i.e., the gradient direction line) is that surface defects, edges, or textures are generally perpendicular to the gradient vectors of pixels on them. The more prominent the gradient magnitude of any given pixel is relative to the gradient magnitude of the reference pixels selected in this way, the more likely it is to be a pixel on a surface defect, edge, or texture. Of course, in other embodiments, any other feasible method of selecting reference pixels can be used, such as directly selecting the neighboring pixels in the 8-neighborhood or 16-neighborhood of any given pixel as reference pixels, etc. The predetermined number can be specifically determined according to the image resolution and the accuracy requirements of surface anomaly detection. In this embodiment, the predetermined number is preferably 8. Correspondingly, four reference pixels are selected before and after the given pixel along the gradient vector line, with the given pixel as the symmetrical point. At this time, the given pixel and its corresponding reference points constitute the local pixel set corresponding to the given pixel.
[0030] Furthermore, the reason for normalizing the gradient magnitude of any current pixel and its reference pixels is that edge textures or defects may change due to factors such as illumination angle. To avoid analysis errors caused by the different degrees of influence of such changes at different locations in the image to be analyzed, this embodiment preferably normalizes the gradient magnitude of each pixel in the local pixel set corresponding to any pixel during the saliency quantification process, so as to eliminate the difference in influence caused by illumination or angle at different locations in the image to be analyzed.
[0031] Specifically, the method for normalizing the gradient magnitude of each pixel in the local pixel set corresponding to any given pixel is as follows:
[0032] in, This represents the normalized gradient magnitude of the current pixel within the local set of pixels corresponding to any given pixel. This represents the gradient magnitude of the current pixel, that is, the original gradient magnitude before normalization. It represents the set of gradient magnitudes formed by the gradient magnitudes of all pixels in the local set of pixels corresponding to any given pixel. This represents the maximum gradient magnitude within the local set of pixels corresponding to any given pixel. It represents the minimum gradient magnitude in the local set of pixels corresponding to any given pixel.
[0033] This allows us to obtain the normalized gradient magnitude of any pixel and its corresponding reference pixels. Then, based on these normalized gradient magnitudes, we can calculate the significance of the local gradient magnitude of any pixel, including: The normalized gradient magnitude of any pixel and the normalized gradient magnitude of each of the reference pixels constitute the set of normalized gradient magnitudes corresponding to any pixel. The absolute value of the difference between the maximum value of the normalized gradient magnitude in the set of normalized gradient magnitudes corresponding to any pixel and the normalized gradient magnitude of any pixel is recorded as the local upper limit difference of the gradient magnitude of any pixel. The difference between the normalized gradient magnitude of any pixel and the mean of the normalized gradient magnitudes of each reference pixel is recorded as the local average difference of the gradient magnitude of any pixel. The local gradient magnitude significance of any pixel is determined based on the local upper limit difference of the gradient magnitude and the local average difference of the gradient magnitude of any pixel. The local gradient magnitude significance of any pixel is inversely proportional to the local upper limit difference of the gradient magnitude and directly proportional to the local average difference of the gradient magnitude.
[0034] Furthermore, as a preferred embodiment, the significance of the local gradient magnitude is:
[0035] in, This represents the significance of the local gradient magnitude at the i-th pixel in the image to be analyzed. Represents the natural constant. This indicates taking the absolute value. This represents the set of normalized gradient magnitudes corresponding to the i-th pixel, consisting of the normalized gradient magnitudes of the i-th pixel and its various reference pixels. This represents the maximum normalized gradient magnitude in the set of normalized gradient magnitudes corresponding to the i-th pixel. This represents the normalized gradient magnitude of the i-th pixel. This represents the mean of the normalized gradient magnitudes of all reference pixels for the i-th pixel. In other words, it is the mean of the normalized gradient magnitudes of all reference pixels remaining in the set of normalized gradient magnitudes corresponding to the i-th pixel after removing the normalized gradient magnitude of the i-th pixel. This represents the absolute value of the difference between the maximum normalized gradient magnitude in the set of normalized gradient magnitudes corresponding to the i-th pixel and the normalized gradient magnitude of the i-th pixel. The smaller this value is, the better. The larger the value, the closer the gradient magnitude of the i-th pixel is to the maximum gradient magnitude in its corresponding local pixel set within the local region formed by the i-th pixel and its reference pixel. Correspondingly, the i-th pixel is more likely to be the pixel with the largest or most prominent gradient magnitude in its corresponding local pixel set, and its gradient magnitude significance is higher. This represents the difference between the normalized gradient magnitude of the i-th pixel and the average normalized gradient magnitude of all reference pixels. The larger this value, the greater the difference. The larger the value, the greater the difference between the gradient magnitude of the i-th pixel and the gradient magnitude of its reference pixels. Correspondingly, the greater the significance of the local gradient magnitude of the i-th pixel.
[0036] The pixel selection module 12 is used to arrange the local gradient magnitude saliency of each pixel in order to form a saliency sequence, determine the saliency segmentation value based on the sequence stability and value difference of the two subsequences obtained by dividing the saliency sequence by the local gradient magnitude saliency of each pixel, and select the pixel whose local gradient magnitude saliency is greater than the saliency segmentation value as the selected pixel.
[0037] The aforementioned modules can obtain the gradient magnitude significance of any pixel in the image to be analyzed. Pixels with low significance may be areas with slow grayscale changes on the device surface or rough surfaces, i.e., they do not belong to edge or defect texture regions. Therefore, in order to identify pixels belonging to edge or defect texture regions and obtain the gradient features at these regions, so as to adaptively determine the threshold for subsequent texture edge extraction in different regions, this module identifies pixels with larger local gradient magnitude significance based on the local gradient magnitude significance of each pixel, and uses these pixels as potential edge or defect texture regions for subsequent analysis.
[0038] To achieve the filtering, this embodiment arranges the local gradient magnitude saliency of each pixel in order to form a saliency sequence. A saliency segmentation value is determined based on the sequence stability and value difference between the two subsequences obtained by dividing the saliency sequence according to the local gradient magnitude saliency of each pixel. Pixels with a local gradient magnitude saliency greater than the saliency segmentation value are selected as the filtered pixels. The order can be ascending or descending.
[0039] It is easy to understand that the larger the local gradient magnitude significance G value, the greater the probability that the pixel is a defective or edge texture region. Conversely, the smaller the local gradient magnitude significance G value, the less likely the pixel is a defective or edge texture region, and the greater the probability that it is a pixel in a region with uniform gray-scale variation on the surface of a precision semiconductor component. At the same time, because the texture features of defective or edge texture regions and regions with uniform gray-scale variation on the surface are significantly different, the local gradient magnitude significance G values of the two types of pixels corresponding to these two regions will also differ greatly. However, the local gradient magnitude significance G values of pixels of the same type are relatively similar.
[0040] Furthermore, since this embodiment arranges the local gradient magnitude saliency G values of each pixel in the image to be analyzed into a saliency sequence, this saliency sequence can be divided into two subsequences based on the local gradient magnitude saliency G values of each pixel. The degree of difference in values between the two subsequences and the stability of the G values within each subsequence are observed to characterize the suitability of the current pixel's local gradient magnitude saliency G value as the true segmentation value of the saliency sequence. That is, the suitability of the current pixel's local gradient magnitude saliency G value as the saliency segmentation value for distinguishing the local gradient magnitude saliency G values of the two types of pixels can be determined. Therefore, determining the saliency segmentation value includes: The two subsequences obtained by dividing the saliency sequence by the saliency magnitude of any local gradient are denoted as the first subsequence and the second subsequence, respectively. The absolute value of the difference between the mean of the magnitude significance of each local gradient in the first subsequence and the mean of the magnitude significance of each local gradient in the second subsequence is denoted as the sequence value difference degree between the first subsequence and the second subsequence. The sum of the coefficients of variation of the first subsequence and the coefficients of variation of the second subsequence is denoted as the sequence instability of the first subsequence and the second subsequence. The confidence level of the segmentation value of any local gradient magnitude significance is determined based on the sequence value difference and the sequence instability. The confidence level of the segmentation value is directly proportional to the sequence value difference and inversely proportional to the sequence instability. The local gradient magnitude significance corresponding to the confidence level of the maximum segmentation value in the significance sequence is used as the significance segmentation value.
[0041] Furthermore, as a preferred embodiment, the confidence level of the segmentation value of the significance of any local gradient magnitude is:
[0042] in, The confidence level of the segmentation value representing the significance of the s-th local gradient magnitude in the significance sequence. This indicates taking the absolute value. This represents the mean function. This represents the subsequence formed by the local gradient magnitude significance values to the left of the s-th local gradient magnitude significance value in the significance sequence, which is also the first subsequence or the second subsequence mentioned above. This represents the subsequence formed by the local gradient magnitudes to the right of the s-th local gradient magnitude significance value in the significance sequence, which is also the second subsequence or the first subsequence mentioned above. This represents the mean of the local gradient magnitude significance of the subsequence formed by all local gradient magnitude significance values to the left of the s-th local gradient magnitude significance value in the significance sequence. This represents the mean of the local gradient magnitude significance of the subsequence formed by all local gradient magnitude significance values to the right of the s-th local gradient magnitude significance value in the significance sequence. This represents the absolute value of the difference between the means of two subsequences. The larger this value, the greater the difference between the two subsequences, and the greater the confidence of the local gradient magnitude used to obtain these two subsequences in the current segmentation, or in other words, the more reliable the true segmentation value. This represents the coefficient of variation of the subsequence formed by the local gradient magnitudes to the left of the s-th local gradient magnitude significance value in the significance sequence. denoted as the coefficient of variation of the subsequence formed by the local gradient magnitudes to the right of the s-th local gradient magnitude significance value in the significance sequence. This represents the sum of the coefficients of variation of the two subsequences. The smaller this value, the more stable the division between the two subsequences after using the significance of the current local gradient magnitude as the dividing point. The larger the value, the more salience of the local gradient magnitude used to obtain the two subsequences is optimal, or the greater the credibility of the true segmentation value.
[0043] Then select the saliency sequence The pixel with the largest G value is grouped into one category with the pixels to its left and another with the pixels to its right. The pixel with the largest G value is then selected as the filter pixels for further analysis.
[0044] The target region determination module 13 is used to divide the image to be analyzed into sub-blocks of a preset size, determine the clustering distance between any two sub-blocks based on the difference in the local gradient magnitude of each selected pixel in any two sub-blocks and the difference in gradient magnitude, cluster all the sub-blocks based on the clustering distance, merge the adjacent sub-blocks belonging to the same cluster into new sub-blocks, and use each sub-block as the target region.
[0045] Since the edge texture intensity varies at different locations, the optimal edge detection threshold may also differ. Therefore, to accurately obtain adaptive thresholds for edge detection in different regions, this module divides the image to be analyzed into sub-blocks of a preset size, resulting in a corresponding number of sub-blocks. The preset size of the sub-blocks can be set in a balanced manner, taking into account both the accuracy requirements for surface anomaly detection and the computational efficiency requirements. In this embodiment, a 3×3 sub-block is preferred, which is the minimum preset size. The minimum size is used to divide the image to be analyzed to maximize the sensitivity for detecting minute defects on the surface of precision semiconductor parts. Therefore, this value is selected in this embodiment.
[0046] After dividing the image to be analyzed into a corresponding number of sub-blocks, the sub-blocks are used as the unit of analysis. By analyzing the similarity between different sub-blocks, the pixels that are suspected to be edges, i.e., the sub-blocks where the previously selected pixels are located, are clustered into different clusters. The same cluster indicates that the gray-level gradient magnitude and the significant features of the pixels containing textures are similar within these sub-blocks. Then, using the idea of superpixel segmentation, the actual positions of these sub-blocks in the same cluster are judged to be adjacent and merged. Each merged sub-block and the original unmerged sub-block are divided into different regions. All clusters are operated in the same way to obtain the various regions on the image to be analyzed.
[0047] During segmentation, since the selected pixels are those located in areas where textures or defects may exist, and the non-selected pixels are smooth areas on the surface of precision semiconductor components, they do not need to be analyzed. Therefore, this embodiment sets up a method to quantify the clustering distance between different sub-blocks during the clustering process based on the selected pixels contained in each sub-block. Correspondingly, the method for determining the clustering distance between any two sub-blocks includes: The two sub-blocks are respectively designated as the first sub-block and the second sub-block. The normalized value of the absolute value of the difference between the mean local gradient magnitude significance of each selected pixel in the first sub-block and the mean local gradient magnitude significance of each selected pixel in the second sub-block is calculated and denoted as the first distance characterization value between the two sub-blocks. Calculate the normalized value of the absolute value of the difference between the mean gradient magnitude of each selected pixel in the first sub-block and the mean gradient magnitude of each selected pixel in the second sub-block, and record it as the second distance characterization value between any two sub-blocks; The clustering distance between any two sub-blocks is determined based on the first distance representation value and the second distance representation value, and the clustering distance between any two sub-blocks is proportional to both the first distance representation value and the second distance representation value.
[0048] Furthermore, as a preferred embodiment, the clustering distance between any two sub-blocks is:
[0049] in, This represents the clustering distance between sub-blocks a and b, and is a metric distance indicating that sub-blocks a and b, which contain the selected pixels, can be classified into the same cluster. Represents the normalization function. This indicates taking the absolute value. This represents the mean of the local gradient magnitude significance of each selected pixel in sub-block a. This represents the mean of the local gradient magnitude significance of each selected pixel in sub-block b. This represents the absolute value of the difference between the mean values of the local gradient magnitude saliency G values of the selected pixels in the a-th and b-th sub-blocks. The smaller this value, the more similar the relative saliency of the texture features that may exist in these two sub-blocks, and the smaller the metric distance should be to classify them into the same cluster. This represents the average gradient magnitude of each selected pixel in sub-block a. This represents the average gradient magnitude of each selected pixel in sub-block b. This represents the absolute value of the difference between the mean values of the original gradient magnitudes of the selected pixels in the a-th and b-th sub-blocks. The smaller this value is, the more similar the texture gradients may be in these two sub-blocks, and the smaller the metric distance should be for classifying them into the same cluster.
[0050] After determining the clustering distance between any two sub-blocks, a clustering algorithm such as HDBSCAN is used to divide each sub-block into different clusters according to the distance metric used in the clustering process. It should be noted that HDBSCAN is an upgraded version of DBSCAN, with fewer parameters and stronger robustness. It only requires setting the `min_cluster_size` parameter, which is preferably set to 2, meaning the minimum feasible cluster size is 2. This is very suitable for identifying tiny defects composed of a very small number of edge points.
[0051] After dividing the image into sub-blocks as described above, it is determined whether different sub-blocks within the same cluster are adjacent in the image to be analyzed. If they are adjacent, they are merged into the same region as a new sub-block. The same operation is performed on the remaining clusters, that is, all clusters are traversed, and adjacent sub-blocks under each cluster are merged into new sub-blocks. Each of these sub-blocks is then used as the target region.
[0052] Anomaly detection module 14 is used to determine the high and low thresholds of the Canny algorithm for any target area based on each of the selected pixels in any target area and to complete the edge detection of the target area. Based on the edge detection results of each target area, it completes the surface anomaly detection of the semiconductor precision parts.
[0053] The traditional Canny edge detection algorithm includes two thresholds: a high threshold and a low threshold. The high threshold is used to directly identify reliable strong edge pixels, while the area between the high and low thresholds is used to filter weak edge pixels that are connected to strong edges. These weak edge pixels may be real edges or non-edges, while pixels below the low threshold are directly regarded as noise and discarded.
[0054] Based on the principle of constructing high and low thresholds for the Canny edge detection algorithm, and combined with the aforementioned principle for determining various target regions in this embodiment, the determination of high and low thresholds for the Canny algorithm used for target regions in this embodiment includes: A local density feature vector of any selected pixel is constructed based on the local gradient magnitude significance of any selected pixel in any target region and the position of any selected pixel. The local density value of any selected pixel is determined based on the local density feature vectors of all selected pixels in any target region. Calculate the product of the local density value and the local gradient magnitude significance of any selected pixel, construct a first weight of any selected pixel that is proportional to the product, determine the first weighted average of the local gradient magnitude significance of all selected pixels in any target region based on the first weight corresponding to each selected pixel in any target region, and use the first weighted average as the high threshold of the Canny algorithm for any target region; Construct a second weight for any selected pixel that is inversely proportional to the product. Determine a second weighted average of the local gradient magnitude significance of all selected pixels in any target region based on the second weight corresponding to each selected pixel in any target region. Use the second weighted average as the low threshold of the Canny algorithm for any target region.
[0055] Specifically, the reason for constructing the local density feature vector of the selected pixels and determining the local density value of the corresponding selected pixels based on the idea of local density is to combine the continuous positional distribution characteristics of edge pixels and the local saliency of grayscale gradient magnitude. In the local density values obtained based on the local density feature vector of the selected pixels, the larger the local density value, the more continuous the selected pixel is in space, and the higher its credibility as an edge pixel. At the same time, a larger local density value corresponds to a larger local gradient magnitude saliency G value for the selected pixel, thus increasing the probability that the selected pixel is a strong edge pixel. Therefore, the product of the local density value and the local gradient magnitude saliency of each selected pixel is used as the first weight of the corresponding selected pixel. The local gradient magnitude saliency G of each selected pixel is then weighted and averaged using the first weight of each selected pixel in a single target region. The result is used as the high threshold for the Canny algorithm for that target region. Simultaneously, to highlight the role of weak edge pixels in the low threshold determination process, the product of the local density value and the local gradient magnitude saliency of each selected pixel is used as an exponential decay function. The independent variable (i.e., x in the exponential decay function) is used to construct a weighting form that is the opposite of the first weight corresponding to the strong edge, i.e., the exponential decay function. As the second weight of the corresponding selected pixel, the local gradient magnitude significance G of each selected pixel is weighted and averaged using the second weight of each selected pixel in a single target region. The result is used as the low threshold of the Canny algorithm for the target region.
[0056] To eliminate the dimensional and magnitude differences between the local gradient magnitude significance feature and the location feature corresponding to the selected pixel, a local density feature vector for any selected pixel is constructed, including: Calculate the normalized value of the local gradient magnitude significance, the normalized value of the x-coordinate, and the normalized value of the y-coordinate of any selected pixel. Use the normalized value of the local gradient magnitude significance, the normalized value of the x-coordinate, and the normalized value of the y-coordinate as the three dimensions of the local density feature vector of any selected pixel to complete the construction of the local density feature vector of any selected pixel.
[0057] Based on the constructed local density feature vectors of each selected pixel in the current target region, and combined with existing local density construction ideas, the local density value of any selected pixel is determined, including: The cosine similarity between the local density feature vector of any selected pixel and the local density feature vector of any other selected pixel within any target region is calculated as the density distance between the selected pixel and the other selected pixel. The mean of the density distances between the selected pixel and all other selected pixels within any target region is recorded as the average density distance corresponding to the selected pixel. The local density value of the selected pixel is determined based on the average density distance. The local density value of the selected pixel is inversely proportional to the average density distance.
[0058] Through the above method, this embodiment comprehensively considers gradient intensity and spatial continuity at the local region scale of each target region, realizing the adaptive determination of the high and low thresholds for Canny edge detection for the corresponding target region. This ensures the continuity and integrity of the edge structure while suppressing noise interference, ultimately completing adaptive edge extraction for each target region. Therefore, surface anomaly detection of the semiconductor precision component can be completed based on the edge detection results of each target region, including: The edge detection results of each target region are used to form the edge detection result map of the image to be analyzed. The edge detection result map corresponding to the surface image of the semiconductor precision part with no surface abnormalities is obtained according to the method of obtaining the edge detection result map of the image to be analyzed and recorded as the template image. The edge detection result map of the image to be analyzed is compared with the template image to complete the surface abnormality detection of the semiconductor precision part.
[0059] Specifically, before comparing the edge detection result image of the image to be analyzed with the template image, the edge detection result image and the template image are first spatially aligned and scaled. For example, affine transformation and rotation operations are used to eliminate the influence of differences in shooting position, angle, and size. Then, in the aligned edge detection result image and template image, matching pairs are constructed for the nearest edge points one by one, the pixel position offset between corresponding edge points is calculated, and the offset is statistically analyzed. Since the edge offset should be close to zero and concentrated within a certain range under normal circumstances, methods such as three times the standard deviation can be used to determine the normal offset range. When the edge matching offset of a certain area exceeds the preset tolerance range, the area is marked as a suspected defect area.
[0060] All suspected defect areas undergo further morphological processing, such as expansion followed by erosion, to eliminate discrete noise and obtain complete and coherent defect areas. Subsequently, the defect type is determined by combining the spatial location, area, and morphological characteristics of the defect area. For example, abnormal areas located inside the device are marked as cracks or scratches, while abnormal areas located at the device edge are marked as edge chipping. Simultaneously, a filtering threshold is set based on the pixel area of the defect area. For example, in chip edge detection, connected regions with a pixel area less than 10 pixels can be considered noise and removed. If abnormal areas still exist after filtering, the device is determined to have a real defect, and an early warning is issued. The above thresholds and parameters can be flexibly adjusted according to the defect tolerance of the specific component and actual inspection requirements.
[0061] In this embodiment of the invention, when performing defect detection on the surface of a semiconductor device, in order to ensure the completeness of edge or texture acquisition, the features of the acquired image are analyzed to identify the screening pixels with prominent gradient features. Then, using the superpixel segmentation concept, the screening pixels are clustered and divided to obtain multiple target regions that all contain local edge texture features. Then, based on the local gradient saliency and positional features of the screening pixels in a single target region, the adaptive Canny edge detection high and low thresholds for that target region are determined. This optimizes and improves the existing high and low threshold confirmation strategy for Canny edge detection, avoiding the influence of a global and fixed threshold on the completeness of edge extraction in the analyzed image, making the acquisition of edges in the image to be analyzed more complete, thereby improving the accuracy and efficiency of detecting surface defects of the device.
[0062] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A surface anomaly detection and processing system for precision semiconductor parts based on image processing, characterized in that, The system includes: The local gradient magnitude significance calculation module is used to acquire grayscale images of the surface of semiconductor precision parts as images to be analyzed, and to determine the local gradient magnitude significance of any pixel based on the gradient vector of each pixel in the image to be analyzed. The pixel selection and determination module is used to arrange the local gradient magnitude saliency of each pixel in order to form a saliency sequence, determine the saliency segmentation value based on the sequence stability and value difference of the two subsequences obtained by segmenting the saliency sequence according to the local gradient magnitude saliency of each pixel in the saliency sequence, and select the pixels whose local gradient magnitude saliency is greater than the saliency segmentation value as the selected pixels. The target region determination module is used to segment the image to be analyzed into sub-blocks of a preset size, determine the clustering distance between any two sub-blocks based on the difference in the significance of the local gradient magnitude of each selected pixel in any two sub-blocks and the difference in the gradient magnitude, cluster all the sub-blocks based on the clustering distance, merge the sub-blocks that belong to the same cluster and are adjacent into a new sub-block, and take each sub-block as the target region. Anomaly detection module is used to determine the high and low thresholds of the Canny algorithm for any target area based on each of the selected pixels in any target area and to complete the edge detection of the target area. Based on the edge detection results of each target area, it completes the surface anomaly detection of the semiconductor precision parts. Determining the significance of the local gradient magnitude of any pixel includes: A predetermined number of reference pixels are selected on the straight line containing the gradient vector of any pixel as reference pixels. Based on the gradient magnitude of any pixel and each of the reference pixels, the normalized gradient magnitude of each pixel and each of the reference pixels is determined. Based on the normalized gradient magnitude, the significance of the local gradient magnitude of any pixel is calculated. The calculation of the local gradient magnitude significance of any pixel based on each of the normalized gradient magnitudes includes: The normalized gradient magnitude of any pixel and the normalized gradient magnitude of each of the reference pixels constitute the set of normalized gradient magnitudes corresponding to any pixel. The absolute value of the difference between the maximum value of the normalized gradient magnitude in the set of normalized gradient magnitudes corresponding to any pixel and the normalized gradient magnitude of any pixel is recorded as the local upper limit difference of the gradient magnitude of any pixel. The difference between the normalized gradient magnitude of any pixel and the mean of the normalized gradient magnitudes of each reference pixel is recorded as the local average difference of the gradient magnitude of any pixel. The local gradient magnitude significance of any pixel is determined based on the local upper limit difference of the gradient magnitude of any pixel and the local average difference of the gradient magnitude of any pixel. The local gradient magnitude significance of any pixel is inversely proportional to the local upper limit difference of the gradient magnitude of any pixel and directly proportional to the local average difference of the gradient magnitude of any pixel. The determination of the high and low thresholds for the Canny algorithm used for the target region includes: A local density feature vector of any selected pixel is constructed based on the local gradient magnitude significance of any selected pixel in any target region and the position of any selected pixel. The local density value of any selected pixel is determined based on the local density feature vectors of all selected pixels in any target region. Calculate the product of the local density value and the local gradient magnitude significance of any selected pixel, construct a first weight of any selected pixel that is proportional to the product, determine the first weighted average of the local gradient magnitude significance of all selected pixels in any target region based on the first weight corresponding to each selected pixel in any target region, and use the first weighted average as the high threshold of the Canny algorithm for any target region; Construct a second weight for any selected pixel that is inversely proportional to the product. Determine a second weighted average of the local gradient magnitude significance of all selected pixels in any target region based on the second weight corresponding to each selected pixel in any target region. Use the second weighted average as the low threshold of the Canny algorithm for any target region. The construction of the local density feature vector of any selected pixel includes: Calculate the normalized value of the local gradient magnitude significance, the normalized value of the x-coordinate, and the normalized value of the y-coordinate of any selected pixel. Use the normalized value of the local gradient magnitude significance, the normalized value of the x-coordinate, and the normalized value of the y-coordinate as the three dimensions of the local density feature vector of any selected pixel to complete the construction of the local density feature vector of any selected pixel. Determining the local density value of any of the selected pixels includes: The cosine similarity between the local density feature vector of any selected pixel and the local density feature vector of any other selected pixel within any target region is calculated as the density distance between the selected pixel and the other selected pixel. The mean of the density distances between the selected pixel and all other selected pixels within any target region is recorded as the average density distance corresponding to the selected pixel. The local density value of the selected pixel is determined based on the average density distance. The local density value of the selected pixel is inversely proportional to the average density distance.
2. The semiconductor precision component surface anomaly detection and processing system based on image processing according to claim 1, characterized in that, Pixels with non-zero gradient magnitudes in the image to be analyzed are taken as pixels to be analyzed, and the significance of the local gradient magnitude of each pixel to be analyzed is calculated.
3. The semiconductor precision component surface anomaly detection and processing system based on image processing according to claim 1, characterized in that, The determination of the saliency segmentation value includes: The two subsequences obtained by dividing the saliency sequence by the saliency magnitude of any local gradient are denoted as the first subsequence and the second subsequence, respectively. The absolute value of the difference between the mean of the magnitude significance of each local gradient in the first subsequence and the mean of the magnitude significance of each local gradient in the second subsequence is denoted as the sequence value difference degree between the first subsequence and the second subsequence. The sum of the coefficients of variation of the first subsequence and the coefficients of variation of the second subsequence is denoted as the sequence instability of the first subsequence and the second subsequence. The confidence level of the segmentation value of any local gradient magnitude significance is determined based on the sequence value difference and the sequence instability. The confidence level of the segmentation value is directly proportional to the sequence value difference and inversely proportional to the sequence instability. The local gradient magnitude significance corresponding to the confidence level of the maximum segmentation value in the significance sequence is used as the significance segmentation value.
4. The semiconductor precision component surface anomaly detection and processing system based on image processing according to claim 1, characterized in that, Determining the clustering distance between any two sub-blocks includes: The two sub-blocks are respectively designated as the first sub-block and the second sub-block. The normalized value of the absolute value of the difference between the mean local gradient magnitude significance of each selected pixel in the first sub-block and the mean local gradient magnitude significance of each selected pixel in the second sub-block is calculated and denoted as the first distance characterization value between the two sub-blocks. Calculate the normalized value of the absolute value of the difference between the mean gradient magnitude of each selected pixel in the first sub-block and the mean gradient magnitude of each selected pixel in the second sub-block, and record it as the second distance characterization value between any two sub-blocks; The clustering distance between any two sub-blocks is determined based on the first distance representation value and the second distance representation value, and the clustering distance between any two sub-blocks is proportional to both the first distance representation value and the second distance representation value.
5. The semiconductor precision component surface anomaly detection and processing system based on image processing according to claim 1, characterized in that, The surface anomaly detection of the semiconductor precision component is completed based on the edge detection results of each target region, including: The edge detection results of each target region are used to form the edge detection result map of the image to be analyzed. The edge detection result map corresponding to the surface image of the semiconductor precision part with no surface abnormalities is obtained according to the method of obtaining the edge detection result map of the image to be analyzed and recorded as the template image. The edge detection result map of the image to be analyzed is compared with the template image to complete the surface abnormality detection of the semiconductor precision part.