Chip side scratch detection method and system, product and medium

Abstract

The invention provides a chip side scratch detection method and system, a product and a medium, and relates to the technical field of image equipment. The region-of-interest guiding model is utilized to extract the scratch region corresponding to the smear mark from the original chip image, so that the difficulty of manual marking is reduced. Related personnel only need to simply smear marks on the scratch areas. According to the simplified labeling mode, labeling efficiency is improved, so that a worker can complete labeling work of more samples within the same time, sufficient training data is obtained, sufficient data support is provided for model training, and the accuracy of the model is improved.

Description

Technical Field

[0001] This application relates to the technical field of image devices, and particularly to a method, system, product and medium for detecting scratches on the side of a chip. Background Art

[0002] With the continuous development of integrated circuit technology, the requirements for product quality in the chip manufacturing process are getting higher and higher. During the chip production process, scratches may occur on the side of the chip due to processes such as cutting and handling. These scratches will affect the performance and reliability of the chip. Therefore, it is necessary to detect the scratches on the side of the chip in a timely and accurate manner.

[0003] Currently, the detection of scratches on the side of a chip mainly uses an image recognition method based on deep learning. This method first collects a large number of chip images, and then processes the collected images through image analysis software. According to features such as brightness and texture in the images, regions where scratches may exist are located, and these regions are used as training samples to input into a deep learning model for training.

[0004] However, the number of training samples obtained in this way is small, resulting in insufficient model training and thus lower accuracy. Summary of the Invention

[0005] This application provides a method, system, product and medium for detecting scratches on the side of a chip, which is used to obtain sufficient training data, provides sufficient data support for model training, and increases the accuracy rate of the model.

[0006] In a first aspect, this application provides a method for detecting scratches on the side of a chip, including: obtaining a chip image set to be trained, where the chip image set includes original chip images and their corresponding annotation images, and the annotation images are obtained by smearing and marking the scratch regions in the original chip images; performing region segmentation processing on the annotation images to extract regions of interest containing the smearing marks; training a chip side scratch detection model using supervised learning, and the training data includes: regions of interest, original chip images, chip images without scratches; where the regions of interest are used to enable the chip side scratch detection model to extract the scratch regions corresponding to the smearing marks from the corresponding original chip images; deploying the trained chip side scratch detection model to the production line to detect scratches on the on-line chips.

[0007] By adopting the above technical solution, using the regions of interest to guide the model to extract the scratch regions corresponding to the smearing marks from the original chip images reduces the difficulty of manual annotation. Relevant personnel only need to simply smear and mark the scratch regions. This simplified annotation method improves the annotation efficiency, enabling staff to complete the annotation work of more samples within the same time, thereby obtaining sufficient training data, providing sufficient data support for model training, and increasing the accuracy rate of the model.

[0008] In some embodiments in combination with some embodiments of the first aspect, before the step of obtaining a chip image set to be trained, the method further includes: obtaining a grazing illumination image of the side of the chip by using a grazing light source, where the included angle between the grazing light source and the normal of the side of the chip is within a preset angle range; obtaining a forward illumination image of the side of the chip by using a forward illumination light source; extracting the surface microtopography features of the forward illumination image; extracting the scattered light intensity distribution of the grazing illumination image, and performing adaptive threshold segmentation on the grazing illumination image according to the scattered light intensity distribution to obtain a defect region; performing feature fusion on the defect region and the surface microtopography features by using a preset weight coefficient to obtain a fused image; and determining the fused image as the original chip image.

[0009] By adopting the above technical solution, combining the advantages of the grazing light source and the forward illumination light source, grazing illumination can highlight the morphological features of surface defects, while forward illumination can obtain complete surface information. By extracting the surface microtopography features of the forward illumination image, the detailed information of the surface structure is retained. At the same time, using the scattered light intensity distribution of the grazing illumination image for adaptive threshold segmentation accurately locates the defect region. Performing feature fusion by using a preset weight coefficient not only maintains the significance of the defect region but also maintains the integrity of the background region, thereby improving the reliability and accuracy of defect detection and effectively solving the problem of the double reflection characteristics of the wide silicon wafer cutting surface.

[0010] In some embodiments in combination with some embodiments of the first aspect, the step of obtaining a grazing illumination image of the side of the chip by using a grazing light source specifically includes: changing the relative position relationship between the side of the chip and the imaging system to obtain an image sequence at multiple observation angles; performing spatial registration on the image sequence according to a unified image coordinate system; extracting the high-brightness regions in each image in the spatially registered image sequence, where the high-brightness regions are regions with brightness higher than a threshold; establishing a brightness distribution matrix according to the high-brightness regions; and removing the pixel points with brightness change higher than the brightness change threshold according to the brightness change characteristics of each pixel point in the brightness distribution matrix to obtain the grazing illumination image.

[0011] By adopting the above technical solution, by changing the relative position relationship between the side of the chip and the imaging system, complete information at multiple observation angles is obtained. Performing spatial registration by using a unified image coordinate system ensures the position correspondence of images at different angles. By extracting the high-brightness regions in each image and establishing a brightness distribution matrix, the optical characteristics of the defect region can be comprehensively analyzed. Removing the abnormally changing points according to the brightness change characteristics of pixel points effectively eliminates the pseudo-defect regions caused by scattered light. Thereby improving the accuracy of defect region extraction and providing a more accurate target region for subsequent defect detection, thus solving the problem of region expansion caused by edge scattering of defects.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after extracting the scattered light intensity distribution of the glancing illumination image, and performing adaptive threshold segmentation on the glancing illumination image according to the scattered light intensity distribution to obtain the step of defective area; determining whether the complexity of the defective area is greater than the complexity threshold; if greater than the complexity threshold, performing a step of feature fusion of the defective area with the surface microscopic morphology features using a preset weight coefficient to obtain a fused image; if not greater than the complexity threshold, transforming the position information of the defective area according to the coordinate transformation relationship between the glancing illumination image and the forward illumination image; focusing according to the changed position information, reusing the forward illumination light source to obtain a new forward illumination image of the chip side; and determining the new forward illumination image as the original chip image.

[0013] By adopting the above technical solution, for defects with higher complexity, the defect features are obtained by feature fusion method; for defects with lower complexity, clearer forward illumination images are obtained by refocusing. The problem of insufficient contrast in defect areas under grazing illumination is solved.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the step of training a chip side scratch detection model using supervised learning specifically includes: constructing a dual-branch attention network, including a spatial attention branch and a channel attention branch; the spatial attention branch is used to adaptively learn the spatial position characteristics of the scratch area to generate a spatial weight map; the channel attention branch is used to adaptively learn the channel response characteristics of the scratch area to generate a channel weight map; the spatial weight map is fused with the channel weight map to obtain a comprehensive attention map; according to the comprehensive attention map, the scratch areas corresponding to the smear marks extracted from the original chip image are clustered; based on the clustering results, the scratch areas corresponding to the smear marks extracted from the original chip image are segmented to obtain refined scratch areas; and the chip side scratch detection model is trained using the refined scratch areas and scratch-free chip images.

[0015] By adopting the above technical solution, a spatial attention weight map is generated through the spatial attention branch to guide the model to pay attention to the actual scratch area location. At the same time, through the channel attention branch, a channel attention weight vector is obtained to highlight the key feature channel of the scratch. The spatial attention weight map and the channel attention weight vector are multiplied to obtain a comprehensive attention map, and then the original smear mark area is clustered according to the guidance of the comprehensive attention map, and the original smear mark area is further segmented according to the clustering results. It can overcome the problem of over-labeling due to conservative considerations by the staff and accurately define the real scratch boundary. Even if the staff adopts the strategy of "prefer more labels to less labels" and causes the labeled area to be too large, the actual scratch area can be accurately located through the guidance of the attention weight, combined with the spatial distribution characteristics and channel response characteristics of the scratches.

[0016] In some embodiments in combination with some embodiments of the first aspect, the step of clustering the scratch regions corresponding to the smear marks extracted from the original chip image according to the comprehensive attention map specifically includes: based on the weight distribution of the comprehensive attention map, setting clustering centers in the scratch regions corresponding to the smear marks extracted from the original chip image, the setting probability of the clustering centers being positively correlated with the attention values, and the total number of clustering centers being a preset number; for the pixel points in the neighborhood of each clustering center, calculating the weighted gradient value by combining the pixel gradient value and the attention weight at the corresponding position; moving the clustering center to the image position corresponding to the minimum weighted gradient in the corresponding neighborhood; searching within a preset range centered on the clustering center and calculating the distance between the searched pixel points and the clustering center; based on the weight of the comprehensive attention map, performing weighted calculation on the distance; moving the clustering center to the position of the pixel point with the minimum weighted distance; returning to the step of searching within a preset range centered on the clustering center until the clustering center no longer changes, and obtaining multiple sub-regions.

[0017] By adopting the above technical solution, by associating the setting of the clustering center with the attention value, the rationality of the clustering starting point is ensured. During the clustering process, the weighted gradient value is calculated by combining the pixel gradient value and the attention weight, realizing more accurate boundary positioning. By iteratively optimizing the position of the clustering center and performing weighted calculation on the distance based on the attention weight, the accuracy of the clustering result is improved. This attention-guided clustering method effectively overcomes the problem of being easily affected by noise and provides a reliable basis for subsequent region segmentation.

[0018] In some embodiments in combination with some embodiments of the first aspect, the step of segmenting the scratch regions corresponding to the smear marks extracted from the original chip image based on the clustering result to obtain the refined scratch regions specifically includes: calculating the average attention value of the pixel points in each sub-region; marking the sub-regions with an average attention value higher than the average threshold as candidate scratch regions; merging the sub-regions according to the connectivity of adjacent candidate scratch regions; and outputting the merged sub-regions as the refined scratch regions.

[0019] By adopting the above technical solution, by calculating the average attention value of the sub-regions, the mislabeled regions are removed through threshold screening. Then, the connectivity of adjacent candidate regions is analyzed, and the related sub-regions are reasonably merged to avoid over-segmentation. This optimization method based on attention value and connectivity not only ensures the integrity of the scratch region but also avoids the problem of over-segmentation. Through multi-level optimization processing, the contour of the finally obtained scratch region is more accurate, improving the reliability and accuracy of the detection result. This optimization strategy not only improves the accuracy of scratch detection but also provides a more reliable detection result for actual production applications.

[0020] Second aspect, the present application provides a chip side scratch detection system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the chip side scratch detection system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] Third aspect, the present application provides a computer program product containing instructions, which when running on a chip side scratch detection system, enables the chip side scratch detection system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] Fourth aspect, the present application provides a computer-readable storage medium, including instructions, which when running on a chip side scratch detection system, enables the chip side scratch detection system to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The region of interest guiding model is used to extract the scratch region corresponding to the smear mark from the original chip image, reducing the difficulty of manual annotation. Relevant personnel only need to simply smear and mark the scratch region. This simplified annotation method improves the annotation efficiency, enabling staff to complete the annotation of more samples within the same time, thereby obtaining sufficient training data, providing sufficient data support for model training, and increasing the accuracy of the model.

[0024] 2. Combining the advantages of the grazing light source and the front illumination light source, grazing illumination can highlight the morphological features of surface defects, while front illumination can obtain complete surface information. By extracting the surface microscopic morphological features of the front illumination image, the detailed information of the surface structure is retained. At the same time, the adaptive threshold segmentation is performed using the scattered light intensity distribution of the grazing illumination image to accurately locate the defect region. The preset weight coefficient is used for feature fusion, which not only maintains the saliency of the defect region but also maintains the integrity of the background region, thereby improving the reliability and accuracy of defect detection and effectively solving the problem of the double reflection characteristics of the wide silicon wafer cutting surface.

[0025] 3. Generate a spatial attention weight map through the spatial attention branch to guide the model to pay attention to the actual scratch area location. At the same time, through the channel attention branch, obtain the channel attention weight vector, which is used to highlight the key feature channel of the scratch. Multiply the spatial attention weight map and the channel attention weight vector to obtain a comprehensive attention map, and then cluster the original smear mark area according to the guidance of the comprehensive attention map, and further segment the original smear mark area according to the clustering results. It can overcome the problem of over-labeling due to conservative considerations of the staff and accurately define the real scratch boundary. Even if the staff adopts the strategy of "prefer more labels to less labels" and causes the labeled area to be too large, the actual scratch area can be accurately located through the guidance of the attention weight, combined with the spatial distribution characteristics and channel response characteristics of the scratches. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a chip side scratch detection method in an embodiment of the present application; Figure 2 This is another schematic diagram of the process of detecting scratches on the side of a chip in an embodiment of the present application; Figure 3 yes Figure 2 Specific flow diagram of step S201; Figure 4 yes Figure 2 Another flowchart after step S204; Figure 5 This is another schematic diagram of the process of detecting scratches on the side of a chip in an embodiment of the present application; Figure 6 yes Figure 5 Specific flow diagram of step S505; Figure 7 yes Figure 5 Specific flow diagram of step S506; Figure 8 It is a schematic diagram of an exemplary hardware structure of a chip side scratch detection system in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0028] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and should not be construed as implying or suggesting relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0029] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for detecting scratches on the side of a chip in an embodiment of the present application; S101. Obtain a set of chip images to be trained, where the set of chip images includes original chip images and their corresponding annotation images, and the annotation images are obtained by smearing and marking the scratch areas in the original chip images; Among them, the set of chip images represents the set of all image samples used for training the model, including paired original images and annotation images; the original chip image refers to the original captured image of the side of the chip without any annotation processing; the annotation image is used to represent the image obtained after the staff smears and marks the scratch area on the basis of the original chip image; the smearing and marking refers to the process in which the staff uses a paintbrush tool to cover and mark the scratch area. For example, for an image of the side of a chip with obvious scratches, the staff can use a red paintbrush tool to smear the scratch area to generate the corresponding annotation image.

[0030] S102. Perform region segmentation processing on the annotation images to extract the regions of interest containing the smearing marks; Among them, the region of interest (ROI) is used to represent the image region containing the target object (here, the smearing mark). For example, for an annotation image containing a red smearing mark, the local regions containing only the red marks can be extracted through region segmentation, and these regions are the regions of interest.

[0031] In some embodiments, the region of interest may be the smallest regular image (such as a rectangle or a circle) including the smearing mark, aiming to reduce the image and increase attention.

[0032] In some specific embodiments, the annotation image is converted to the HSV color space; a color threshold range is set to extract the smearing mark region; connected region analysis is used to obtain independent regions; the minimum bounding rectangle of each region is calculated, and a certain boundary is extended as the final region of interest, which is not limited herein.

[0033] S103. Train the chip side scratch detection model using supervised learning. The training data includes: regions of interest, original chip images, and chip images without scratches. Among them, the regions of interest are used to enable the chip side scratch detection model to extract the scratch regions corresponding to the smear marks from the corresponding original chip images. Among them, supervised learning refers to a machine learning method that uses labeled data to train the model; the chip side scratch detection model refers to a deep learning model used to automatically detect and locate scratches on the chip side; the training data is used to represent the data set input into the model for learning; the chip image without scratches represents a chip image sample with qualified quality and intact surface. For example, during the training process, the model learns the feature representation of scratches by comparing chip images with and without scratches, and extracts the actual scratch regions under the guidance of the regions of interest.

[0034] In some embodiments, the training process includes the following steps: Use the regions of interest as guiding information to guide the model to locate the actual scratch positions in the original chip images. At the same time, introduce chip images without scratches as negative samples to help the model learn the features for distinguishing normal regions and scratch regions. Finally, optimize the model parameters through the backpropagation algorithm to enable the model to accurately detect scratches.

[0035] S104. Deploy the trained chip side scratch detection model to the production line to detect scratches on the online chips.

[0036] It can be seen that using the regions of interest to guide the model to extract the scratch regions corresponding to the smear marks from the original chip images reduces the difficulty of manual annotation. Relevant personnel only need to simply smear and mark the scratch regions. This simplified annotation method improves the annotation efficiency, enabling staff to complete the annotation of more samples within the same time, thereby obtaining sufficient training data, providing sufficient data support for model training, and increasing the accuracy of the model.

[0037] The above embodiments reduce the difficulty of the annotation work and increase the accuracy of the model. However, in actual use, due to the special surface microstructure of the wide silicon wafer cutting surface, it has both specular reflection and diffuse reflection characteristics. When light irradiates the cutting surface, part of the light undergoes specular reflection, and the other part of the light is scattered on the surface microstructure to form diffuse reflection. This dual reflection characteristic results in a poor presentation effect of the features in the defect regions, thereby affecting subsequent training.

[0038] Please refer to Figure 2 , Figure 2 which is another process schematic diagram of the chip side scratch detection method in the embodiments of the present application; Therefore, in some embodiments: Before step S101, it further includes: S201. Obtain a grazing illumination image of the side of the chip using a grazing light source, where the angle between the grazing light source and the normal of the chip side is within a preset angle range; Among them, the grazing light source refers to a light source that irradiates the surface of an object at a small incident angle; the grazing illumination image refers to an image obtained when using a grazing light source for illumination; the normal refers to a straight line perpendicular to the side of the chip; the preset angle range is used to represent the incident angle range that is empirically proven to be most suitable for observing surface defects. For example, when the grazing light source irradiates at a 15-degree angle to the side of the chip, it can better highlight the fine scratches on the surface.

[0039] It should be noted that in the actual use process, based on the principle that during the chip cutting and transportation processes, due to the limitations of the movement direction of the cutting tool and the handling trajectory of the chip, the scratches generated are mainly distributed vertically; when using a grazing light source to irradiate from the side, the light is almost parallel to the scratch direction. This parallel relationship causes obvious light and shadow changes due to the unevenness of the scratches, and the vertical scratches will block or scatter the grazing light, forming a strong light and dark contrast in the image. While the normal cutting surface presents a uniform low-brightness state under the grazing light, forming a dark background; the microscopic unevenness at the scratch will scatter the grazing light, forming obvious bright areas on the dark background. This strong contrast effect of "dark background + bright target" improves the visibility of the scratches.

[0040] In some embodiments, according to the size and installation position of the chip, determine the installation position and irradiation angle of the grazing light source; then, use a precision angle adjustment mechanism to fix the grazing light source within a preset angle range (usually 10 - 30 degrees); then, adjust the illumination intensity and uniformity of the light source to ensure that the light covers the entire detection area; finally, collect the chip image under grazing illumination through an industrial camera.

[0041] S202. Obtain a front illumination image of the side of the chip using a front illumination light source; Among them, the front illumination light source refers to a light source that irradiates perpendicular to the surface of the chip; the front illumination image refers to an image obtained under the irradiation of the front light source; front illumination is used to represent a lighting method in which light is incident on the surface to be measured at an angle close to perpendicular. For example, when the light source irradiates the chip surface perpendicularly, a clear image of the overall surface topography features can be obtained.

[0042] In some embodiments, install an annular LED light source directly above the industrial camera to ensure that the center of the light source coincides with the optical axis of the camera; then, adjust the working distance from the light source to the chip surface so that the illumination area completely covers the detection range; then, set appropriate light source brightness and exposure parameters to ensure that the image is not overexposed or underexposed; finally, collect the chip image under front illumination.

[0043] S203. Extract the surface microscopic topography features of the front illumination image; Among them, the surface micro-topography features represent the micro-structural information of the chip surface; feature extraction refers to the process of obtaining effective information from an image; the micro-topography is used to represent the fine undulations and texture features of an object's surface.

[0044] After obtaining the forward illumination image, it is necessary to extract the surface feature information therein. In some embodiments, the feature extraction process includes the following steps: First, preprocess the forward illumination image, including denoising and contrast enhancement; then, use a multi-scale analysis method to extract the texture features of the image; next, calculate the statistical features of the local region, such as gradients, directions, etc.; finally, normalize the extracted features to generate a feature map.

[0045] S204. Extract the scattered light intensity distribution of the grazing illumination image, and perform adaptive threshold segmentation on the grazing illumination image according to the scattered light intensity distribution to obtain the defect region; Among them, the scattered light intensity distribution represents the energy distribution of light after scattering on the surface; adaptive threshold segmentation refers to a method of automatically determining the segmentation threshold according to the local features of the image; the defect region is used to represent the image region containing surface defects such as scratches. For example, when light irradiates a scratch, a different scattering pattern from the normal region will be generated, and defects can be identified by analyzing this difference.

[0046] After obtaining the grazing illumination image, it is necessary to perform defect detection. In some embodiments, the defect detection process includes the following steps: First, calculate the local light intensity distribution characteristics of the image to construct a scattered light intensity map; then, adaptively calculate the segmentation threshold based on the statistical characteristics of the local region; next, use the calculated threshold to segment the image; finally, optimize the segmentation result through morphological processing to obtain the final defect region.

[0047] S205. Feature fusion is performed on the defect region and the surface micro-topography features using a preset weight coefficient to obtain a fused image; Among them, the preset weight coefficient represents the importance degree of different features in the fusion process; feature fusion refers to the process of integrating multiple feature information; the fused image is used to represent a comprehensive image combining multiple feature information.

[0048] After obtaining various features, feature fusion is required. In some embodiments, different features are normalized to make their numerical ranges consistent; appropriate weight coefficients are determined according to the discrimination ability of each feature; the features are combined using weighted summation or other fusion strategies; and the fusion result is optimized to generate the final fused image.

[0049] In some other embodiments, the defect region can be fused with the forward illumination image with the corresponding region removed to achieve the same function.

[0050] S206. Determine the fused image as the original chip image.

[0051] The image after feature fusion contains richer defect information and is suitable as the input data for model training.

[0052] It can be seen that by combining the advantages of the grazing light source and the front illumination light source, grazing illumination can highlight the morphological features of surface defects, while front illumination can obtain complete surface information. By extracting the surface micro-morphological features of the front illumination image, the detailed information of the surface structure is retained. At the same time, adaptive threshold segmentation is performed using the scattered light intensity distribution of the grazing illumination image to accurately locate the defect area. Feature fusion is carried out using a preset weight coefficient, which not only maintains the significance of the defect area but also maintains the integrity of the background area, thereby improving the reliability and accuracy of defect detection and effectively solving the problem of the double reflection characteristics of the wide silicon wafer cutting surface.

[0053] In the above embodiment, the input solves the problem caused by the reflection characteristics. However, when using a grazing light source to obtain the grazing illumination image of the chip side, the incident light is approximately parallel to the chip surface. When the light hits the defect edge, due to the roughness and irregularity of the defect surface, the incident light will scatter at the defect edge. These scattered lights will irradiate the non-defect area behind the defect, resulting in an abnormal brightness distribution in these areas during imaging. This optical effect causes the range of the defect area in the image to be wrongly enlarged, including not only the real defect area but also the normal area illuminated by the scattered light.

[0054] Please refer to Figure 3 , Figure 3 which Figure 2 is the specific process schematic diagram of step S201 in Therefore, in some embodiments, step S201 specifically includes: S301. Change the relative position relationship between the chip side and the imaging system to obtain an image sequence at multiple observation angles; Among them, the relative position relationship represents the spatial position and angular relationship between the chip and the camera; the observation angle refers to the perspective of the camera imaging the chip; the image sequence is used to represent a set of consecutive images obtained at different angles.

[0055] It should be clear that the entire chip is rotated, and the physical structure and surface features of the chip itself do not change during this process. No matter from which observation angle the image is taken, the obtained image is the imaging of the same chip. Only due to the different shooting angles, the positions of the pixels in the image are different, but the physical position on the chip corresponding to each pixel is fixed and unique, and there is no pixel loss, which is an important basis for the subsequent steps to be accurately implemented.

[0056] In some embodiments, the chip is fixed on a rotating table; the rotation step angle and range are set; the rotating table is controlled to move by the step angle; acquisition is triggered at each angular position; and the corresponding angular information is recorded, which is not limited herein.

[0057] S302. Perform spatial registration on the image sequence according to a unified image coordinate system; After obtaining the image sequence, image alignment processing is required. In some embodiments, an image in the image sequence is selected as a reference image; feature points and descriptors in each image are extracted; the corresponding relationship of feature points between images is established; the geometric transformation matrix between images is calculated; and coordinate transformation is performed on all images to achieve spatial alignment.

[0058] It should be noted that, continuing with the above example, the image can be rotated accordingly according to the rotated angle.

[0059] S303. Extract the high-brightness regions in each image in the spatially registered image sequence, where the high-brightness region is a region with a brightness higher than the threshold; Among them, the high-brightness region represents a set of pixels with relatively high brightness values in the image; the threshold is the brightness critical value used to distinguish the high-brightness region; the region with a brightness higher than the threshold is used to represent the image region where the pixel brightness value exceeds the set threshold. For example, under grazing illumination, the defect region often forms a brightness value higher than the background.

[0060] In some embodiments, perform brightness normalization processing on the registered image; determine a suitable threshold based on the brightness distribution characteristics of the image; segment the image using the threshold; and perform morphological processing on the segmentation result to obtain a complete high-brightness region.

[0061] S304. Establish a brightness distribution matrix according to the high-brightness region; Among them, the brightness distribution matrix represents a two-dimensional array that records the brightness values of each pixel point in the image, and the brightness distribution matrix covers all pixels. For those pixels belonging to the high-brightness, their actual brightness values in different images will be recorded accordingly, while for those pixels that are not high-brightness, they are not ignored or there is no corresponding information, but they are marked as an empty set (or assigned a value of 0 or other suitable representation methods, which can be determined specifically according to the actual data structure and algorithm requirements); matrix establishment refers to constructing a data structure containing complete brightness information; the distribution characteristics are used to represent the variation law of brightness values in space.

[0062] In some embodiments, create an empty matrix with the same size as the image; map the high-brightness region information in each image to the corresponding position in the matrix; record the brightness values of each pixel point in different images; and finally, perform normalization processing on the matrix to make the data comparable.

[0063] S305. According to the brightness change characteristics of each pixel point in the brightness distribution matrix, the pixel points with brightness change higher than the brightness change threshold are removed to obtain the grazing illumination image.

[0064] After completing the analysis of the brightness distribution matrix, it is necessary to screen the real defect areas. In some embodiments, the brightness change of each pixel point is calculated; the amplitude and law of the brightness change are analyzed; screening is performed according to a preset change threshold; the areas with stable brightness change are retained to generate the final grazing illumination image.

[0065] In some embodiments, in steps S303 to S305, the high-brightness areas in the grazing illumination image have been extracted and screened to obtain the areas that may contain defects. These high-brightness areas will be used as an important reference for the subsequent step S204 to more accurately extract the defect areas; In other embodiments, generally speaking, the operations from S303 to S305 are, overall, to complete the goal expected to be achieved in step S204 - to extract the scattered light intensity distribution of the grazing illumination image, and perform adaptive threshold segmentation on the grazing illumination image according to the scattered light intensity distribution to obtain the defect areas. Therefore, steps S303 - S305 are equivalent to step S204, and when the present embodiment executes to the stage of step S204, the results generated by S303 - S305 can be directly applied.

[0066] It can be seen that by changing the relative position relationship between the side of the chip and the imaging system, the complete information of multiple observation angles is obtained. Using a unified image coordinate system for spatial registration ensures the position correspondence relationship of images at different angles. By extracting the high-brightness areas in each image and establishing a brightness distribution matrix, the optical characteristics of the defect areas can be comprehensively analyzed. Abnormal change points are removed according to the brightness change characteristics of pixel points, effectively eliminating the pseudo-defect areas caused by scattered light. Furthermore, the accuracy of defect area extraction is improved, providing a more accurate target area for subsequent defect detection, thus solving the problem of enlarged area caused by edge scattering of defects.

[0067] In the above embodiments, the problems caused by the reflection characteristics are solved. However, when using a grazing light source to obtain the grazing illumination image of the side of the chip, the angle between the incident light and the surface of the silicon wafer is small, resulting in most of the light being reflected and only a small amount of light being able to effectively interact with the defect areas. Although this illumination method is beneficial to highlighting the surface micro-topography, it also significantly reduces the contrast between the defect areas and the background. Subsequently, when using a preset weight coefficient for feature fusion, since the defect features in the original image are not obvious in themselves, the defect areas are still difficult to be effectively highlighted in the fused image.

[0068] Please refer to Figure 4 , Figure 4 which isFigure 2 Another process schematic diagram after step S204; Therefore, in some embodiments, after step S204, it further includes: S401. Determine whether the complexity of the defect area is greater than the complexity threshold; Wherein, the complexity is the perimeter - area ratio of the defect area, or the boundary curvature change, or the shape irregularity; the complexity threshold is a standard value used to determine whether a defect is complex, used to determine whether there are sufficient features for subsequent model training.

[0069] S402. If it is greater than the complexity threshold, then execute step S205; S403. If it is not greater than the complexity threshold, then transform the position information of the defect area according to the coordinate transformation relationship between the grazing - incidence illumination image and the forward - illumination image; In some embodiments, establish a transformation model between the grazing - incidence illumination and the forward - illumination images; extract the position coordinates of the defect area; apply the transformation matrix for coordinate mapping; determine the transformed position range; output the transformed position information.

[0070] It should be noted that the defect area in this step refers to the defect area below the complexity threshold, and the position information of the defect area in the grazing - incidence illumination image is obtained through the coordinate transformation relationship to get the position information in the forward - illumination image.

[0071] S404. Focus according to the changed position information, and re - use the forward - illumination light source to obtain a new forward - illumination image of the side of the chip; It should be noted that the new forward - illumination image should be set to have the same parameters as the previously captured forward - illumination image (except for the different focusing targets) to ensure the clarity and comparability of the images.

[0072] After completing the position transformation, it is necessary to re - collect the image. In some embodiments, locate the target area according to the transformed position information; adjust the illumination system to ensure uniform illumination; perform autofocus optimization; then, set appropriate exposure parameters; collect a high - quality forward - illumination image.

[0073] S405. Determine the new forward - illumination image as the original chip image.

[0074] It can be seen that for defects with higher complexity, the defect features are obtained by using the feature fusion method; for defects with lower complexity, a clearer forward - illumination image is obtained by refocusing. The problem of insufficient contrast in the defect area under grazing - incidence illumination is solved.

[0075] The above embodiments reduce the difficulty of labeling and increase the accuracy of the model. However, in actual use, due to the influence of fine hand movement control and delayed visual feedback, it is difficult to accurately control the marking handwriting on the actual boundary of the scratch; furthermore, in order to avoid missing defective areas, staff tend to mark suspicious areas together and adopt a conservative strategy of "it is better to mark more than less". The superposition of these human factors causes the final marked area to be significantly larger than the actual scratch range, affecting the accuracy of subsequent model training based on these marks.

[0076] See also Figure 5 , Figure 5 This is another schematic flow chart of the chip side scratch detection method in the embodiment of the present application; Therefore, in some embodiments, step S103 specifically includes: S501, construct a dual-branch attention network, including a spatial attention branch and a channel attention branch; The dual-branch attention network represents a neural network structure with two parallel processing branches; the spatial attention branch refers to the network path that processes spatial position information; and the channel attention branch is used to represent the network path that processes feature channel information. The dual-branch structure can simultaneously focus on the location and feature expression of scratches.

[0077] Before starting model training, a specific network structure needs to be built. In some embodiments, a backbone network architecture is designed; spatial and channel attention modules are constructed; and then, the connection method between branches is designed, which will not be repeated here.

[0078] S502, the spatial attention branch is used to adaptively learn the spatial position features of the scratch area and generate a spatial weight map; After the network is constructed, it is necessary to implement spatial feature learning. In some embodiments, multi-scale spatial features are extracted; spatial attention weights are calculated; spatial attention maps are generated; attention mechanisms are applied to enhance features; and spatial weight distributions are output.

[0079] S503, the channel attention branch is used to adaptively learn the channel response characteristics of the scratch area and generate a channel weight map; Among them, the channel response feature represents the activation degree of the scratch on different feature channels; the channel weight map refers to the one-dimensional vector representing the importance of different channels; and adaptive learning is used to indicate that the network automatically adjusts the importance weights of each channel.

[0080] While learning spatial features, channel features need to be learned. Extract global features of each channel; calculate dependencies between channels; generate channel attention maps; apply channel weighted enhancement; and output channel weight distribution.

[0081] In some specific embodiments, a channel attention module is constructed; the relationships between channels are learned; attention scores are calculated; channel representations are updated; and weight vectors are output.

[0082] S504. Fuse the spatial weight map and the channel weight map to obtain a comprehensive attention map. In some embodiments, align the spatial and channel weight dimensions; perform weight fusion calculation; normalize the fusion result; and generate a comprehensive attention map.

[0083] S505. Cluster the scratch regions corresponding to the smear marks extracted from the original chip image according to the comprehensive attention map. In some specific embodiments, density-based clustering is adopted: calculate local density; determine cluster centers; assign class labels; merge similar clusters; and optimize cluster boundaries.

[0084] Please refer to Figure 6 , Figure 6 which Figure 5 is the schematic flow diagram of step S505 in In some specific embodiments, step S505 specifically includes: S601. Based on the weight distribution of the comprehensive attention map, set cluster centers in the scratch regions corresponding to the smear marks extracted from the original chip image. The setting probability of the cluster centers is positively correlated with the attention value, and the total number of cluster centers is a preset number. Among them, the smear mark represents the suspicious scratch range manually marked; the scratch region refers to the target region containing actual defects; the cluster center is used to represent the core point of region segmentation; the setting probability represents the possibility of initializing the cluster center; and the attention value refers to the degree of attention at a specific position.

[0085] After obtaining the fused attention information, the starting point of the clustering process needs to be initialized. In some embodiments, calculate the position importance score according to the comprehensive attention map; map the smear mark region to the original image space; calculate the sampling probability distribution based on the attention value; randomly sample a preset number of cluster centers according to the probability distribution; and record the positions of the selected cluster centers.

[0086] S602. For the pixel points in the neighborhood of each cluster center, calculate the weighted gradient value by combining the pixel gradient value and the attention weight at the corresponding position. Among them, the neighborhood represents the local area around the cluster center; the pixel point refers to the smallest data unit in the image; the pixel gradient value represents the degree of image intensity change; the attention weight is the importance coefficient of the position; and the weighted gradient value is used to represent the gradient feature that comprehensively considers spatial and attention information.

[0087] In some embodiments, determine the neighborhood range of each cluster center; calculate the gradient values of all pixels within the neighborhood; obtain the weights at the corresponding positions from the attention map; multiply the gradient values by the attention weights; and obtain the weighted gradient features.

[0088] S603. Move the cluster center to the image position corresponding to the minimum weighted gradient within the corresponding neighborhood; Among them, the neighborhood refers to the local range centered on the center point; the weighted gradient value refers to the gradient feature after considering the attention weights; and the minimum value represents the local minimum point of the feature quantity.

[0089] In some embodiments, traverse the weighted gradient values of all pixel points within the neighborhood; compare and record the positions of the minimum gradient values; calculate the offset between the current center and the target position; then, update the coordinates of the cluster center; and record the moving distance of the center point.

[0090] S604. Search within a preset range centered on the cluster center, calculate the distances between the searched pixel points and the cluster center; and perform weighted calculation on the distances based on the weights of the comprehensive attention map; Among them, the preset range represents the spatial range limit of the search; the pixel point refers to the basic unit in the image; the distance represents the differential measure of the spatial position; the comprehensive attention map is used to represent the importance distribution of positions; and the weighted calculation refers to the numerical operation considering the weight factor.

[0091] In some embodiments, determine the search range of each center point; calculate the distances from the pixels within the range to the center; obtain the corresponding weights from the attention map; combine the distances with the weights; and obtain the weighted distance measure.

[0092] S605. Move the cluster center to the position of the pixel point with the minimum weighted distance; S606. Return to step S604, and until the cluster center no longer changes, obtain multiple sub-regions.

[0093] S506. Based on the clustering result, segment the scratch region corresponding to the smeared mark extracted from the original chip image to obtain a refined scratch region; S507. Use the refined scratch region and the scratch-free chip image to train the chip side scratch detection model.

[0094] It can be seen that the spatial attention weight map is generated through the spatial attention branch to guide the model to pay attention to the real scratch area position. At the same time, the channel attention weight vector is obtained through the channel attention branch to highlight the key feature channel of the scratch. The spatial attention weight map and the channel attention weight vector are multiplied to obtain a comprehensive attention map, and then the original smear mark area is clustered according to the guidance of the comprehensive attention map, and the original smear mark area is further segmented according to the clustering result. It can overcome the problem of over-labeling due to conservative considerations of the staff and accurately define the real scratch boundary. Even if the staff adopts the strategy of "prefer more labels to less labels" and causes the labeled area to be too large, the actual scratch area can be accurately located through the guidance of the attention weight, combined with the spatial distribution characteristics and channel response characteristics of the scratch.

[0095] See also Figure 7 , Figure 7 yes Figure 5 Specific flow diagram of step S506; Step S506 specifically includes: S701, calculating the average attention value of the pixels in each sub-region; Before merging regions, it is necessary to quantitatively evaluate the attention distribution of each sub-region. In some embodiments, the pixel range of each sub-region is determined; the attention values ​​of all pixels in the region are collected; the attention values ​​in the region are accumulated; the number of pixels in the region is calculated; and the average attention value of the region is obtained.

[0096] S702, marking the sub-regions whose average attention values ​​are higher than the average threshold as candidate scratch regions; S703, merging sub-regions according to the connectivity of adjacent candidate scratch regions; S704: Output the merged sub-region as the refined scratch region.

[0097] It can be seen that by calculating the average attention value of the sub-region, the mislabeled region is removed through threshold screening. Then the connectivity of adjacent candidate regions is analyzed, and the associated sub-regions are reasonably merged to avoid over-segmentation. This optimization method based on attention value and connectivity not only ensures the integrity of the scratch area, but also avoids the problem of over-segmentation. Through multi-level optimization processing, the final scratch area contour is more accurate, which improves the reliability and accuracy of the detection results. This optimization strategy not only improves the accuracy of scratch detection, but also provides more reliable detection results for actual production applications.

[0098] The following introduces an exemplary chip side scratch detection system 800 provided in an embodiment of the present application. Figure 8 It is a schematic diagram of an exemplary hardware structure of a chip side scratch detection system 800 provided in an embodiment of the present application.

[0099] In some embodiments, the chip side scratch detection system 800 is a computer device or includes a computer device. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with other external terminals or servers through a network connection. In some embodiments, the network interface can be a wired network interface, and in some embodiments, the network interface can also be a wireless network interface. The computer program, when executed by the processor, implements the method in the embodiments of the present application.

[0100] Those skilled in the art can understand that Figure 8 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0101] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0102] As used in the above embodiments, depending on the context, the term "when..." can be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if detecting (the stated condition or event)" can be interpreted to mean "if determining...", or "in response to determining...", or "when detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)".

[0103] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive), etc.

[0104] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware with a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above method embodiments. The foregoing storage medium includes: various media that can store program codes such as ROM or random access memory RAM, magnetic disks, or optical discs.

Claims

1. A method for detecting side scratches of a chip, characterized in that, Including: Obtain a chip image set to be trained, where the chip image set includes original chip images and their corresponding annotation images, and the annotation images are obtained by smearing and marking the scratch areas in the original chip images; Perform region segmentation processing on the annotation images to extract regions of interest containing the smear marks; Use supervised learning to train the chip side scratch detection model, and the training data includes: the regions of interest, the original chip images, and chip images without scratches; where the regions of interest are used to enable the chip side scratch detection model to extract the scratch areas corresponding to the smear marks from the corresponding original chip images; Deploy the trained chip side scratch detection model to the production line to detect scratches on online chips.

2. The method according to claim 1, wherein Before the step of obtaining the chip image set to be trained, the method further includes: Obtain a grazing illumination image of the chip side using a grazing light source, where the angle between the grazing light source and the normal of the chip side is within a preset angle range; Obtain a forward illumination image of the chip side using a forward illumination light source; Extract the surface micro-topography features of the forward illumination image; Extract the scattered light intensity distribution of the grazing illumination image, and perform adaptive threshold segmentation on the grazing illumination image according to the scattered light intensity distribution to obtain a defect region; Perform feature fusion on the defect region and the surface micro-topography features using a preset weight coefficient to obtain a fused image; Determine the fused image as the original chip image.

3. The method according to claim 2, wherein The step of obtaining a grazing illumination image of the chip side using a grazing light source specifically includes: Change the relative position relationship between the chip side and the imaging system to obtain an image sequence at multiple observation angles; Perform spatial registration on the image sequence according to a unified image coordinate system; Extract the high-brightness regions in each image in the spatially registered image sequence, where the high-brightness regions are regions with brightness higher than a threshold; Establish a brightness distribution matrix based on the high-brightness regions; According to the brightness change characteristics of each pixel point in the brightness distribution matrix, eliminate the pixel points with brightness change higher than the brightness change threshold to obtain the grazing illumination image.

4. The method according to claim 2, wherein After the step of extracting the scattered light intensity distribution of the grazing illumination image and performing adaptive threshold segmentation on the grazing illumination image according to the scattered light intensity distribution to obtain a defect region; Judge whether the complexity of the defect region is greater than a complexity threshold; If it is greater than the complexity threshold, perform the step of performing feature fusion on the defect region and the surface micro-topography features using a preset weight coefficient to obtain a fused image; If it is not greater than the complexity threshold, transform the position information of the defect region according to the coordinate transformation relationship between the grazing illumination image and the forward illumination image; Perform focusing according to the changed position information, and re-obtain a new forward illumination image of the chip side using the forward illumination light source; Determine the new forward illumination image as the original chip image.

5. The method according to claim 1, wherein The step of using supervised learning to train the chip side scratch detection model specifically includes: Construct a dual-branch attention network, including a spatial attention branch and a channel attention branch; The spatial attention branch is used to adaptively learn the spatial position features of the scratch area and generate a spatial weight map; The channel attention branch is used to adaptively learn the channel response features of the scratch area and generate a channel weight map; Fuse the spatial weight map and the channel weight map to obtain a comprehensive attention map; Cluster the scratch areas corresponding to the smear marks extracted from the original chip image according to the comprehensive attention map; Segment the scratch areas corresponding to the smear marks extracted from the original chip image based on the clustering result to obtain refined scratch areas; Use the refined scratch areas and the scratch-free chip image to train the chip side scratch detection model.

6. The method according to claim 5, wherein The step of clustering the scratch areas corresponding to the smear marks extracted from the original chip image according to the comprehensive attention map specifically includes: Based on the weight distribution of the comprehensive attention map, set clustering centers in the scratch areas corresponding to the smear marks extracted from the original chip image. The setting probability of the clustering centers is positively correlated with the attention values, and the total number of the clustering centers is a preset number; For the pixel points in the neighborhood of each clustering center, calculate the weighted gradient value by combining the pixel gradient value and the attention weight at the corresponding position; Move the clustering center to the image position corresponding to the minimum weighted gradient in the corresponding neighborhood; Search within a preset range centered on the clustering center, and calculate the distance between the searched pixel points and the clustering center; based on the weight of the comprehensive attention map, perform weighted calculation on the distance; Move the clustering center to the position of the pixel point with the minimum weighted distance; Return to the step of searching within a preset range centered on the clustering center until the clustering center no longer changes, and obtain multiple sub-regions.

7. The method according to claim 6, wherein The step of segmenting the scratch areas corresponding to the smear marks extracted from the original chip image based on the clustering result to obtain refined scratch areas specifically includes: Calculate the average attention value of the pixel points in each sub-region; Mark the sub-regions with an average attention value higher than the average threshold as candidate scratch areas; Merge the sub-regions according to the connectivity of adjacent candidate scratch areas; Output the merged sub-regions as refined scratch areas.

8. A chip side scratch detection system, characterized in that, The chip side scratch detection system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the chip side scratch detection system to execute the method according to any one of claims 1-7.

9. A computer program product comprising instructions, characterized in that, When the computer program product runs on the chip side scratch detection system, it enables the chip side scratch detection system to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, comprising instructions, characterized in that, When the instruction runs on the chip side scratch detection system, the chip side scratch detection system is caused to execute the method described in any one of claims 1-7.

Images