Defect image labeling method and device, computer device, and storage medium

Abstract

The application relates to a defect image labeling method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a defect image of a detection target and defect information of the defect image, and a reference image corresponding to the defect image, wherein the defect information comprises defect position data and a defect category; performing difference detection on the defect image and the corresponding reference image to obtain a result image; performing unsupervised learning based on the result image to obtain a candidate defect region in the defect image; determining a detection defect region in the defect image according to the defect position data of the defect image, performing set processing on the detection defect region and the candidate defect region to obtain a defect labeling region; and generating defect labeling information of the defect labeling region in the defect image based on the defect category of the defect image. The application can improve the accuracy of image labeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a defect image labeling method and device, computer equipment and storage medium. BACKGROUND

[0002] With the rapid development of deep learning algorithms, especially the iterative optimization of neural networks in target classification, target detection and target segmentation, deep learning algorithm technology is increasingly used in automatic defect detection / classification in the semiconductor field, and higher and higher detection / classification accuracy is obtained.

[0003] Before being used for automatic defect detection / classification, a deep learning model needs to be trained. Image labeling is a process of labeling images of a data set and training a deep learning model, and its goal is to provide accurate position and shape information for defects in the image to facilitate subsequent defect detection and classification; therefore, image data used for training has a crucial impact on the training results. For example, for model training of wafer detection, defect labeling of wafer defect images is needed.

[0004] Currently, image labeling mainly has two ways: manual labeling and automatic labeling. Manual labeling mainly involves quality personnel using image labeling software to draw defect regions and mark defect categories. Automatic labeling is to establish a pre-training model through deep learning based on the visual content of the image, and to automatically label and process defects in the image by using the pre-training model through supervised learning. However, manual labeling simply relies on the human eye to find defect positions from defect images and then set defect categories, and is affected by the diversity of defect styles, human vision, attention concentration, and subjective differences, which can easily result in mislabeling and missing labeling, leading to labeling errors and low accuracy. Automatic labeling uses a pre-training model through supervised learning to label and process, and completely relies on model automatic recognition, which can also easily result in incorrect recognition, leading to labeling errors. SUMMARY

[0005] Therefore, it is necessary to provide a defect image labeling method, device, computer equipment and storage medium capable of improving defect labeling accuracy to solve the technical problem of low labeling accuracy.

[0006] A defect image labeling method comprises:

[0007] Obtaining a defect image of a detection target and defect information of the defect image, and obtaining a reference image corresponding to the defect image, wherein the defect information comprises defect position data and defect categories;

[0008] Performing difference detection on the defect image and the corresponding reference image to obtain a result image;

[0009] unsupervised learning based on the result image to obtain a candidate defect region in the defect image;

[0010] determining a detected defect region in the defect image according to defect position data of the defect image, and performing set processing on the detected defect region and the candidate defect region to obtain a defect labeling region;

[0011] generating defect labeling information of the defect labeling region in the defect image based on a defect category of the defect image.

[0012] In one of the embodiments, the defect position data includes defect coordinates; and the obtaining of the reference image corresponding to the defect image comprises:

[0013] obtaining a non-defect template image of the detection target;

[0014] determining an image size of the defect image;

[0015] locating defect coordinates of the defect image in the non-defect template image, and obtaining the reference image corresponding to the defect image by taking an image with the size equal to the image size at the center of the defect coordinates.

[0016] In one of the embodiments, the result image includes a difference image; and the difference detection on the defect image and the corresponding reference image to obtain a result image comprises:

[0017] performing pixel difference calculation on the defect image and the corresponding reference image, and obtaining a difference image based on the calculation result.

[0018] In one of the embodiments, the pixel difference calculation on the defect image and the corresponding reference image to obtain a difference image based on the calculation result comprises:

[0019] subtracting the gray value of each pixel point in the corresponding reference image from the gray value of each pixel point in the defect image to obtain a bright defect region image composed of each pixel point with a new gray value;

[0020] subtracting the gray value of each pixel point in the corresponding defect image from the gray value of each pixel point in the reference image to obtain a dark defect region image composed of each pixel point with a new gray value;

[0021] adding the gray value of the bright defect region image and the gray value of the dark defect region image, and generating a difference image based on the added gray value.

[0022] In one of the embodiments, after the difference detection on the defect image and the corresponding reference image to obtain a result image, and before the unsupervised learning based on the result image to obtain a candidate defect region in the defect image, the method further comprises:

[0023] performing a morphological opening operation on the result image to obtain a processed result image.

[0024] In one of the embodiments, the unsupervised learning based on the result image to obtain a candidate defect region in the defect image comprises:

[0025] performing a clustering analysis on the result image to obtain a clustering cluster;

[0026] generating a minimum circumscribed rectangle by traversing the clustering cluster;

[0027] drawing a region corresponding to a coordinate position of the minimum circumscribed rectangle in the defect image to obtain a candidate defect region.

[0028] In one of the embodiments, the clustering analysis on the result image to obtain a clustering cluster comprises:

[0029] performing a clustering on the result image by using an HDBSCAN clustering algorithm to obtain a clustering cluster.

[0030] In one of the embodiments, the defect position data comprises defect coordinates and defect size; and the method of determining a detection defect region in the defect image according to the defect position data of the defect image, and performing a set operation on the detection defect region and the candidate defect region to obtain a defect labeling region comprises:

[0031] determining a detection defect region in the defect image by taking the defect coordinates as a center and by determining a region with a size equal to the defect size;

[0032] performing a set operation on the detection defect region and the candidate defect region to obtain a defect labeling region.

[0033] A defect image labeling device comprises:

[0034] an information acquisition module configured to acquire a defect image of a detection target and defect information of the defect image, and acquire a reference image corresponding to the defect image, wherein the defect information comprises defect position data and defect categories;

[0035] a difference detection module configured to perform a difference detection on the defect image and the corresponding reference image to obtain a result image;

[0036] an image analysis module configured to perform unsupervised learning based on the result image to obtain a candidate defect region in the defect image;

[0037] a labeled region generation module configured to determine a detected defect region in the defect image according to defect position data of the defect image, and perform set processing on the detected defect region and the candidate defect region to obtain a defect labeled region;

[0038] a labeled information generation module configured to generate defect labeled information of the defect labeled region in the defect image based on a defect category of the defect image.

[0039] A computer device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0040] obtaining a defect image of a detection target and defect information of the defect image, and obtaining a reference image corresponding to the defect image, wherein the defect information comprises defect position data and a defect category;

[0041] performing difference detection on the defect image and the corresponding reference image to obtain a result image;

[0042] performing unsupervised learning based on the result image to obtain a candidate defect region in the defect image;

[0043] determining a detected defect region in the defect image according to defect position data of the defect image, and performing set processing on the detected defect region and the candidate defect region to obtain a defect labeled region;

[0044] generating defect labeled information of the defect labeled region in the defect image based on a defect category of the defect image.

[0045] A computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0046] obtaining a defect image of a detection target and defect information of the defect image, and obtaining a reference image corresponding to the defect image, wherein the defect information comprises defect position data and a defect category;

[0047] performing difference detection on the defect image and the corresponding reference image to obtain a result image;

[0048] performing unsupervised learning based on the result image to obtain a candidate defect region in the defect image;

[0049] determine a detected defect region in the defect image according to defect position data of the defect image, and perform set processing on the detected defect region and the candidate defect region to obtain a defect labeling region;

[0050] generate defect labeling information of the defect labeling region in the defect image based on a defect category of the defect image.

[0051] The defect image labeling method, device, computer device, and storage medium described above, after obtaining a defect image, a reference image corresponding to the defect image, and defect information, perform difference detection on the reference image and the defect image to obtain a result image, automatically extract a candidate defect region by performing unsupervised learning on the result image, perform set processing on the candidate defect region and a defect detection region determined based on defect position data in the defect information to obtain a defect labeling region, and generate defect labeling information according to an existing defect category in the defect information. In this way, the defect labeling region and the defect labeling information are obtained by fully utilizing the defect information, and automatic labeling of the defect image is implemented. Compared with existing manual labeling, the method avoids pure reliance on human eyes for searching and subjective identification and labeling, avoids problems such as incorrect labeling and missing labeling of the defect image, improves image labeling accuracy, and effectively improves labeling efficiency. Compared with a traditional automatic labeling method that uses a pre-trained model and is implemented by supervised learning, the present application realizes obtaining of the defect labeling region based on unsupervised learning by combining the existing defect information, and realizes generation of the defect labeling information by using the existing defect information, thereby avoiding automatic identification errors caused by complete reliance on the pre-trained model, and improving the accuracy of image labeling. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0053] Figure 1 A flowchart of a defect image labeling method in one embodiment;

[0054] Figure 2 A non-defect template image of a wafer;

[0055] Figure 3 A defect image of a wafer;

[0056] Figure 4 A Figure 3 A reference image corresponding to the defect image shown in FIG. 8;

[0057] Figure 5Flowchart of a defect image labeling method in another embodiment;

[0058] Figure 6 Morphological opening effect diagram;

[0059] Figure 7 Schematic diagram of detecting a defect region and a candidate defect region in an embodiment;

[0060] Figure 8 Schematic diagram of Figure 7 Schematic diagram of merging a defect labeling region in an embodiment of detecting a defect region and a candidate defect region;

[0061] Figure 9 Schematic diagram of drawing a defect labeling region in a defect image of a wafer;

[0062] Figure 10 Structural block diagram of a defect image labeling device in an embodiment. DETAILED DESCRIPTION

[0063] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms, and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments of the present application, and are not intended to limit the present application.

[0065] As used herein, the singular forms "a", "an" and "the" can also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "comprise / comprising" or "have / having" specify the presence of stated features, integers, steps, operations, components, parts, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

[0066] Reference Figure 1 In an embodiment, a defect image labeling method is provided, which can be applied to a computer device. The computer device can be a computer, a smart terminal, a wearable smart device, etc. which can execute a computer program. Taking the application to the computer device as an example, the method comprises the following steps S110 to S190.

[0067] S110: Obtain a defect image of a detection target and defect information of the defect image, and obtain a reference image corresponding to the defect image; the defect information includes defect position data and a defect category.

[0068] The detection target refers to a target that needs to be detected for defects to obtain a defect image and defect information. The defect image is an image containing a region corresponding to a detected defect; the defect position data represents the position of the defect in the defect image; and the defect category is information obtained by classifying the defect, which can be a category obtained by manual classification. Taking wafer detection as an example, the detection target is a wafer, and a defect detection device is used to detect defects of the wafer to obtain a defect image and defect position data. After wafer detection is completed, a person classifies the detected defects, and the defect category is input into the defect detection device. The defect detection device saves and processes the defect image, the defect position data, the defect category, and the like. A computer device obtains the defect image and the defect information, and obtains a reference image corresponding to the defect image based on the defect image and the defect information. Specifically, the defect information can be sent by the defect detection device to the computer device, or can be transferred to the computer device by other means, and the computer device loads the defect information.

[0069] S130: Perform difference detection on the defect image and the corresponding reference image to obtain a result image.

[0070] The number of defect images can be one or multiple; for multiple defect images, each has a corresponding reference image. Difference detection is performed on the defect image and the corresponding reference image to obtain a result image representing the difference between the two images.

[0071] S150: Perform unsupervised learning based on the result image to obtain a candidate defect region in the defect image.

[0072] Difference detection is performed on the defect image and the reference image to obtain a result image corresponding to the defect image; and unsupervised learning is performed on the result image to obtain a region as a candidate defect region in the defect image corresponding to the result image.

[0073] S170: Determine a detected defect region in the defect image according to the defect position data of the defect image, and perform set union processing on the detected defect region and the candidate defect region to obtain a defect labeling region.

[0074] Specifically, in the defect image, set union processing is performed on the detected defect region and the candidate defect region to obtain a defect labeling region of the defect image.

[0075] S190: Generate defect labeling information of the defect labeling region in the defect image based on the defect category of the defect image.

[0076] Defect categories are obtained based on defect detection of the target before image annotation. Defect annotation information is generated according to the defect category, which can effectively utilize the defect information obtained from defect detection to generate accurate defect annotation information.

[0077] Specifically, for all defect images obtained in step S110, steps S130 to S190 are performed to obtain the defect annotation areas and defect annotation information of all defect images.

[0078] The aforementioned defect image annotation method, after acquiring the defect image, its corresponding reference image, and defect information, performs difference detection on the reference and defect images to obtain a result image. Candidate defect regions are automatically extracted from the result image through unsupervised learning. These candidate defect regions are then combined with defect detection regions determined based on defect location data in the defect information to obtain the defect annotation region. Defect annotation information is generated based on existing defect categories in the defect information. Thus, by fully utilizing defect information to obtain the defect annotation region and generate the defect annotation information, automatic annotation of defect images is achieved. Compared to existing manual annotation methods, this avoids relying solely on human visual searching and subjective identification, preventing mis-annotation and omissions, improving image annotation accuracy and efficiency. Compared to traditional automatic annotation methods using pre-trained models through supervised learning, this application combines the acquisition of defect annotation regions based on unsupervised learning using existing defect information and the generation of defect annotation information using existing defect information. This avoids automatic identification errors that occur when relying entirely on pre-trained models, thus improving the accuracy of image annotation.

[0079] In one embodiment, the defect location data includes defect coordinates; step S110, the step of obtaining a reference image corresponding to the defect image includes: obtaining a defect-free template image of the detection target; determining the image size of the defect image; locating the defect coordinates of the defect image in the defect-free template image; cropping an image with a size equal to the image size centered on the defect coordinates to obtain a reference image corresponding to the defect image.

[0080] In this context, a defect-free template image refers to the image corresponding to an ideal target of the same category as the target being inspected, without any detected defects. By locating the defect based on its coordinates and determining its size range according to the image dimensions, a reference image can be extracted from the defect-free template image to obtain a relatively accurate image of the defect image corresponding to its location within the defect-free template image. Taking wafer inspection as an example, using the die template image as the defect-free template image, such as... Figure 2 As shown, determine Figure 3 The image size of the defect image of the wafer shown is determined by cropping a reference image from the die template image, based on the defect coordinates of the loaded wafer defect image, with the same coordinates as the defect coordinates and the same size as the image. For example...Figure 4 As shown.

[0081] In one embodiment, the result image comprises a difference image; the step S130 comprises a pixel difference calculation step: calculating the pixel difference between the defect image and the corresponding reference image, and obtaining the difference image based on the calculation result.

[0082] The difference image of the defect image and the reference image is obtained by calculating the pixel difference of the images, which reflects the image difference, i.e., highlights the defect information on the defect image. It can be understood that in other embodiments, other methods can also be used for difference detection, such as edge contour extraction and calculation of edge contour difference.

[0083] In one embodiment, the pixel difference calculation step comprises steps (a1) to (a3).

[0084] Step (a1): subtract the gray value of each pixel point in the corresponding reference image from the gray value of each pixel point in the defect image to obtain a bright defect region image composed of each pixel point with a new gray value.

[0085] That is, the gray value of each pixel point in the bright defect region image is equal to the gray value of the corresponding position pixel point in the defect image minus the gray value of the corresponding position pixel point in the corresponding reference image.

[0086] Step (a2): subtract the gray value of each pixel point in the corresponding defect image from the gray value of each pixel point in the reference image to obtain a dark defect region image composed of each pixel point with a new gray value.

[0087] That is, the gray value of each pixel point in the dark defect region image is equal to the gray value of the corresponding position pixel point in the reference image minus the gray value of the corresponding position pixel point in the corresponding defect image.

[0088] Step (a3): add the gray value of the bright defect region image and the gray value of the dark defect region image, and generate a difference image based on the added gray value.

[0089] In the actual wafer detection process, the image pixel gray value of the wafer defect region will be higher or lower than the image pixel gray value of the corresponding position in the Die template image. In this embodiment, the bright defect region image is generated based on the pixel points in the defect image whose gray value is higher than the corresponding position in the reference image, and the dark defect region image is generated based on the pixel points in the reference image whose gray value is higher than the corresponding position in the defect image. The difference image finally obtained contains both the bright defect region image and the dark defect region image, which can accurately reflect the image difference.

[0090] Specifically, the calculation formula of the difference image is as follows:

[0091]

[0092] Wherein, HighDifferenceImg represents the bright defect area image, DefectImge represents the defect image, InterceptImg represents the reference image, DarkDifferenceImg represents the dark defect area image, and DifferenctImg represents the difference image. By obtaining the inter-frame difference information of the reference image and the defect image, the relevant defect information in the defect image is retained, and the effectiveness of the subsequent unsupervised learning processing data is ensured.

[0093] In one of the embodiments, the reference Figure 5 , after step S130 and before step S150, further comprises the step of:

[0094] S140: performing morphological opening operation processing on the result image to obtain a processed result image.

[0095] In this embodiment, step S150 is to obtain candidate defect areas in the defect image based on the processed result image through unsupervised learning.

[0096] Morphology is one of the most widely used techniques in image processing, which is mainly used to extract image components that are meaningful to express and depict the shape of the region from the image, so that the subsequent recognition work can grasp the most essential shape features of the target object, such as boundary and connected region, etc. Opening operation is a processing of first eroding and then dilating the image, and the operation effect diagram is as shown in Figure 6 The mathematical expression is:

[0097] dst = open(src) = dilate(erode(src, element), element)

[0098] Wherein:

[0099] open(), representing morphological opening operation.

[0100] dilate(), representing dilation operation, which is an operation of "expanding" the selected area. The principle is to use a self-defined structure element to perform a sliding operation similar to "filtering" on the binary image to be processed, and then compare the corresponding pixels of the binary image with the pixels of the structure element, and the union obtained is the pixel of the dilated image.

[0101] erode(), representing erosion operation, which is an operation of "shrinking" the selected area, and can be used to eliminate edges and noise points.

[0102] Element represents a structural element, which is similar to a "filter kernel" element, or a "small window" that "slides" on the original image.

[0103] By performing a morphological opening operation on the difference image, small isolated pixel points and burrs in the difference image can be eliminated, objects can be separated at thin parts, and boundaries of large pixel blocks can be smoothed.

[0104] In one of the embodiments, the step S150 includes steps (b1) to (b3) as follows.

[0105] Step (b1): performing clustering analysis on the result image to obtain a clustering cluster.

[0106] Step (b2): generating a minimum circumscribed rectangle by traversing the clustering cluster.

[0107] Step (b3): drawing a region corresponding to the coordinate position of the minimum circumscribed rectangle in the defect image to obtain a candidate defect region.

[0108] By using the clustering algorithm idea of unsupervised learning to divide the pixel point set of the result image into subsets, the minimum circumscribed rectangle corresponding to the subsets is generated to obtain the candidate defect region in the defect image.

[0109] In one of the embodiments, step (b1) includes: using an HDBSCAN clustering algorithm to cluster the result image to obtain a clustering cluster.

[0110] Traditional unsupervised learning clustering algorithms mainly include partitioning-based methods (K-MEANS, K-MEDOIDS, CLARANS), hierarchical methods (BIRCH, CURE, CHAMELEON), density-based methods (DBSCAN, OPTICS, DENCLUE), grid-based methods (STING, CLIQUE, WAVE-CLUSTER), and model-based methods (statistical, neural networks). Except for DBSCAN, other clustering algorithms require setting the number of clusters, making them ineffective at extracting regions. This embodiment uses the HDBSCAN clustering algorithm, which combines density-based and hierarchical clustering, integrating DBSCAN with hierarchical clustering. Compared to traditional clustering algorithms, it can adaptively identify clusters of different densities, automatically determine the number of clusters, and, for some difficult-to-process datasets, identify small clusters that other clustering algorithms might classify as noisy data. The candidate defect regions to be extracted are uncertain. In this embodiment, the HDBSCAN clustering algorithm is used to cluster the result image, which can solve the problem of processing spherical clusters in other clustering processes. It can effectively extract defect clusters of arbitrary shapes, so as to effectively extract accurate candidate defect regions.

[0111] In one embodiment, the defect location data includes defect coordinates and defect size; step S170 includes: in the defect image, determining a region centered on the defect coordinates and equal in size to the defect size to obtain a detected defect region; performing a union operation on the detected defect region and the candidate defect region to obtain a defect annotation region.

[0112] The defect area determined based on defect coordinates and defect size is the defect area detected by the defect detection equipment, such as... Figure 7 As shown. Specifically, based on the coordinate system of the defect image, the detected defect region and the candidate defect region are processed by region union. (Reference) Figure 7 and Figure 8 A rectangular bounding box method can be used, employing the minimum bounding rectangle of the region union to generate the defect annotation region, which is then drawn on the defect image. Figure 9 As shown, by utilizing the defect location data in the defect information of the defect detection results, candidate defect regions generated by association and clustering are generated to produce the final defect annotation region, thereby improving the effectiveness of the defect annotation region.

[0113] In one embodiment, reference Figure 5 Step S190 is followed by:

[0114] S200: receiving a labeling data adjustment instruction, and adjusting the defect labeling region and / or the defect labeling information in response to the labeling data adjustment instruction.

[0115] The labeling data adjustment instruction is used to indicate adjustment of at least one of the defect labeling region and the defect labeling information, such as modification of the defect category in the defect labeling information. In this way, after the defect labeling region and the defect labeling information are generated, manual auditing, proofreading, and fine-tuning of the labeling information of the defect image can be allowed, further improving the accuracy of image labeling.

[0116] It should be understood that, although Figure 1 , Figure 5 the steps in the flowchart are displayed in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figure 1 , Figure 5 At least part of the steps in

[0117] In one embodiment, as shown in Figure 10 , a defect image labeling apparatus is provided, comprising: an information acquisition module 910, a difference detection module 930, an image analysis module 950, a labeling region generation module 970, and a labeling information generation module 990, wherein:

[0118] The information acquisition module 910 is configured to acquire a defect image of a detection target and defect information of the defect image, and acquire a reference image corresponding to the defect image, wherein the defect information includes defect position data and a defect category.

[0119] The difference detection module 930 is configured to perform difference detection on the defect image and the corresponding reference image to obtain a result image.

[0120] The image analysis module 950 is configured to perform unsupervised learning based on the result image to obtain a candidate defect region in the defect image.

[0121] The labeling region generation module 970 is configured to determine a detection defect region in the defect image according to the defect position data of the defect image, and perform set union processing on the detection defect region and the candidate defect region to obtain a defect labeling region.

[0122] The labeling information generation module 990 is configured to generate defect labeling information of the defect labeling region in the defect image based on a defect category of the defect image.

[0123] The defect image labeling device described above, after obtaining the defect image, the reference image corresponding to the defect image, and the defect information, performs difference detection on the reference image and the defect image to obtain a result image, automatically extracts a candidate defect region from the result image through unsupervised learning, performs set processing on the candidate defect region and a defect detection region determined based on defect position data in the defect information to obtain a defect labeling region, and generates defect labeling information according to an existing defect category in the defect information. In this way, the defect labeling region is obtained and the defect labeling information is generated by making full use of the defect information, and automatic labeling of the defect image is realized. Compared with the existing manual labeling, the problem of incorrect labeling and missed labeling of the defect image is avoided, the image labeling accuracy is improved, and the labeling efficiency is effectively improved. Compared with the automatic labeling mode realized by using a pre-trained model through supervised learning in the prior art, the defect labeling region is obtained based on unsupervised learning by combining the existing defect information, and the defect labeling information is generated by using the existing defect information, which avoids automatic recognition errors caused by complete dependence on the pre-trained model, and can improve the image labeling accuracy.

[0124] The specific limitations of the defect image labeling device can be referred to the limitations of the defect image labeling method in the foregoing, and will not be described herein. Each module in the defect image labeling device described above can be realized by software, hardware, or a combination thereof. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical function division. In actual implementation, another division mode can be used.

[0125] In one embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the steps in each method embodiment described above when executing the computer program.

[0126] The computer device described above can implement the steps in each method embodiment described above, and therefore can improve the image labeling accuracy.

[0127] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in each method embodiment described above.

[0128] The computer readable storage medium described above can improve the accuracy of image labeling because the computer program stored therein can implement the steps in the above method embodiments.

[0129] A person of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a nonvolatile computer readable storage medium and can include the processes in the above method embodiments when executed. Any reference to memory, storage, database or other medium in the embodiments provided by the present application can include at least one of nonvolatile and volatile memory. The nonvolatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, the RAM can be in various forms such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0130] In the description of the present specification, the description of the terms "some embodiments", "other embodiments", "ideal embodiments" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example.

[0131] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present application.

[0132] The above embodiments only express several implementation manners of the present application, and the description is specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A defect image labeling method characterized by comprising: The method comprises the following steps: acquiring a defect image of a detection target and defect information of the defect image, acquiring a reference image corresponding to the defect image based on the defect image and the defect information, wherein the defect information comprises defect position data and defect category; performing difference detection on the defect image and the corresponding reference image to obtain a result image; performing unsupervised learning based on the result image to obtain a candidate defect region in the defect image; determining a detection defect region in the defect image according to the defect position data of the defect image, performing set operation on the detection defect region and the candidate defect region to obtain a defect labeling region; generating defect labeling information of the defect labeling region in the defect image based on the defect category of the defect image; the defect position data comprises defect coordinates and defect size; the step of determining the detection defect region in the defect image according to the defect position data of the defect image, performing set operation on the detection defect region and the candidate defect region to obtain a defect labeling region comprises: determining a region with a size equal to the defect size in the defect image with the defect coordinates as the center to obtain the detection defect region; performing set operation on the detection defect region and the candidate defect region to obtain the defect labeling region.

2. The method of claim 1, wherein, the defect position data comprises defect coordinates; the step of acquiring the reference image corresponding to the defect image comprises: acquiring a defect-free template image of the detection target; determining the image size of the defect image; positioning the defect coordinates of the defect image in the defect-free template image, and cutting an image with a size equal to the image size with the defect coordinates as the center to obtain the reference image corresponding to the defect image.

3. The method of claim 1, wherein, the result image comprises a difference image; the step of performing difference detection on the defect image and the corresponding reference image to obtain a result image comprises: performing pixel difference calculation on the defect image and the corresponding reference image, and obtaining a difference image based on the calculation result.

4. The method of claim 3, wherein, the step of performing pixel difference calculation on the defect image and the corresponding reference image, and obtaining a difference image based on the calculation result comprises: subtracting the gray value of each pixel point in the corresponding reference image from the gray value of each pixel point in the defect image to obtain a bright defect region image composed of each pixel point with a new gray value; subtracting the gray value of each pixel point in the corresponding defect image from the gray value of each pixel point in the reference image to obtain a dark defect region image composed of each pixel point with a new gray value; adding the gray value of the bright defect region image and the gray value of the dark defect region image to generate a difference image based on the added gray value.

5. The method according to any one of claims 1 to 4, characterized in that, after the step of performing difference detection on the defect image and the corresponding reference image to obtain a result image, and before the step of performing unsupervised learning based on the result image to obtain a candidate defect region in the defect image, the method further comprises: performing morphological opening operation processing on the result image to obtain a processed result image.

6. The method of claim 1, wherein, the step of performing unsupervised learning based on the result image to obtain a candidate defect region in the defect image comprises: performing clustering analysis on the result image to obtain a clustering cluster; generating a minimum circumscribed rectangle by traversing the clustering cluster; drawing a region corresponding to a coordinate position of the minimum circumscribed rectangle in the defect image to obtain a candidate defect region.

7. The method of claim 6, wherein, The performing clustering analysis on the result image to obtain a clustering cluster comprises: performing clustering on the result image by using an HDBSCAN clustering algorithm to obtain a clustering cluster.

8. A defect image labeling apparatus characterized by comprising: comprise: an information acquisition module, configured to acquire a defect image of a detection target and defect information of the defect image, and acquire a reference image corresponding to the defect image based on the defect image and the defect information, wherein the defect information comprises defect position data and defect categories; a difference detection module, configured to perform difference detection on the defect image and the corresponding reference image to obtain a result image; an image analysis module, configured to perform unsupervised learning based on the result image to obtain a candidate defect region in the defect image; a labeled region generation module, configured to determine a detected defect region in the defect image according to the defect position data of the defect image, and perform set union processing on the detected defect region and the candidate defect region to obtain a defect labeled region; a labeled information generation module, configured to generate defect labeled information of the defect labeled region in the defect image based on the defect categories of the defect image; The defect position data comprises defect coordinates and defect sizes; and the determining a detected defect region in the defect image according to the defect position data of the defect image, and performing set union processing on the detected defect region and the candidate defect region to obtain a defect labeled region comprises: determining a region equal to the defect sizes with the defect coordinates as the center in the defect image to obtain a detected defect region; performing set union processing on the detected defect region and the candidate defect region to obtain a defect labeled region. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

Images