Defect labeling method and device, server and storage medium

Abstract

The invention discloses a defect labeling method and device, a server and a storage medium. The defect labeling method comprises the following steps: acquiring a target image of a target object; determining a region image of at least one candidate defect region of the target object from the target image; performing defect area identification processing based on each area image to obtain a target defect area in each area image; determining a defect category corresponding to each target defect area based on the similarity between the target defect areas; and based on the defect category corresponding to each target defect area, defect category labeling is performed on the target image, so that various defects can be effectively and adaptively extracted and classified, and the accuracy and efficiency of defect labeling are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a defect marking method, device, server and storage medium. Background Art

[0002] Currently, industrial defect detection solutions using deep learning are a popular research field in machine vision. They are of great significance in industrial intelligent manufacturing. They can greatly reduce the cost of manual quality inspection and significantly improve the performance and efficiency of inspection. However, most deep learning models are highly dependent on data. The richer the data, the higher the accuracy. For this reason, many solutions have proposed to use small samples and semi-supervised solutions to build and train deep learning networks. However, when the appearance defects are more complex, additional data is still required for training convergence. At the same time, industrial defect data also has its own difficulties. The defect location, size, and shape are not fixed, and the regional discreteness is not uniform. There are high requirements for manual labeling of data, resulting in low efficiency in defect data labeling. Summary of the invention

[0003] The embodiments of the present application provide a defect annotation method, device, server, and storage medium, which can improve the accuracy and efficiency of defect annotation to at least partially solve the above-mentioned technical problems.

[0004] On the one hand, an embodiment of the present application provides a defect marking method, including:

[0005] Acquire a target image of a target object;

[0006] Determining, from the target image, a regional image of at least one candidate defect region of the target object;

[0007] Perform defect area recognition processing based on each area image to obtain a target defect area in each area image;

[0008] Determine the defect category corresponding to each target defect area based on the similarity between each target defect area;

[0009] Based on the defect categories corresponding to the target defect areas, the target image is labeled with defect categories.

[0010] In some embodiments of the present application, determining a regional image of at least one candidate defect region of the target object from the target image includes:

[0011] Extracting multiple candidate regions from the target image;

[0012] Determine a first similarity between each candidate region;

[0013] Merging the plurality of candidate regions based on the first similarity to obtain the candidate defect region;

[0014] The region image is determined based on the candidate defect regions.

[0015] In some embodiments of the present application, determining the regional image based on the candidate defect region includes:

[0016] Obtaining the region coordinates corresponding to the candidate defect region in the target image;

[0017] The target image is cropped based on the region coordinates to obtain the region image.

[0018] In some embodiments of the present application, the defect area recognition process is performed based on each regional image to obtain the target defect area in each regional image, including:

[0019] Performing feature extraction on the regional image to obtain image features corresponding to the regional image;

[0020] Based on the image features, calculating the defect probability corresponding to each position in the regional image;

[0021] The target defect region is determined from the region image based on the defect probability.

[0022] In some embodiments of the present application, determining the target defect area from the regional image based on the defect probability includes:

[0023] Based on the defect probability of each position in the regional image, determining a probability map corresponding to the regional image, wherein the pixel points in the probability map correspond one-to-one to the positions in the regional image;

[0024] Compare the defect probability corresponding to each pixel in the probability map with a first preset threshold;

[0025] Determine a target pixel point whose defect probability is not less than the first preset threshold;

[0026] Based on the first region where the target pixel point in the probability map is located, a region corresponding to the first region is determined from the regional image to obtain the target defect region.

[0027] In some embodiments of the present application, the step of determining the area corresponding to the first area in the regional image based on the first area where the target pixel point is located in the probability map to obtain the target defect area includes:

[0028] generating a binary mask image based on the first region;

[0029] A connected region is extracted from the binary mask image, and the connected region is mapped to the region image to obtain the target defect region.

[0030] In some embodiments of the present application, the method further includes:

[0031] Acquire location information of the regional image in the target image;

[0032] Extracting a position feature corresponding to the position information, and fusing the position feature with the image feature to obtain a fused feature;

[0033] The calculating the defect probability corresponding to each position in the regional image based on the image feature includes:

[0034] Based on the fused features, the defect probability corresponding to each position in the regional image is calculated.

[0035] In some embodiments of the present application, determining the defect category corresponding to each target defect area based on the similarity between the target defect areas includes:

[0036] Obtaining a second similarity between each target defect area;

[0037] Clustering is performed on each target defect region based on the second similarity to obtain a plurality of clusters, and a defect category is set for each cluster to determine the defect category corresponding to the target defect region in the cluster.

[0038] In some embodiments of the present application, clustering is performed on each target defect area based on the second similarity to obtain a plurality of clusters, including:

[0039] Selecting m target defect regions from the target defect regions as cluster centers of m initial clustering clusters;

[0040] Based on the similarity between the remaining target defect regions and the cluster center, the remaining target defect regions are clustered to obtain a plurality of cluster clusters.

[0041] In some embodiments of the present application, the remaining target defect regions are clustered based on the similarity between the remaining target defect regions and the cluster center to obtain multiple cluster clusters, including:

[0042] If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is not less than a second preset threshold, classifying the remaining target defect regions into the initial cluster corresponding to the cluster center;

[0043] If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is less than the second preset threshold, classifying the remaining target defect regions into a new cluster;

[0044] Based on the initial clustering cluster and the new clustering cluster, a plurality of clustering clusters are obtained.

[0045] In some embodiments of the present application, obtaining the second similarity between the target defect regions includes:

[0046] Obtaining the similarity of each target defect area in at least one dimension;

[0047] The second similarity is determined based on the similarity in at least one of the dimensions.

[0048] In some embodiments of the present application, the step of acquiring a target image of a target object includes:

[0049] Acquiring multiple images of the target object in multiple modalities;

[0050] The multiple images are fused to obtain the target image.

[0051] On the other hand, an embodiment of the present application further provides a defect marking device, comprising:

[0052] A first acquisition unit, used to acquire a target image of a target object;

[0053] A first determining unit, configured to determine a region image of at least one candidate defect region of the target object from the target image;

[0054] A first processing unit is used to perform defect area recognition processing based on each area image to obtain a target defect area in each area image;

[0055] A second processing unit, configured to determine a defect category corresponding to each target defect area based on the similarity between the target defect areas;

[0056] The second determining unit is used to label the target image with a defect category based on the defect category corresponding to each target defect area.

[0057] On the other hand, an embodiment of the present application further provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor performs the following steps:

[0058] Acquire a target image of a target object;

[0059] Determining, from the target image, a regional image of at least one candidate defect region of the target object;

[0060] Perform defect area recognition processing based on each area image to obtain a target defect area in each area image;

[0061] Determine the defect category corresponding to each target defect area based on the similarity between each target defect area;

[0062] Based on the defect categories corresponding to the target defect areas, the target image is labeled with defect categories.

[0063] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, including a computer program, wherein the computer program is loaded by a processor to execute the steps in any defect marking method provided in the embodiment of the present application.

[0064] On the other hand, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of any defect marking method provided in the embodiment of the present application.

[0065] The embodiment of the present application obtains a target image of a target object; determines a regional image of at least one candidate defect area of ​​the target object from the target image; performs defect area recognition processing based on each regional image to obtain a target defect area in each regional image; determines a defect category corresponding to each target defect area based on a similarity between each target defect area; and labels the target image with a defect category based on the defect category corresponding to each target defect area. In this way, various defects can be effectively and adaptively extracted and classified, thereby improving the accuracy and efficiency of defect labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 It is a schematic diagram of an embodiment of the defect marking method provided in the embodiments of the present application;

[0068] Figure 2 is a schematic diagram of the structure of an embodiment of a defect marking device provided in an embodiment of the present application;

[0069] Figure 3 It is a schematic diagram of the structure of the server provided in the embodiment of the present application. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0071] In the following description, specific embodiments of the present invention will be described with reference to steps and symbols performed by one or more computers, unless otherwise specified. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and computer execution referred to herein includes operations by a computer processing unit of electronic signals representing data in a structured form. This operation converts the data or maintains it at a location in the memory system of the computer, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to testers in the field. The data structure maintained by the data is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present invention are described in the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.

[0072] The term "module" or "unit" used herein can be regarded as a software object executed on the computing system. The different components, modules, engines and services described herein can be regarded as implementation objects on the computing system. The apparatus and method described herein are preferably implemented in software, but can also be implemented in hardware, all within the scope of protection of the present invention.

[0073] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0074] The embodiments of the present application provide a defect labeling method, device, server and storage medium. Specifically, the defect labeling method of the embodiments of the present application can be executed by an electronic device or a server. The electronic device can be a desktop terminal or a mobile terminal, for example, a mobile phone, a tablet computer, a laptop computer, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0075] The following describes it in detail with reference to specific embodiments.

[0076] In this embodiment, the defect marking method is applied in a server for illustration.

[0077] The present invention provides a defect labeling method, which includes: acquiring a target image of a target object; determining at least one regional image from the target image based on image information of the target image; performing defect region recognition processing based on each regional image to obtain a target defect region in each regional image; determining a defect category corresponding to each target defect region based on a similarity between each target defect region; and obtaining defect information of the target object based on each target defect region and the defect category to which each target defect region belongs.

[0078] For example, see Figure 1 , Figure 1 : is a schematic diagram of an embodiment of a defect marking method provided in an embodiment of the present application, and the defect marking method includes:

[0079] 101. Obtain a target image of a target object.

[0080] In the embodiment of the present application, the target object refers to an object with defects, for example, the target object may include an industrial product with defects, etc.

[0081] The target image refers to an image containing a target object, which can be obtained by capturing an image of the target object.

[0082] In some embodiments, in order to accurately identify the defective part of the target object, the step of "obtaining a target image of the target object" may include the following process:

[0083] Acquire multiple images of the target object in multiple modalities;

[0084] Multiple images are fused to obtain the target image.

[0085] Among them, multiple modalities refer to different image acquisition conditions. For example, multiple modalities may refer to different spectral ranges. Then, acquiring multiple images of the target object under multiple modalities may include: acquiring images of the target object in different spectral ranges.

[0086] For example, the multiple spectral ranges may include: visible light, infrared light, and ultraviolet light, and the multiple images collected may include: infrared light images, visible light images, and ultraviolet light images.

[0087] In some embodiments, acquiring images of the target object in different spectral ranges may include: acquiring images of the target object in different spectral ranges by using an optical sensor.

[0088] The fusing of the multiple images may include: fusing the channels of the multiple images.

[0089] In some embodiments, the image sizes of the target object in multiple modalities may be the same, but the image colors may be different. Then, channel fusion of the multiple images may be to fuse the information of the color channels of the multiple images to obtain a fused image, which may be used as the target image.

[0090] In an embodiment of the present application, by fusing multiple images of a target object in multiple modalities, the obtained target image can contain all key information and features of the original image, thereby improving the quality and information content of the image.

[0091] 102. Determine, from the target image, a region image of at least one candidate defect region of the target object.

[0092] The candidate defect region refers to a region initially screened out from the target image and containing defective parts of the target object.

[0093] In some embodiments, the step of “determining a regional image of at least one candidate defect region of the target object from the target image” may include the following operations:

[0094] Extract multiple candidate regions from the target image;

[0095] Determine a first similarity between each candidate region;

[0096] Based on the first similarity, multiple candidate regions are merged to obtain a candidate defect region;

[0097] A region image is determined based on the candidate defect regions.

[0098] The step of extracting a plurality of candidate regions from the target image may include segmenting the target image to obtain a plurality of regions in the target image as candidate regions.

[0099] Determining the first similarity between the candidate regions may include: determining the similarity between the candidate regions as the first similarity based on image information of the candidate regions.

[0100] The image information may at least include: color, texture, size and other information.

[0101] Among them, merging multiple candidate regions based on the first similarity to obtain a candidate defect region may include: merging candidate regions whose first similarity is greater than a preset similarity, that is, merging similar candidate regions among the multiple candidate regions to obtain a candidate defect region.

[0102] In some embodiments, determining a regional image of at least one candidate defect region of the target object from the target image may include: determining a regional image of at least one candidate defect region of the target object from the target image by using a region proposal algorithm.

[0103] Among them, the region proposal algorithm is an algorithm that can automatically identify and extract potential target areas from an image. The purpose is to reduce the computational complexity of the target detection task, and by generating a series of high-quality candidate regions, the subsequent classifier can locate and identify the target more efficiently. Region proposal algorithms can include: Selective Search algorithm, etc.

[0104] In some embodiments, the target image may be processed by a Selective Search algorithm to extract potential defect areas from the target image as candidate areas.

[0105] Specifically, the step of extracting the region of the target image by the Selective Search algorithm may include: Selective Search calculates the hierarchical grouping of similar regions based on the color, texture, size and shape compatibility of the target image. First, the target image is segmented using a graph-based segmentation method (such as the algorithm of Felzenszwalb and Huttenlocher) to generate a series of initial small regions (i.e., multiple candidate regions). Then, by calculating the similarity between these small regions, the most similar regions are gradually merged to generate larger and more complex candidate regions as candidate defect regions.

[0106] In some embodiments, in order to accurately obtain the regional image corresponding to the candidate defect region, the step of “determining the regional image based on the candidate defect region” may include the following operations:

[0107] Obtain the region coordinates corresponding to the candidate defect region in the target image;

[0108] The target image is cropped based on the region coordinates to obtain a region image.

[0109] The region coordinates refer to the position coordinates of the candidate region in the target image.

[0110] The region coordinates corresponding to the candidate defect region in the target image are obtained, and the candidate defect region can be processed by a morphological filtering method to determine the region coordinates corresponding to the candidate defect region.

[0111] Among them, the basic principle of morphological filtering method is to use morphological operators to process images. Morphological operators are special filters, usually composed of structural elements and operation functions. The structural element is a small image block, which defines the local shape and size of the morphological operation on the image. The operation function defines the movement and deformation of the structural element on the image. Different morphological operations can be achieved through different operation functions.

[0112] In an embodiment of the present application, the step of processing the candidate defect region by a morphological filtering method may include: by a morphological gradient operation, the edge information of the candidate defect region in the target image may be highlighted. The morphological gradient is the difference between the dilation map and the erosion map. Performing this operation on a binary image may highlight the edge of the mass. Thus, according to the edge position of the candidate defect region, the regional coordinates of the candidate defect region in the target image may be determined.

[0113] The step of cropping the target image based on the regional coordinates to obtain the regional image may include: cropping a portion of the image corresponding to the regional coordinates from the target image as the regional image corresponding to the candidate defect region.

[0114] In some embodiments, the candidate defect regions may include multiple ones, and then the regional images corresponding to the candidate defect regions may be cropped from the target image to obtain multiple regional images.

[0115] 103. Perform defect region recognition processing based on each region image to obtain a target defect region in each region image.

[0116] The regional image refers to an image corresponding to the initially screened candidate defect region. In order to further determine the precise defect contour in the regional image, defect region recognition processing may also be performed on the regional image.

[0117] In some embodiments, in order to obtain accurate defect contour information, the step of "performing defect region identification processing based on each region image to obtain a target defect region in each region image" includes:

[0118] Extract features from the regional image to obtain image features corresponding to the regional image;

[0119] Based on the image features, the defect probability corresponding to each position in the regional image is calculated;

[0120] A target defect region is determined from the region image based on the defect probability.

[0121] In the embodiment of the present application, defective region recognition processing is performed on the regional image, and the defective region recognition processing can be performed on the regional image through a segmentation network. The segmentation network can include a Segment Anything (SAM) segmentation algorithm, etc.

[0122] Among them, the principle of the Segment Anything segmentation algorithm is mainly based on the combination of convolutional neural network (CNN) and Transformer architecture.

[0123] The SAM algorithm uses an encoder-decoder structure similar to U-Net. This structure includes an encoder part and a decoder part. Among them, the encoder: consists of multiple convolutional layers and pooling layers, which are used to extract image features. Each convolutional layer usually includes operations such as convolution kernels, activation functions, and batch normalization to extract features and reduce the dimension of the input image. The pooling layer is used to downsample the feature map to reduce the amount of calculation and memory consumption; the decoder: consists of multiple deconvolution layers and upsampling layers, which are used to restore the feature map to the original image size and generate segmentation results. Each deconvolution layer usually includes operations such as deconvolution kernels, activation functions, and batch normalization to upsample and fuse feature maps. The upsampling layer is used to upsample the feature map to restore the original image size.

[0124] Among them, extracting features from the regional image to obtain image features corresponding to the regional image may include: inputting the regional image into a segmentation network, extracting features from the regional image through an encoder in the segmentation network, that is, mapping image information of the regional image to a high-dimensional image vector, thereby obtaining image features corresponding to the regional image.

[0125] Wherein, based on the image features, calculating the defect probability corresponding to each position in the regional image may include: inputting the image features into a decoder of the segmentation network, and calculating the defect probability corresponding to each position in the regional image. The defect probability refers to the probability of being a defect.

[0126] The decoder in the segmentation network can efficiently map the image embedding, hint embedding, and output token to the mask. A modified version of the Transformer decoder block is used, followed by a dynamic mask prediction head, to update all embeddings through self-attention and cross-attention (hint to image embedding and vice versa). Finally, the output token is mapped to a dynamic linear classifier through an MLP (MultilayerPerceptron) to calculate the mask foreground probability for each image position.

[0127] In some embodiments, the step of “determining a target defect region from the region image based on the defect probability” may include the following operations:

[0128] Determine a probability map corresponding to the regional image based on the defect probability of each position in the regional image;

[0129] Compare the defect probability corresponding to each pixel in the probability map with a first preset threshold;

[0130] Determine a target pixel point whose defect probability is not less than a first preset threshold;

[0131] Based on the first region where the target pixel point in the probability map is located, a region corresponding to the first region is determined from the regional image to obtain a target defect region.

[0132] Among them, the encoder of the segmentation network is responsible for extracting high-level features of the image, while the decoder maps these features back to the spatial dimensions of the original image to generate the segmentation result. In this process, the network outputs one or more probability maps, each corresponding to a category. Each pixel value in the probability map represents the probability that the pixel belongs to the corresponding category. For example, in a two-classification problem, the probability map may contain only one channel, where the area with a pixel value close to 1 indicates that the area is likely to belong to the foreground category, and the area with a pixel value close to 0 indicates that the area is likely to belong to the background category.

[0133] In the embodiment of the present application, the segmentation network can output a probability map corresponding to the regional image based on the defect probability of each position in the regional image calculated. Each pixel in the probability map corresponds to each position in the regional image one by one, and the pixel value of each pixel represents the probability of the position corresponding to it.

[0134] Among them, in the probability map corresponding to the regional image, the area with a pixel value close to 1 indicates that the area is likely to belong to the foreground category (that is, a defect), and the area with a pixel value close to 0 indicates that the area is likely to belong to the background category (that is, not a defect).

[0135] In order to obtain the final segmentation result from the probability map, a threshold is usually applied to the probability map. For example, in a binary classification problem, pixels with a probability greater than 0.5 can be considered as foreground categories, while pixels with a probability less than or equal to 0.5 can be considered as background categories.

[0136] In the embodiment of the present application, a first preset threshold may be applied to the probability map to accurately obtain the defective area from the probability map, that is, to compare the defect probability corresponding to each pixel in the probability map with the first preset threshold. For example, the first preset threshold may be: 0.5.

[0137] The defect probability corresponding to each pixel is the probability represented by the pixel value of each pixel.

[0138] Among them, determining the target pixel points whose defect probability is not less than the first preset threshold value may include: screening out pixel points whose pixel values ​​are greater than or equal to the first preset threshold value from all the pixel points in the probability map as target pixel points, and the area where these target pixel points are located may be the defective area.

[0139] The first region refers to the region where the target pixel point is located in the probability map.

[0140] Furthermore, the target pixel point in the first region where the probability map is located may be mapped to the regional image, thereby obtaining a target defect region belonging to the defect part in the regional image.

[0141] In some embodiments, in order to improve the accuracy of obtaining the target defect area, the step of "determining the area corresponding to the first area from the regional image based on the first area where the target pixel point is located in the probability map to obtain the target defect area" may include the following steps:

[0142] generating a binary mask image based on the first region;

[0143] The connected regions are extracted from the binary mask image and mapped to the region image to obtain the target defect region.

[0144] Among them, the binary mask image is a special mask whose pixel values ​​are only 0 or 1. This mask is usually used to identify specific areas in an image, for example, separation of foreground and background, extraction of target objects, etc.

[0145] Specifically, the steps of generating a binary mask image may include: first, converting the original image into a grayscale image, and then setting a threshold: setting a threshold according to the image content to convert the grayscale image into a binary image. The selection of the threshold is usually based on image histogram analysis or empirical values. Finally, performing a binarization process: setting the pixel values ​​in the grayscale image that are greater than or equal to the threshold to 1 (white), and setting the pixel values ​​that are less than the threshold to 0 (black), thereby obtaining a binary mask image. The binary mask image can be used to conveniently extract specific areas in the image and realize image segmentation.

[0146] For example, in an embodiment of the present application, the pixel values ​​in the first area of ​​the probability map may be set to 1, and the pixel values ​​outside the first area may be set to 0, thereby obtaining a binary mask map with the first area as the mask area.

[0147] A connected region is a region in an image that consists of adjacent pixels (usually pixels with the same pixel value). Extracting connected regions from an image is a common task in image processing and is often used in applications such as image segmentation and feature extraction.

[0148] In an embodiment of the present application, a connected region analysis algorithm, such as a Two-Pass algorithm, a Seed Filling algorithm, or a connectedComponents function of OpenCV, may be used to extract connected regions from a binary mask image.

[0149] Among them, the Two-Pass algorithm needs to traverse the image twice to extract connected regions: the first traversal: assign a temporary label to each pixel of interest (pixel with value 1). If the left and upper neighbors of a pixel are both 0, assign a new label to the pixel; otherwise, set the label of the pixel to the minimum value of the labels in its left and upper neighbors. At the same time, record the connected region to which each label belongs; the second traversal: replace the temporary label of each pixel with the minimum label of the connected region to which it belongs. In this way, pixels with the same label form the same connected region.

[0150] Among them, the Seed Filling algorithm extracts connected regions, including: starting from a seed pixel, searching for neighboring pixels with the same pixel value around it, and marking them with the same label. Then, continue to search for new seeds among these neighboring pixels until there are no more neighboring pixels to search. In this way, a connected region is found. Repeat this process until all connected regions in the image are found.

[0151] Among them, the connectedComponents function extracts connected regions by accepting a binary image as input and returning a label image, in which each connected region is assigned a unique label. In addition, the connectedComponentsWithStats function also provides statistical information of each connected region, such as area, bounding rectangle, centroid, etc.

[0152] In an embodiment of the present application, mapping a connected area to a regional image to obtain a target defect area can include: obtaining the position coordinates of the connected area in a binary mask image, mapping the position coordinates to the regional image, and determining the area corresponding to the position coordinates in the regional image, which can be used as the target defect area.

[0153] In some embodiments, in order to improve the accuracy of segmentation network processing, the method may further include the following steps:

[0154] Obtaining the location information of the regional image in the target image;

[0155] Extracting the position features corresponding to the position information, and fusing the position features with the image features to obtain fused features;

[0156] The position information of the regional image in the target image may include the position coordinates of the candidate defect region corresponding to the regional image in the target image.

[0157] Among them, to extract the position features corresponding to the position information, the position information can be input into the encoder of the segmentation network, and the encoder can be used to extract features of the position information, that is, to map the position information to a high-dimensional encoding vector as the position feature.

[0158] Specifically, the SAM algorithm can also support promptable segmentation tasks, that is, returning a valid segmentation mask based on segmentation prompts. Hints can contain spatial or textual information for identifying objects in images.

[0159] The segmentation network includes a hint encoder, which can be used to encode hints. Two types of hints are considered: sparse (points, boxes, text) and dense (masks). Points and boxes are represented using positional encoding and summed with the learned embeddings of each hint type and free-form text. Dense hints (i.e., masks) are embedded using convolutions and summed element-by-element with the image embedding.

[0160] For example, the position information is encoded by the prompt encoder to obtain a sparse vector corresponding to the position information as a position feature, and then the position feature is fused into the image feature to obtain a fused feature. Thus, the fused feature includes the image information of the regional image and the position information.

[0161] In some embodiments, the step of “calculating the defect probability corresponding to each position in the regional image based on the image features” may include the following operations:

[0162] Based on the fused features, the defect probability corresponding to each position in the regional image is calculated.

[0163] Specifically, the fused features are input into the decoder of the segmentation network, and the decoder calculates the defect probability corresponding to each position in the regional image according to the fused features.

[0164] 104. Determine the defect category corresponding to each target defect area based on the similarity between the target defect areas.

[0165] In some embodiments, the step of “determining the defect category corresponding to each target defect region based on the similarity between each target defect region” may include the following operations:

[0166] Obtaining a second similarity between each target defect area;

[0167] Clustering is performed on each target defect region based on the second similarity to obtain a plurality of clusters, and a defect category is set for each cluster to determine the defect category corresponding to the target defect region in the cluster.

[0168] The second similarity refers to the similarity between the target defect regions.

[0169] In the embodiment of the present application, a plurality of target defect regions may be clustered according to the similarities between the target defect regions to obtain a plurality of defect categories and a defect category corresponding to each target region.

[0170] Among them, clustering processing can be performed on multiple target defect areas by using a clustering algorithm. The clustering algorithm divides a data set into different classes or clusters according to a specific standard (such as distance or similarity, etc.), so that the similarity of data objects in the same cluster is as large as possible, and the difference of data objects not in the same cluster is as large as possible.

[0171] In some embodiments, in order to improve the accuracy of defect classification, the step of “clustering each target defect region based on the second similarity to obtain multiple clusters” may include the following operations:

[0172] Select m target defect areas from the target defect areas as cluster centers of m initial clustering clusters;

[0173] Based on the similarity between the remaining target defect regions and the cluster center, the remaining target defect regions are clustered to obtain multiple clusters.

[0174] A cluster is a category combination, and a cluster may include multiple objects belonging to the same category.

[0175] Among them, selecting m target defect areas from the target defect areas as the cluster centers of the m initial clustering clusters may include: randomly selecting m target defect areas from multiple target defect areas, where m is less than the total number of target defect areas; and using the m target defect areas as the cluster centers of the m initial clustering clusters.

[0176] In some embodiments, when m target defect regions are randomly selected from a plurality of target defect regions, m target defect regions having relatively large similarity differences may be selected.

[0177] The remaining target defect regions refer to target defect regions among the multiple target defect regions except the m target defect regions selected as cluster centers.

[0178] Furthermore, the remaining target defects may be clustered according to the similarity between the remaining target defect regions and the target defect region serving as the cluster center, thereby obtaining a plurality of cluster clusters corresponding to all the target defect regions.

[0179] In some embodiments, the step of “clustering the remaining target defect regions based on the similarity between the remaining target defect regions and the cluster center to obtain a plurality of cluster clusters” may include the following operations:

[0180] If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is not less than a second preset threshold, the remaining target defect regions are classified into the initial cluster corresponding to the cluster center;

[0181] If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is less than a second preset threshold, the remaining target defect regions are classified into a new cluster;

[0182] Based on the initial clustering cluster and the new clustering cluster, multiple clustering clusters are obtained.

[0183] Among them, the second preset threshold is used to judge the similarity between each target defect area; if the second similarity between two target defect areas is greater than or equal to the second preset threshold, it can be determined that the two target defect areas are similar; conversely, if the second similarity between the two target defect areas is less than the second preset threshold, it can be determined that the two target defect areas are not similar.

[0184] Specifically, the second similarities between the remaining target defect regions and the target defect regions corresponding to the cluster centers are compared with the second preset threshold value. If the second similarity between the target defect region corresponding to the cluster center and the remaining target defect regions is not less than the second preset threshold value, the remaining target defect regions can be classified into the initial clustering cluster corresponding to the cluster center.

[0185] If there is no target defect area corresponding to the cluster center and the second similarity between the remaining target defect areas is not less than the second preset threshold, the remaining target defect areas can be classified into a new cluster and the remaining target defect areas are used as the cluster center of the new cluster.

[0186] In this way, by cyclically comparing the similarity between each target defect region and the target defect region corresponding to the cluster center of each cluster with the second preset threshold, multiple clusters can be clustered so that the similarity between the target defect regions in any two clusters is less than the second preset threshold. That is, the target defect regions in the same cluster belong to similar regions.

[0187] In some embodiments, the step of “obtaining the second similarity between each target defect region” may include the following operations:

[0188] Obtaining the similarity of each target defect area in at least one dimension;

[0189] Based on the similarity in at least one dimension, a second similarity is determined.

[0190] The at least one dimension may include: brightness, contrast and / or structure.

[0191] Specifically, obtaining the similarity of each target defect region in at least one dimension may include: obtaining the similarity of each target defect region in brightness, the similarity in contrast, and the similarity in structure.

[0192] Determining the second similarity based on the similarity in at least one dimension may include: determining the second similarity between the target defect regions based on similarity in brightness, similarity in contrast, and / or similarity in structure.

[0193] In some embodiments, the similarity between target defect regions may be constructed by a Structural Similarity (SSIM) algorithm, and the algorithm function is as follows:

[0194] S k1k2 =F(L k1k2 , C k1k2 , U k1k2 );

[0195] Among them, Sk1k2 Refers to the second similarity between the target defect area K1 and the target defect area K2; L k1k2 Refers to the brightness similarity between the target defect area K1 and the target defect area K2; C k1k2 Refers to the contrast similarity between the target defect area K1 and the target defect area K2; U k 1 k 2 refers to the structural similarity between the target defect area K1 and the target defect area K2. By integrating the brightness similarity, contrast similarity and structural similarity, S k1k2。

[0196] Among them, SSIM is a perception-based image quality evaluation index that measures the similarity of two images from three aspects: brightness, contrast, and structure. The larger its value, the smaller the image distortion and the more similar the two images are. SSIM defines structural information as an attribute that is independent of brightness and contrast and reflects the structure of objects in the scene, and models distortion as a combination of three different factors: brightness, contrast, and structure.

[0197] 105. Based on the defect categories corresponding to the target defect areas, the target image is labeled with defect categories.

[0198] In an embodiment of the present application, the target defect areas determined from the target image and the defect categories corresponding to each target defect area can be saved. For example, they can be stored in a json file to facilitate the subsequent reading of the target defect areas and defect categories from the json file for defect labeling processing of the image of the defective object.

[0199] In some embodiments, when the defect categories are saved, a defect name may be given to each defect category to obtain a defect name corresponding to each defect category.

[0200] In some embodiments, labeling an image of a defect object based on a target defect area and a defect category may include: determining an area from the image that matches the target defect area, and then labeling the defect category to which the area in the image belongs based on the defect category to which the target defect area belongs. In this way, the workload of manual labeling can be reduced and labeling efficiency can be improved.

[0201] The embodiment discloses a defect labeling method, including: obtaining a target image of a target object; determining a regional image of at least one candidate defect region of the target object from the target image; performing defect region recognition processing based on each regional image to obtain a target defect region in each regional image; determining a defect category corresponding to each target defect region based on the similarity between each target defect region; and labeling the target image with a defect category based on the defect category corresponding to each target defect region. In this way, various defects can be effectively extracted and classified adaptively, thereby improving the accuracy and efficiency of defect labeling.

[0202] In order to better implement the defect labeling method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above defect labeling method. The meanings of the terms are the same as those in the above defect labeling method, and the specific implementation details can refer to the description in the method embodiment.

[0203] See also Figure 2 , Figure 2 : is a schematic diagram of the structure of an embodiment of a defect marking device provided in an embodiment of the present application, wherein the defect marking device includes:

[0204] A first acquisition unit 301 is used to acquire a target image of a target object;

[0205] A first determining unit 302 is used to determine a regional image of at least one candidate defect region of the target object from the target image;

[0206] The identification unit 303 is used to perform defect area identification processing based on each area image to obtain a target defect area in each area image;

[0207] A second determining unit 304 is used to determine the defect category corresponding to each target defect area based on the similarity between each target defect area;

[0208] The third determining unit 305 is configured to label the target image with a defect category based on the defect category corresponding to each target defect area.

[0209] In some embodiments, the first determining unit 302 may include:

[0210] A division subunit, used for extracting a plurality of candidate regions from the target image;

[0211] A first determination subunit, used to determine a first similarity between the candidate regions;

[0212] a merging subunit, configured to merge the plurality of candidate regions based on the first similarity to obtain the candidate defect region;

[0213] The second determining subunit is configured to determine the region image based on the candidate defect region.

[0214] In some embodiments, the second determining subunit may be specifically used for:

[0215] Obtaining the region coordinates corresponding to the candidate defect region in the target image;

[0216] The target image is cropped based on the region coordinates to obtain the region image.

[0217] In some embodiments, the identification unit 303 may include:

[0218] An extraction subunit, used for performing feature extraction on the regional image to obtain image features corresponding to the regional image;

[0219] A calculation subunit, used for calculating the defect probability corresponding to each position in the regional image based on the image features;

[0220] The third determining subunit is configured to determine the target defect region from the region image based on the defect probability.

[0221] In some embodiments, the third determining subunit may be specifically used for:

[0222] Based on the defect probability of each position in the regional image, determining a probability map corresponding to the regional image, wherein the pixel points in the probability map correspond one-to-one to the positions in the regional image;

[0223] Compare the defect probability corresponding to each pixel in the probability map with a first preset threshold;

[0224] Determine a target pixel point whose defect probability is not less than the first preset threshold;

[0225] Based on the first region where the target pixel point in the probability map is located, a region corresponding to the first region is determined from the regional image to obtain the target defect region.

[0226] In some embodiments, the third determining subunit may be specifically used for:

[0227] Based on the defect probability of each position in the regional image, determining a probability map corresponding to the regional image, wherein the pixel points in the probability map correspond one-to-one to the positions in the regional image;

[0228] Compare the defect probability corresponding to each pixel in the probability map with a first preset threshold;

[0229] Determine a target pixel point whose defect probability is not less than the first preset threshold;

[0230] generating a binary mask image based on the first region;

[0231] A connected region is extracted from the binary mask image, and the connected region is mapped to the region image to obtain the target defect region.

[0232] In some embodiments, the apparatus may further include:

[0233] A second acquisition unit, used for acquiring position information of the regional image in the target image;

[0234] The extraction unit is used to extract the position feature corresponding to the position information, and fuse the position feature with the image feature to obtain a fused feature.

[0235] In some embodiments, the computing subunit may be specifically used for:

[0236] Based on the fused features, the defect probability corresponding to each position in the regional image is calculated.

[0237] In some embodiments, the third determining unit 305 may include:

[0238] An acquisition subunit, used for acquiring a second similarity between each target defect area;

[0239] The clustering subunit is used to perform clustering processing on each target defect area based on the second similarity to obtain multiple clusters, set a defect category for each cluster, and determine the defect category corresponding to the target defect area in the cluster.

[0240] In some embodiments, the clustering subunit may be specifically used to:

[0241] Selecting m target defect regions from the target defect regions as cluster centers of m initial clustering clusters;

[0242] Based on the similarity between the remaining target defect regions and the cluster center, the remaining target defect regions are clustered to obtain a plurality of cluster clusters.

[0243] In some embodiments, the clustering subunit may be specifically used to:

[0244] Selecting m target defect regions from the target defect regions as cluster centers of m initial clustering clusters;

[0245] If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is not less than a second preset threshold, classifying the remaining target defect regions into the initial cluster corresponding to the cluster center;

[0246] If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is less than the second preset threshold, classifying the remaining target defect regions into a new cluster;

[0247] Based on the initial clustering cluster and the new clustering cluster, a plurality of clustering clusters are obtained.

[0248] In some embodiments, the acquisition subunit may be specifically used for:

[0249] Obtaining the similarity of each target defect area in at least one dimension;

[0250] The second similarity is determined based on the similarity in at least one of the dimensions.

[0251] In some embodiments, the first acquisition unit 301 may include:

[0252] An acquisition subunit, used for acquiring multiple images of the target object in multiple modalities;

[0253] The fusion subunit is used to fuse the multiple images to obtain the target image.

[0254] The embodiment of the present application discloses a defect labeling device, including: a first acquisition unit 301 acquires a target image of a target object; a first determination unit 302 determines a regional image of at least one candidate defect region of the target object from the target image; an identification unit 303 performs defect region identification processing based on each regional image to obtain a target defect region in each regional image; a second determination unit 304 determines the defect category corresponding to each target defect region based on the similarity between each target defect region; a third determination unit 305 labels the target image with a defect category based on the defect category corresponding to each target defect region. In this way, various defects can be effectively extracted and classified adaptively, thereby improving the accuracy and efficiency of defect labeling.

[0255] The present application also provides a server, such as Figure 3 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present application, specifically:

[0256] The server may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 3 The server structure shown in the figure does not constitute a limitation on the server, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0257] in:

[0258] The processor 801 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. By running or executing software programs and / or modules stored in the memory 802, and calling data stored in the memory 802, the processor 801 performs various functions of the server and processes data, thereby monitoring the server as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operation storage medium, user interface and application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0259] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating storage medium, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0260] The server also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management storage medium, so as to manage charging, discharging, and power consumption through the power management storage medium. The power supply 803 can also include any components such as one or more DC or AC power supplies, recharge storage media, power failure detection circuits, power converters or inverters, and power status indicators.

[0261] The server may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0262] Although not shown, the server may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0263] Acquire a target image of a target object; determine a regional image of at least one candidate defect region of the target object from the target image; perform defect region recognition processing based on each regional image to obtain a target defect region in each regional image; determine a defect category corresponding to each target defect region based on a similarity between each target defect region; and label the target image with a defect category based on the defect category corresponding to each target defect region.

[0264] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0265] To this end, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the steps in any defect labeling method provided in the embodiment of the present application. For example, the computer program can be loaded by a processor to execute the following steps:

[0266] Acquire a target image of a target object;

[0267] Determining, from the target image, a regional image of at least one candidate defect region of the target object;

[0268] Perform defect area recognition processing based on each area image to obtain a target defect area in each area image;

[0269] Determine the defect category corresponding to each target defect area based on the similarity between each target defect area;

[0270] Based on the defect categories corresponding to each target defect area, the target image is labeled with defect categories.

[0271] The embodiment of the present application obtains a target image of a target object; determines a regional image of at least one candidate defect area of ​​the target object from the target image; performs defect area recognition processing based on each regional image to obtain a target defect area in each regional image; determines a defect category corresponding to each target defect area based on a similarity between each target defect area; and labels the target image with a defect category based on the defect category corresponding to each target defect area. In this way, various defects can be effectively and adaptively extracted and classified, thereby improving the accuracy and efficiency of defect labeling.

[0272] The specific implementation of the above operations / steps can be found in the previous embodiments and will not be repeated here.

[0273] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0274] Since the computer program stored in the computer-readable storage medium can execute the steps in any defect labeling method provided in the embodiments of the present application, the beneficial effects that can be achieved by any defect labeling method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0275] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of an electronic device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the electronic device executes the method provided in various optional implementations of the above embodiments.

[0276] The above is a detailed introduction to a defect marking method, device, server and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A defect marking method, characterized in that: The method comprises: Acquire a target image of a target object; Determining, from the target image, a regional image of at least one candidate defect region of the target object; Perform defect area recognition processing based on each area image to obtain a target defect area in each area image; Determine the defect category corresponding to each target defect area based on the similarity between each target defect area; Based on the defect categories corresponding to the target defect areas, the target image is labeled with defect categories.

2. The method according to claim 1, characterized in that Determining, from the target image, a regional image of at least one candidate defect region of the target object, comprises: Extracting multiple candidate regions from the target image; Determine a first similarity between each candidate region; Merging the plurality of candidate regions based on the first similarity to obtain the candidate defect region; The region image is determined based on the candidate defect regions.

3. The method according to claim 2, characterized in that The determining the regional image based on the candidate defect region comprises: Obtaining the region coordinates corresponding to the candidate defect region in the target image; The target image is cropped based on the region coordinates to obtain the region image.

4. The method according to claim 1, characterized in that: The defect area recognition process is performed based on each area image to obtain the target defect area in each area image, including: Performing feature extraction on the regional image to obtain image features corresponding to the regional image; Based on the image features, calculating the defect probability corresponding to each position in the regional image; The target defect region is determined from the region image based on the defect probability.

5. The method according to claim 4, characterized in that The determining the target defect area from the area image based on the defect probability comprises: Based on the defect probability of each position in the regional image, determining a probability map corresponding to the regional image, wherein the pixel points in the probability map correspond one-to-one to the positions in the regional image; Compare the defect probability corresponding to each pixel in the probability map with a first preset threshold; Determine a target pixel point whose defect probability is not less than the first preset threshold; Based on the first region where the target pixel point in the probability map is located, a region corresponding to the first region is determined from the regional image to obtain the target defect region.

6. The method according to claim 5, characterized in that The step of determining, based on the first region where the target pixel point in the probability map is located, a region corresponding to the first region from the regional image to obtain the target defect region comprises: generating a binary mask image based on the first region; A connected region is extracted from the binary mask image, and the connected region is mapped to the region image to obtain the target defect region.

7. The method according to claim 4, characterized in that The method further comprises: Acquire location information of the regional image in the target image; Extracting a position feature corresponding to the position information, and fusing the position feature with the image feature to obtain a fused feature; The calculating the defect probability corresponding to each position in the regional image based on the image feature includes: Based on the fused features, the defect probability corresponding to each position in the regional image is calculated.

8. The method according to claim 1, characterized in that The step of determining the defect category corresponding to each target defect area based on the similarity between the target defect areas includes: Obtaining a second similarity between each target defect area; Clustering is performed on each target defect region based on the second similarity to obtain a plurality of clusters, and a defect category is set for each cluster to determine the defect category corresponding to the target defect region in the cluster.

9. The method according to claim 8, characterized in that The clustering process is performed on each target defect area based on the second similarity to obtain a plurality of clusters, including: Selecting m target defect regions from the target defect regions as cluster centers of m initial clustering clusters; Based on the similarity between the remaining target defect regions and the cluster center, the remaining target defect regions are clustered to obtain a plurality of cluster clusters.

10. The method according to claim 9, characterized in that Based on the similarity between the remaining target defect regions and the cluster center, the remaining target defect regions are clustered to obtain a plurality of cluster clusters, including: If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is not less than a second preset threshold, classifying the remaining target defect regions into the initial cluster corresponding to the cluster center; If the second similarity between the remaining target defect regions and the target defect region corresponding to the cluster center is less than the second preset threshold, classifying the remaining target defect regions into a new cluster; Based on the initial clustering cluster and the new clustering cluster, a plurality of clustering clusters are obtained.

11. The method according to claim 8, characterized in that The obtaining of the second similarity between the target defect regions includes: Obtaining the similarity of each target defect area in at least one dimension; The second similarity is determined based on the similarity in at least one of the dimensions.

12. The method according to claim 1, characterized in that The step of acquiring a target image of a target object comprises: Acquiring multiple images of the target object in multiple modalities; The multiple images are fused to obtain the target image.

13. A defect marking device, characterized in that: The device comprises: A first acquisition unit, used to acquire a target image of a target object; A first determining unit, configured to determine a region image of at least one candidate defect region of the target object from the target image; An identification unit, used to perform defect area identification processing based on each area image to obtain a target defect area in each area image; A second determination unit, configured to determine a defect category corresponding to each target defect region based on the similarity between the target defect regions; The third determining unit is used to label the target image with a defect category based on the defect category corresponding to each target defect area.

14. A server, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Acquire a target image of a target object; Determining, from the target image, a regional image of at least one candidate defect region of the target object; Perform defect area recognition processing based on each area image to obtain a target defect area in each area image; Determine the defect category corresponding to each target defect area based on the similarity between each target defect area; Based on the defect categories corresponding to the target defect areas, the target image is labeled with defect categories.

15. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the method according to any one of claims 1 to 12.

Images