Defect detection method, device, equipment, storage medium and product

Abstract

The application discloses a defect detection method, device, equipment, storage medium and product, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a to-be-detected image and a reference image used for training of a defect detection model, wherein the pixel value distribution of the to-be-detected image is different from that of the reference image; performing clustering processing on pixel points in the to-be-detected image and the reference image respectively, thereby obtaining a plurality of source pixel sets of the to-be-detected image and a plurality of reference pixel sets of the reference image; determining reference pixel sets matched with the source pixel sets; performing histogram matching on the source pixel sets of the to-be-detected image and the reference pixel sets matched with the source pixel sets, so that the pixel value distribution of the to-be-detected image is the same as that of the reference image; and performing defect detection on a detection target in the to-be-detected image subjected to the histogram matching through the defect detection model, thereby obtaining a defect detection result. The method can improve defect detection precision.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a defect detection method, apparatus, device, storage medium, and product. Background Technology

[0002] With the development of Industry 4.0 and intelligent manufacturing, automated production lines are placing increasingly higher demands on product quality. Traditional quality control methods, such as manual inspection, are not only inefficient and costly, but also susceptible to human error, leading to significant inaccuracies. Therefore, automated visual inspection technology is gradually becoming a key means to improve production efficiency and ensure product quality.

[0003] In related technologies, defect detection is generally performed using a defect detection model based on an image of the target object. However, due to variations in lighting and inconsistent shooting angles, the captured image differs from the reference image used during the training of the defect detection model, resulting in low detection accuracy. Therefore, there is an urgent need to provide a defect detection method to improve its accuracy.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a defect detection method, apparatus, equipment, storage medium, and product that can improve the accuracy of defect detection.

[0006] In a first aspect, this application provides a defect detection method, performed by a defect detection device, the method comprising the following steps:

[0007] Obtain an image to be detected and a reference image, wherein the pixel value distributions of the image to be detected and the reference image are different;

[0008] Clustering is performed on the pixels in the image to be detected and the reference image respectively to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image;

[0009] Determine the reference pixel set that matches each of the source pixel sets;

[0010] Histogram matching is performed on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same;

[0011] The defect detection model is used to detect defects in the target in the image to be detected after histogram matching, and the defect detection result is obtained.

[0012] The defect detection scheme provided in this application takes into account the differences between the acquired image to be detected and the reference image due to differences in the shooting environment. These differences are mainly reflected in the different pixel value distributions of the image to be detected and the reference image. Therefore, to reduce the differences between the image to be detected and the reference image, a histogram matching method is used to adjust the pixel value distribution of the image to be detected, so that the pixel value distributions of the image to be detected and the reference image are the same. To avoid reducing the relative differences in pixel values ​​in the image to be detected through global matching and weakening or hiding the original defects, this scheme adopts a region-based histogram matching strategy. That is, the pixels in the image to be detected and the reference image are clustered separately to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image. Since the pixels in each pixel set have similar visual characteristics, the reference pixel sets that match each source pixel set are subsequently determined. Histogram matching of each source pixel set of the image to be detected and the reference pixel sets that match the source pixel sets is actually finding corresponding regions in the reference image that have similar visual characteristics to the image to be detected, and adjusting the pixel value distribution of the corresponding regions in the image to be detected based on the pixel value distribution of each region in the reference image. Therefore, the pixel value distribution of each region is adjusted according to its own visual characteristics. This scheme, by adjusting the pixel value distribution of the image to be detected to match that of the reference image, not only ensures that the texture structure of the image to be detected remains unchanged, but also avoids drastically affecting the overall contrast of the image, thus helping to maintain the visibility of defects in the image to be detected. Therefore, this scheme can reduce the difference between the image to be detected and the reference image without weakening the defects in the image to be detected. Thus, by using a defect detection model to detect defects in the target in the image to be detected after histogram matching, the accuracy of defect detection can be improved.

[0013] In some embodiments, the step of clustering the pixels in the image to be detected and the reference image to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image includes:

[0014] Using a target clustering model, mean-shift clustering is performed on the image to be detected and the reference image respectively to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0015] In this embodiment, since mean-shift clustering considers not only pixel values ​​but also pixel positions during clustering, using a target clustering model to perform mean-shift clustering on the image can accurately group pixels with similar features into the same set. This ensures that each source pixel set and reference pixel set can more precisely represent a specific part of the image. Furthermore, using a target clustering model for clustering significantly improves the efficiency of clustering, accelerating the image processing and providing usable images for subsequent defect detection procedures.

[0016] In some embodiments, the training process of the target clustering model includes:

[0017] Obtain sample images and sample clustering results, wherein the sample clustering results are obtained by performing mean-shift clustering on the sample images;

[0018] The feature extraction network in the target clustering model is used to extract features from the sample image to obtain sample image features;

[0019] The clustering network in the target clustering model is used to perform clustering processing on the features of the sample image to obtain the predicted clustering results;

[0020] The target clustering model is trained based on the predicted clustering results and the sample clustering results.

[0021] In this embodiment, the target clustering model is trained using the sample clustering results obtained by mean-shift clustering of sample images. This allows the target clustering model to better understand the relationship between different features in the image and the clustering results, thereby further optimizing the feature extraction process and the feature-based clustering process, so that the predicted clustering results are the same as the sample clustering results. This allows for the training of a target clustering model that can accurately perform mean-shift clustering on images.

[0022] In some embodiments, the clustering network includes multiple upsampling layers and an output layer;

[0023] The step of clustering the sample image features through the clustering network in the target clustering model to obtain the predicted clustering results includes:

[0024] Through multiple upsampling layers in the clustering network, upsampling processing is performed sequentially based on the sample image features to obtain the image features output by the last upsampling layer;

[0025] The predicted clustering result is obtained by performing feature transformation on the image features output by the last upsampling layer through the output layer of the clustering network.

[0026] In this embodiment, stacking multiple upsampling layers allows for the gradual refinement and adjustment of sample image features, enhancing their representational power and making them richer and more representative. This helps the clustering network better identify and distinguish different cluster objects, improving the accuracy and reliability of clustering.

[0027] In some embodiments, the feature extraction network includes multiple downsampling layers, which are used to extract image features at different levels;

[0028] The process of sequentially upsampling the sample image features through multiple upsampling layers in the clustering network to obtain the image features output by the last upsampling layer includes:

[0029] For any upsampling layer in the clustering network, the input image features are upsampled through the upsampling layer, and the upsampled image features are fused with the image features of the corresponding layer in the feature extraction network to obtain the image features output by the upsampling layer.

[0030] In this embodiment, the image features extracted by different downsampling layers of the feature extraction network contain information at different scales. Shallower image features retain more detailed image information, while deeper image features contain more abstract semantic information. During the upsampling process, each upsampling layer combines image features from different levels of the feature extraction network, fusing the upsampled image features with the corresponding level of image features in the feature extraction network. This fully utilizes the multi-scale information of the image, making the clustering results more accurate and detailed.

[0031] In some embodiments, training the target clustering model based on the predicted clustering results and the sample clustering results includes:

[0032] Based on the predicted clustering results and the sample clustering results, a first clustering loss value is determined;

[0033] Multiple second clustering loss values ​​are determined based on the image features output by the multiple upsampling layers and the downsampling features of the corresponding layers of the predicted clustering results;

[0034] The target clustering model is trained based on the first clustering loss value and the plurality of second clustering loss values.

[0035] In this embodiment, by simultaneously considering a first clustering loss value based on both the predicted clustering result and the sample clustering result, and multiple second clustering loss values ​​based on the upsampling layer output features and the downsampling features corresponding to the predicted clustering result, the difference between the model's predicted result and the actual situation can be comprehensively measured from different perspectives. The first clustering loss value directly reflects the deviation between the final predicted clustering result and the actual sample clustering result, while the multiple second clustering loss values ​​focus on the consistency between the changes in features during the intermediate upsampling process and the final clustering result. This multi-dimensional loss measurement method enables more comprehensive optimization of model training, avoiding the problem of focusing only on the final result while ignoring the intermediate process, thereby improving the overall performance of the model.

[0036] In some embodiments, the training process of the target clustering model further includes:

[0037] The image reconstruction network in the target clustering model is used to reconstruct the features of the sample image to obtain a reconstructed image;

[0038] The target clustering model is trained based on the reconstructed image and the sample image.

[0039] In this embodiment, the process of the image reconstruction network reconstructing the features of the sample image prompts the model to delve deeper into the image's intrinsic features. If the reconstructed image is very similar to the sample image, it indicates that the image features extracted by the model accurately reflect the information of the original image, demonstrating strong feature representation capabilities. Conversely, if the reconstructed image differs significantly from the sample image, the model will adjust its parameters through training to improve feature extraction and representation methods, thereby enhancing the quality of the reconstructed image. This feedback mechanism helps the model continuously optimize feature representation, making the extracted features more beneficial for subsequent clustering tasks.

[0040] In some embodiments, the training process of the target clustering model further includes:

[0041] The image judgment network in the target clustering model is used to judge the sample image and the reconstructed image respectively, and the judgment result is obtained. The judgment result indicates whether the currently judged image is a sample image or a reconstructed image.

[0042] Based on the judgment result, the target clustering model is trained.

[0043] In this embodiment, the process by which the image judgment network judges the sample image and the reconstructed image helps the model capture more detailed and discriminative feature information in the image, such as subtle changes in texture and subtle differences in color. In this way, the model can mine deeper features in the image, improve its understanding and learning ability of image features, and thus improve the accuracy of clustering based on image features.

[0044] In some embodiments, determining the reference pixel set that matches each of the source pixel sets includes:

[0045] Determine the position coordinates of the center pixel in each source pixel set and each reference pixel set in the image coordinate system, and the center pixel in each pixel set is the cluster center of multiple pixels in each pixel set;

[0046] The source pixel set whose position coordinates are closest to the center pixel point and the reference pixel set are matched to obtain the reference pixel set that matches each of the source pixel sets.

[0047] In this embodiment, the position coordinates of the center pixels within the source and reference pixel sets in the image coordinate system are used as the matching basis. This eliminates the need for complex calculations, making the entire matching process easy to understand and implement, and improving matching efficiency. This is highly advantageous for defect detection scenarios with high real-time requirements, saving significant time and computational resources.

[0048] In some embodiments, before performing histogram matching on each source pixel set of the image to be detected and the reference pixel set matching the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same, the method further includes:

[0049] The pixel values ​​of multiple pixels within the same pixel set are normalized.

[0050] In this embodiment, normalization is performed before histogram matching to prevent excessive changes to the internal relative structure of the image to be detected during the matching process, thus avoiding limitations on the image's usability in various subsequent application scenarios. For example, in image analysis in industrial production, defect detection is subsequently required on the image to be detected. The stability of the internal relative structure of the image is fundamental to ensuring the accuracy and effectiveness of defect detection. Therefore, normalization can constrain pixel value variations to a certain extent, maintaining the internal relative structure of the image to be detected and ensuring that the image meets the requirements for defect detection in subsequent use.

[0051] In some embodiments, performing histogram matching on each source pixel set of the image to be detected and the reference pixel set matching the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same, includes:

[0052] For any source pixel set, the cumulative distribution function of the source pixel set is determined based on the pixel values ​​of the pixels within the source pixel set;

[0053] Based on the pixel values ​​of the pixels in the reference pixel set matched by the source pixel set, the cumulative distribution function of the reference pixel set is determined;

[0054] The pixel values ​​of multiple pixels within the source pixel set are adjusted so that the cumulative distribution function of the adjusted source pixel set is the same as the cumulative distribution function of the reference pixel set.

[0055] In this embodiment, by making the cumulative distribution functions of the source pixel set and the reference pixel set identical, the pixel value distribution of the image to be detected can be effectively adjusted to be consistent with the reference image. This eliminates interference from the difference in pixel value distribution between the image to be detected and the reference image. Furthermore, since the cumulative distribution function primarily reflects the statistical characteristics of pixel values, rather than the specific location and arrangement of pixels, adjusting the pixel values ​​of pixels within the source pixel set to make its cumulative distribution function identical to the reference pixel set does not significantly damage the structure of the image to be detected, thus facilitating defect detection based on the structure of the image to be detected.

[0056] In some embodiments, acquiring the image to be detected and the reference image used for training the defect detection model includes:

[0057] The target to be detected is captured by a camera to obtain the image to be detected;

[0058] The reference image is determined by any training image used by the defect detection model. The training image is obtained by taking pictures of other targets of the same type as the current detection target, and the training image is compressed.

[0059] In this embodiment, since the defect detection model is trained based on training images, when the pixel value distribution of the image to be detected is the same as that of the training image, the model can process the image to be detected more stably, reducing false positives and false negatives caused by differences in image features. Therefore, using any training image used by the defect detection model as a reference image and matching the pixel value distribution of the image to be detected and the reference image through histogram matching helps to improve the stability of the defect detection model and make the defect detection results more reliable.

[0060] Secondly, to achieve the above objectives, this application also proposes a defect detection device, the device comprising:

[0061] The image acquisition module is used to acquire the image to be detected and the reference image used for training the defect detection model, wherein the pixel value distribution of the image to be detected and the reference image are different;

[0062] The image clustering module is used to perform clustering processing on the pixels in the image to be detected and the reference image respectively, to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0063] An image matching module is used to determine the reference pixel set that matches each of the source pixel sets;

[0064] The distribution adjustment module is used to perform histogram matching on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same.

[0065] The defect detection module is used to perform defect detection on the target in the image to be detected after histogram matching using the defect detection model, and obtain the defect detection result.

[0066] In some embodiments, the image clustering module is used to perform mean-shift clustering on the image to be detected and the reference image respectively using a target clustering model to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0067] In some embodiments, the training process of the target clustering model includes:

[0068] Obtain sample images and sample clustering results, wherein the sample clustering results are obtained by performing mean-shift clustering on the sample images;

[0069] The feature extraction network in the target clustering model is used to extract features from the sample image to obtain sample image features;

[0070] The clustering network in the target clustering model is used to perform clustering processing on the features of the sample image to obtain the predicted clustering results;

[0071] The target clustering model is trained based on the predicted clustering results and the sample clustering results.

[0072] In some embodiments, the clustering network includes multiple upsampling layers and an output layer;

[0073] The step of clustering the sample image features through the clustering network in the target clustering model to obtain the predicted clustering results includes:

[0074] Through multiple upsampling layers in the clustering network, upsampling processing is performed sequentially based on the sample image features to obtain the image features output by the last upsampling layer;

[0075] The predicted clustering result is obtained by performing feature transformation on the image features output by the last upsampling layer through the output layer of the clustering network.

[0076] In some embodiments, the feature extraction network includes multiple downsampling layers, which are used to extract image features at different levels;

[0077] The process of sequentially upsampling the sample image features through multiple upsampling layers in the clustering network to obtain the image features output by the last upsampling layer includes:

[0078] For any upsampling layer in the clustering network, the input image features are upsampled through the upsampling layer, and the upsampled image features are fused with the image features of the corresponding layer in the feature extraction network to obtain the image features output by the upsampling layer.

[0079] In some embodiments, training the target clustering model based on the predicted clustering results and the sample clustering results includes:

[0080] Based on the predicted clustering results and the sample clustering results, a first clustering loss value is determined;

[0081] Multiple second clustering loss values ​​are determined based on the image features output by the multiple upsampling layers and the downsampling features of the corresponding layers of the predicted clustering results;

[0082] The target clustering model is trained based on the first clustering loss value and the plurality of second clustering loss values.

[0083] In some embodiments, the training process of the target clustering model further includes:

[0084] The image reconstruction network in the target clustering model is used to reconstruct the features of the sample image to obtain a reconstructed image;

[0085] The target clustering model is trained based on the reconstructed image and the sample image.

[0086] In some embodiments, the training process of the target clustering model further includes:

[0087] The image judgment network in the target clustering model is used to judge the sample image and the reconstructed image respectively, and the judgment result is obtained. The judgment result indicates whether the currently judged image is a sample image or a reconstructed image.

[0088] Based on the judgment result, the target clustering model is trained.

[0089] In some embodiments, the image matching module is used to determine the position coordinates of the center pixel in each source pixel set and each reference pixel set in the image coordinate system, wherein the center pixel in each pixel set is the cluster center of multiple pixels in each pixel set; and to match the source pixel set and the reference pixel set whose position coordinates are closest to the center pixel to obtain the reference pixel set that matches each source pixel set.

[0090] In some embodiments, the distribution adjustment module is further configured to normalize the pixel values ​​of multiple pixels within the same pixel set before performing histogram matching on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same.

[0091] In some embodiments, the distribution adjustment module is configured to, for any source pixel set, determine a cumulative distribution function of the source pixel set based on the pixel values ​​of the pixels within the source pixel set; determine a cumulative distribution function of the reference pixel set based on the pixel values ​​of the pixels within the reference pixel set that match the source pixel set; and adjust the pixel values ​​of multiple pixels within the source pixel set so that the adjusted cumulative distribution function of the source pixel set is the same as the cumulative distribution function of the reference pixel set.

[0092] In some embodiments, the image acquisition module is used to capture the current detection target with a camera to obtain the image to be detected; and to determine any training image used by the defect detection model as the reference image, wherein the training image is obtained by capturing other targets of the same type as the current detection target, and the training image is compressed.

[0093] Thirdly, to achieve the above objectives, this application also proposes a defect detection device, the device comprising: a memory, a processor, and a defect detection program stored in the memory and executable on the processor, the defect detection program being configured to implement the steps of the defect detection method described above.

[0094] Fourthly, to achieve the above objectives, this application also proposes a storage medium storing a defect detection program, which, when executed by a processor, implements the steps of the defect detection method described above.

[0095] Fifthly, to achieve the above objectives, this application also proposes a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the defect detection method described above.

[0096] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0097] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0098] Figure 1 A flowchart illustrating a defect detection method provided in some embodiments of this application;

[0099] Figure 2 A schematic diagram illustrating the process of a target clustering model processing sample images, provided in some embodiments of this application;

[0100] Figure 3 A schematic diagram illustrating a histogram matching process provided for some embodiments of this application;

[0101] Figure 4 A schematic diagram illustrating a defect detection process provided for some embodiments of this application;

[0102] Figure 5 A schematic diagram of the module structure of a defect detection device provided in some embodiments of this application;

[0103] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the defect detection method in this application embodiment.

[0104] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0105] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0106] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0107] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0108] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0109] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0110] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0111] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0112] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0113] With the development of Industry 4.0 and intelligent manufacturing, automated production lines are placing increasingly higher demands on product quality. Traditional quality control methods, such as manual inspection, are not only inefficient and costly, but also susceptible to human error, leading to significant inaccuracies. Therefore, automated visual inspection technology has gradually become a key means to improve production efficiency and ensure product quality. However, in the context of large-scale production, issues such as changes in lighting and inconsistent shooting angles significantly interfere with automated visual inspection. Taking visual defect detection as an example, when different production lines need to use the same defect detection model, the images to be inspected must belong to the same domain, i.e., have the same pixel value distribution, and the pixel value distribution of the images to be inspected must be the same as that of the training images used during model training. However, due to differences in shooting environments, this condition cannot be met in reality, resulting in low defect detection accuracy.

[0114] The scheme in this application uses the training image of the defect detection model as a reference image and employs histogram matching to adjust the pixel value distribution of the image to be detected, so that the pixel value distribution of the image to be detected is the same as that of the reference image. Furthermore, to avoid global matching reducing the relative differences in pixel values ​​in the image to be detected and weakening or hiding the defects originally present in the image to be detected, this scheme adopts a region-based histogram matching strategy. That is, the pixels in the image to be detected and the reference image are clustered separately to obtain multiple source pixel sets in the image to be detected and multiple reference pixel sets in the reference image. Reference pixel sets that match each source pixel set are determined, and histogram matching is performed on each source pixel set of the image to be detected and the reference pixel sets that match the source pixel sets. In other words, corresponding regions in the reference image with similar visual characteristics to the image to be detected are found, and the pixel value distribution of the corresponding regions in the image to be detected is adjusted based on the pixel value distribution of each region in the reference image. This approach, by adjusting the pixel value distribution of the image to be detected to be consistent with that of the reference image, not only ensures that the texture structure of the image to be detected remains unchanged, but also avoids drastic impact on the overall contrast of the image to be detected. This helps maintain the visibility of defects in the image to be detected and ensures the accuracy of defect detection based on the image to be detected.

[0115] Because the above-mentioned defect detection scheme has high processing efficiency, low resource consumption, and does not require training or the use of large image transformation models such as CycleGAN (Cycle Generative Adversarial Network), it is very suitable for scenarios involving defect detection of industrial images.

[0116] Please refer to Figure 1 , Figure 1 This application provides a flowchart illustrating a defect detection method according to some embodiments, executed by a defect detection device. The method includes the following steps:

[0117] S10: Obtain the image to be detected and the reference image used for training the defect detection model. The pixel value distribution of the image to be detected and the reference image are different.

[0118] S20, perform clustering processing on the pixels in the image to be detected and the reference image respectively to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0119] S30, determine the reference pixel set that matches each source pixel set.

[0120] S40, perform histogram matching on the source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same.

[0121] S50 uses a defect detection model to detect defects in the target in the image to be detected after histogram matching, and obtains the defect detection result.

[0122] The image to be detected refers to the original image that needs adjustment; it is the actual image data that needs to be analyzed and processed, and it is the initial input of the entire defect detection process. For example, the image to be detected is an image captured by a camera of the target being detected, reflecting the actual visual information of the target. The reference image is an image that provides the distribution of the target's pixel values; the image to be detected will attempt to mimic its statistical characteristics. The reference image is the training image used by the defect detection model. For example, the reference image is an image captured of another target of the same type as the target being detected, and it has been compressed. The reference image serves as a standard or reference, providing a basis for processing the image to be detected.

[0123] Pixel value distribution refers to the frequency and distribution of different pixel values ​​in an image. For example, in a color image, pixel value distribution can represent the proportion of pixels with different values ​​in the image. If the pixel value distribution of the image to be detected differs from that of the reference image, it means that they differ in the statistical characteristics of their pixel values. This could be due to factors such as differences in the shooting environment, equipment, or variations in the target itself.

[0124] Clustering is a process of grouping similar pixels into the same pixel set based on their position and value in an image. In this scheme, clustering groups pixels in the image to be detected and a reference image into multiple source pixel sets and reference pixel sets. Clustering is based on the positional relationships and the similarity of pixel values ​​between pixels. For example, each source pixel set and each reference pixel set obtained through clustering contains a center pixel and surrounding pixels. The similarity of pixel values ​​between a surrounding pixel and the center pixel within the same pixel set is greater than the similarity of pixel values ​​between a surrounding pixel and the center pixels in other pixel sets, and / or, the distance between a surrounding pixel and the center pixel within the same pixel set is less than the distance between a surrounding pixel and the center pixels in other pixel sets. This clustering method groups pixels with similar characteristics into one category, for example, grouping pixels with similar pixel values ​​and / or close positions together, allowing for more detailed analysis and processing of the image subsequently.

[0125] The source pixel set is a collection of pixels obtained after clustering the image to be detected. Each source pixel set contains a center pixel and surrounding pixels that are similar in position and pixel value, and are grouped together using a clustering algorithm. The source pixel set is the basic unit for analyzing and processing the image to be detected; subsequent operations will revolve around these sets. The reference pixel set is a collection of pixels obtained after clustering a reference image. Similar to the source pixel set, each reference pixel set also contains a center pixel and surrounding pixels. These pixels are similar in position and pixel value, and are grouped together using a clustering algorithm. The reference pixel set is used in subsequent processing to match with the source pixel set, enabling comparison and analysis between the image to be detected and the reference image.

[0126] The center pixel is the pixel selected to represent the characteristics of a pixel set; it serves as the cluster center for multiple pixels within that set. Surrounding pixels have specific relationships with the center pixel, such as high pixel value similarity or proximity, causing them to be grouped into the same set. The center pixel can serve as a key feature point for that set, describing and representing some characteristics of the entire pixel set. Surrounding pixels are pixels that have a specific relationship with the center pixel within the same pixel set. These pixels, surrounding the center pixel, collectively constitute the pixel set. It can be understood that the similarity or proximity of pixel values ​​between surrounding pixels and the center pixel determines whether they belong to the same pixel set.

[0127] Pixel similarity is a metric used to measure the degree of similarity between the pixel values ​​of two pixels. It is determined by calculating a certain distance or difference between pixel values, such as Euclidean distance or Manhattan distance. The distance between pixels refers to the spatial distance between pixels in the image coordinate system.

[0128] Determining the reference pixel set to match each source pixel set means finding the set of reference pixels that is most similar in features to each source pixel set using certain algorithms and rules. The matching criteria can include the position of the center pixel, the statistical characteristics of the pixel value set, etc. Through matching, a correspondence is established between the source pixel set and the reference pixel set, providing a foundation for subsequent histogram matching and image analysis.

[0129] Histogram matching is an image processing technique that adjusts the pixel values ​​of each pixel set in the image to be detected so that its cumulative distribution function is the same as that of the reference pixel set, thereby achieving consistency in the pixel value distribution between the image to be detected and the reference image. Histogram matching can eliminate differences in pixel value distribution between the image to be detected and the reference image caused by factors such as shooting conditions, equipment differences, and image compression.

[0130] A defect detection model is a trained model used to identify defects in objects within an image. It uses training images as its learning foundation. When processing an image, the defect detection model employs its internal algorithms and rules to determine whether a defect exists in the target object, as well as the type and location of the defect, ultimately outputting the defect detection result.

[0131] The defect detection scheme provided in this application takes into account the differences between the acquired image to be detected and the reference image due to differences in the shooting environment. These differences are mainly reflected in the different pixel value distributions of the image to be detected and the reference image. Therefore, to reduce the differences between the image to be detected and the reference image, a histogram matching method is used to adjust the pixel value distribution of the image to be detected, so that the pixel value distributions of the image to be detected and the reference image are the same. To avoid reducing the relative differences in pixel values ​​in the image to be detected through global matching and weakening or hiding the original defects, this scheme adopts a region-based histogram matching strategy. That is, the pixels in the image to be detected and the reference image are clustered separately to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image. Since the pixels in each pixel set have similar visual characteristics, the reference pixel sets that match each source pixel set are subsequently determined. Histogram matching of each source pixel set of the image to be detected and the reference pixel sets that match the source pixel sets is actually finding corresponding regions in the reference image that have similar visual characteristics to the image to be detected, and adjusting the pixel value distribution of the corresponding regions in the image to be detected based on the pixel value distribution of each region in the reference image. Therefore, the pixel value distribution of each region is adjusted according to its own visual characteristics. This scheme, by adjusting the pixel value distribution of the image to be detected to match that of the reference image, not only ensures that the texture structure of the image to be detected remains unchanged, but also avoids drastically affecting the overall contrast of the image, thus helping to maintain the visibility of defects in the image to be detected. Therefore, this scheme can reduce the difference between the image to be detected and the reference image without weakening the defects in the image to be detected. Thus, by using a defect detection model to detect defects in the target in the image to be detected after histogram matching, the accuracy of defect detection can be improved.

[0132] In some embodiments, to ensure the accuracy and efficiency of clustering, a target clustering model is used to implement the clustering process. Accordingly, the above-mentioned clustering of pixels in the image to be detected and the reference image to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image includes: performing mean-shift clustering on the image to be detected and the reference image respectively using a target clustering model to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0133] Mean-shift clustering is an unsupervised machine learning algorithm based on a sliding window, which can be used to cluster pixels in an image. For example, mean-shift clustering includes the following steps: 1. Determine the search radius. For example, the search radius is one-tenth of the image width. 2. Construct the search window. For each pixel, create a circular search window centered on that pixel with a radius equal to the search radius. 3. Calculate the mean vector. Within each search window, calculate the mean vector of all pixels. The mean vector is calculated in a high-dimensional feature space, which includes not only the pixel value but also the spatial location information of the pixels. 4. Move the window. Move the center pixel of the search window to the position of the newly calculated mean vector. 5. Iterate and update. Repeat steps 3 and 4 until the center pixel of the search window no longer moves significantly, or the movement is less than a preset threshold. At this point, the center pixel of a pixel set is considered to have been found. 6. Merge similar center pixels. If the distance between two center pixels is less than a certain threshold, they may belong to the same pixel set, and therefore these center pixels can be merged. 7. Assign each center pixel to the surrounding pixels. Assign each pixel to the set of pixels represented by the nearest center pixel.

[0134] It should be noted that, in order to improve the clustering speed, this embodiment does not directly use the mean-shift clustering algorithm for clustering processing. Instead, the result of mean-shift clustering is used as a supervision label, which is then used to supervise the training of the target clustering model, enabling the target clustering model to perform the mean-shift clustering task. After the target clustering model is trained, it is used to perform clustering processing on the image to be detected and the reference image.

[0135] In this embodiment, since mean-shift clustering considers not only pixel values ​​but also pixel positions during clustering, using a target clustering model to perform mean-shift clustering on an image can accurately group pixels with similar features into the same set. This ensures that each source pixel set and reference pixel set can more precisely represent a specific part of the image. Furthermore, using a target clustering model for clustering significantly improves the efficiency of clustering, accelerating the image processing and providing usable images promptly for subsequent defect detection procedures.

[0136] In some embodiments, the training process of the target clustering model includes: acquiring sample images and sample clustering results, wherein the sample clustering results are obtained by mean-shift clustering of the sample images; extracting features from the sample images through the feature extraction network in the target clustering model to obtain sample image features; performing clustering processing on the sample image features through the clustering network in the target clustering model to obtain predicted clustering results; and training the target clustering model based on the predicted clustering results and the sample clustering results.

[0137] Sample images refer to the image dataset used to train the target clustering model. The sample clustering result is obtained after processing the sample images using mean shift. The sample clustering result serves as a supervision label, guiding the model's learning process.

[0138] Feature extraction networks, also known as encoding networks, are part of target clustering models and are constructed using deep learning architectures such as convolutional neural networks. Their task is to automatically learn and extract useful image features from input sample images. These image features capture important information in the image, such as edges, textures, colors, and shapes. Sample image features are the output of the feature extraction network. These image features are an abstract representation of the original image, designed to highlight information important for the clustering task while reducing noise and other irrelevant details. The quality of the image features directly affects the subsequent clustering results.

[0139] Clustering networks, also known as decoding networks, are part of the target clustering model. Their input is the sample image features output by the feature extraction network, and they are responsible for performing clustering operations based on these features. That is, they assign similar pixels in the image to the same cluster, i.e., the same set of pixels, thereby generating predicted clustering results.

[0140] Training a target clustering model involves adjusting model parameters by comparing the predicted clustering results with the sample clustering results. For example, a loss function is defined that quantifies the difference between the two clustering results. Then, the backpropagation algorithm is used to minimize this loss, iteratively updating the model weights so that the model performs better and better on new data.

[0141] In this embodiment, the target clustering model is trained using the sample clustering results obtained by mean-shift clustering of sample images. This allows the target clustering model to better understand the relationship between different features in the image and the clustering results, thereby further optimizing the feature extraction process and the feature-based clustering process, so that the predicted clustering results are the same as the sample clustering results. For example, the target clustering model may find that certain specific texture features or color combinations are closely related to specific clustering results, thus focusing more on these features in subsequent feature extraction, improving the targeting and effectiveness of feature extraction. Alternatively, the clustering network can optimize the clustering strategy to make the clustering results more accurate and detailed. In this way, a target clustering model that can accurately perform mean-shift clustering on images can be trained.

[0142] In some embodiments, the clustering network includes multiple upsampling layers and an output layer. Accordingly, the clustering network in the target clustering model performs clustering processing on sample image features to obtain a predicted clustering result, including: sequentially upsampling the sample image features through multiple upsampling layers in the clustering network to obtain the image features output by the last upsampling layer; and performing feature transformation on the image features output by the last upsampling layer through the output layer in the clustering network to obtain the predicted clustering result.

[0143] Upsampling is an operation that transforms low-resolution image features into high-resolution image features. An upsampling layer is the network layer that performs this operation. In this clustering network, multiple upsampling layers process sample image features sequentially. There are various upsampling methods, such as transposed convolution (deconvolution), bilinear interpolation, and nearest neighbor interpolation. Its purpose is to gradually restore the image resolution reduced by previous processing, making the image features gradually approach the resolution of the original image in size. This is done to enable more accurate clustering of different regions in the image later, because high-resolution image features contain more detailed information, helping to more accurately divide different cluster regions. With each upsampling operation, the size of the image features continuously increases, and the feature information becomes increasingly rich and refined.

[0144] For example, multiple downsampling layers in a clustering network integrate residual connection structures. For instance, a clustering network might include multiple Tconv resnetblock2d modules. Tconv stands for Transposed Convolution, also known as deconvolution. It's primarily used for upsampling, converting low-resolution image features into high-resolution ones. resnetblock2d is a two-dimensional residual network module that, by introducing residual connections, allows the network to be trained effectively even with increased layer depth, aiding in the extraction and learning of image features. In other words, Tconv resnetblock2d does more than just upsampling; it utilizes the resnetblock2d structure to further extract, fuse, and optimize features while performing upsampling. Residual connections preserve and pass on feature information from different levels, enabling the network to better utilize previously learned image features during resolution enhancement, avoiding information loss or feature inaccuracies that might result from simple upsampling.

[0145] The output layer, as the final layer of the clustering network, receives image features from the output of the last upsampling layer. Its main function is to further transform these high-resolution image features, converting them into the final predicted clustering result. The output layer typically contains specific computational operations, such as convolution and fully connected layers. These operations map the high-dimensional feature vectors to a low-dimensional space, where each dimension may correspond to a different clustering category. Ultimately, the output layer outputs a result representing the clustering category to which each pixel belongs; this result is the predicted clustering result.

[0146] In this embodiment, stacking multiple upsampling layers allows for the gradual refinement and adjustment of sample image features, enhancing their representational power and making them richer and more representative. This helps the clustering network better identify and distinguish different cluster objects, improving the accuracy and reliability of clustering.

[0147] In some embodiments, the feature extraction network and clustering network employ a U-Net (U-shaped network) structure. U-Net is a classic neural network structure, shaped like the letter "U," consisting of an encoder and a decoder. In this application, the feature extraction network corresponds to the encoder part, progressively reducing the image resolution through multiple downsampling layers while extracting image features at different levels. These features contain various information ranging from image details to abstract semantics. The clustering network is similar to the decoder part, progressively restoring the image resolution through multiple upsampling layers, ultimately achieving operations such as image clustering.

[0148] Correspondingly, the defect detection device performs upsampling processing on the sample image features sequentially through multiple upsampling layers in the clustering network to obtain the image features output by the last upsampling layer. This includes: for any upsampling layer in the clustering network, upsampling processing is performed on the input image features through this upsampling layer, and the upsampling image features are fused with the image features of the corresponding level in the feature extraction network to obtain the image features output by this upsampling layer.

[0149] Downsampling is an operation that reduces the resolution of an image, and a downsampling layer is a network layer that performs this operation. In feature extraction networks, multiple downsampling layers process the input image sequentially. Common downsampling methods include pooling operations, such as max pooling, average pooling, and convolution operations with a stride greater than 1. With each downsampling layer, the image resolution decreases, and the image features are further abstracted and compressed. Different levels of downsampling layers can extract image features at different levels; shallower downsampling layers retain more detailed image information, while deeper downsampling layers extract more abstract and representative features. These different levels of image features complement each other, collectively describing the image content and providing multi-dimensional information support for subsequent clustering operations.

[0150] For example, multiple downsampling layers in a feature extraction network integrate residual connection structures. For instance, a feature extraction network might include multiple `resnetblock2d conv2d(s2)` layers. `resnetblock2d` is a two-dimensional module in a residual network, possessing strong feature extraction capabilities and able to learn features at different levels of the image. `conv2d(s2)` is a two-dimensional convolution operation with a stride of 2, which halves the size of the image features during convolution, achieving downsampling. In other words, `resnetblock2d conv2d(s2)` not only includes downsampling functionality but also integrates residual connection structures to address gradient problems in deep network training, helping the network better learn and extract complex features.

[0151] Feature extraction networks extract image features at different levels through downsampling layers, while clustering networks, during upsampling, fuse the upsampled image features with corresponding layers of image features from the feature extraction network. "Corresponding layers" refers to corresponding positions in the network structure and processing flow. For example, image features processed by shallower upsampling layers in the clustering network are fused with features extracted by deeper downsampling layers in the feature extraction network, while features processed by deeper upsampling layers are fused with features extracted by shallower downsampling layers in the feature extraction network. This corresponding-layer feature fusion fully utilizes the rich information extracted by the feature extraction network at different levels, improving the quality and accuracy of the upsampled image features and contributing to more accurate clustering. Exemplary image feature fusion methods include addition and concatenation. Through fusion operations, the representational power of image features can be enhanced, providing stronger support for subsequent clustering tasks and helping to improve the accuracy and reliability of clustering.

[0152] In this embodiment, the image features extracted by different downsampling layers of the feature extraction network contain information at different scales. Shallower image features retain more detailed image information, while deeper image features contain more abstract semantic information. During the upsampling process, each upsampling layer combines image features from different levels of the feature extraction network, fusing the upsampled image features with the corresponding level of image features in the feature extraction network. This fully utilizes the multi-scale information of the image, making the clustering results more accurate and detailed.

[0153] In some embodiments, training a target clustering model based on predicted clustering results and sample clustering results includes: determining a first clustering loss value based on predicted clustering results and sample clustering results; determining multiple second clustering loss values ​​based on image features output from multiple upsampling layers and downsampling features at the corresponding levels of the predicted clustering results; and training the target clustering model based on the first clustering loss value and the multiple second clustering loss values.

[0154] The first clustering loss is a value calculated based on the difference between the predicted clustering result and the sample clustering result, used to measure the degree of difference between the two. The smaller the loss value, the closer the predicted clustering result is to the sample clustering result, and the better the model's predictive performance. Examples of methods for calculating the first clustering loss include cross-entropy loss and mean squared error loss. During training, the model's goal is to continuously reduce the first clustering loss value by adjusting the parameters, thereby improving the model's accuracy.

[0155] The second clustering loss is a series of loss values ​​calculated based on the image features output by multiple upsampling layers and the downsampling features of the corresponding levels of the predicted clustering results. Each upsampling layer corresponds to a second clustering loss value, which measures the difference between the image features output by the upsampling layer and the downsampling features of the corresponding level. The purpose of calculating the second clustering loss value is to ensure that the features output by the upsampling layers can reasonably reflect the information of the downsampling features at the corresponding level during feature processing, thereby ensuring that feature learning and processing at different levels are consistent and accurate. By optimizing these second clustering loss values, the model can be made more stable and effective in feature extraction and clustering. For example, methods for calculating the first clustering loss value include cross-entropy loss and mean squared error loss. During training, the goal of the model is to continuously reduce the first clustering loss value by adjusting the parameters, thereby improving the accuracy of the model.

[0156] For example, training a target clustering model based on a first clustering loss value and multiple second clustering loss values ​​includes: performing a weighted summation of the first clustering loss value and multiple second clustering loss values ​​to obtain a third clustering loss value, and adjusting the model parameters to reduce the third clustering loss value.

[0157] In this embodiment, by simultaneously considering a first clustering loss value based on both the predicted clustering result and the sample clustering result, and multiple second clustering loss values ​​based on the upsampling layer output features and the downsampling features corresponding to the predicted clustering result, the difference between the model's predicted result and the actual situation can be comprehensively measured from different perspectives. The first clustering loss value directly reflects the deviation between the final predicted clustering result and the actual sample clustering result, while the multiple second clustering loss values ​​focus on the consistency between the changes in features during the intermediate upsampling process and the final clustering result. This multi-dimensional loss measurement method enables more comprehensive optimization of model training, avoiding the problem of focusing only on the final result while ignoring the intermediate process, thereby improving the overall performance of the model.

[0158] In some embodiments, the training process of the target clustering model further includes: reconstructing the sample image features through the image reconstruction network in the target clustering model to obtain a reconstructed image; and training the target clustering model based on the reconstructed image and the sample image.

[0159] Image reconstruction networks are a component of target clustering models, specifically responsible for reconstructing images from input sample image features. Their function is to reconstruct a complete image—that is, an image reconstruction—through a series of processing and transformations of the sample image features extracted by the feature extraction network.

[0160] For example, an image reconstruction network includes multiple layers, such as convolutional layers, transposed convolutional layers, and activation layers. These layers work together to transform feature vectors into an image. For instance, an image reconstruction network may include multiple Transformer convolutional layers (a network layer combining the Transformer architecture and convolutional operations). The Transformer has a powerful self-attention mechanism, enabling global information exchange and feature extraction from sequential data. Convolutional operations excel at extracting local features. This fused layer captures both local details of the image and processes global information using the self-attention mechanism. In image reconstruction tasks, it allows for more comprehensive processing and transformation of image features, contributing to the generation of more accurate and semantically meaningful reconstructed images.

[0161] A reconstructed image is the image obtained by an image reconstruction network after reconstructing the features of a sample image. The quality of the reconstructed image and its similarity to the sample image are important indicators for evaluating the performance of the image reconstruction network and the overall performance of the target clustering model. When training a target clustering model, comparing the reconstructed image with the sample image allows us to evaluate the accuracy and effectiveness of the model in the feature extraction and reconstruction process, and adjust the model's parameters accordingly to optimize its performance.

[0162] A target clustering model is trained based on the reconstructed image and the sample image, that is, the target clustering model is optimized by utilizing the differences between the reconstructed image and the sample image. For example, an image loss value is determined based on the reconstructed image and the sample image, which reflects the magnitude of the difference between the reconstructed image and the sample image. The training objective of the model is to improve the quality of image reconstruction by adjusting model parameters, such as weights and biases in the network, to continuously reduce this loss value.

[0163] In this embodiment, the process of the image reconstruction network reconstructing the features of the sample image prompts the model to delve deeper into the image's intrinsic features. If the reconstructed image is very similar to the sample image, it indicates that the image features extracted by the model accurately reflect the information of the original image, demonstrating strong feature representation capabilities. Conversely, if the reconstructed image differs significantly from the sample image, the model will adjust its parameters through training to improve feature extraction and representation methods, thereby enhancing the quality of the reconstructed image. This feedback mechanism helps the model continuously optimize feature representation, making the extracted features more beneficial for subsequent clustering tasks.

[0164] In some embodiments, the training process of the target clustering model further includes: judging the sample image and the reconstructed image respectively through the image judgment network in the target clustering model, obtaining the judgment result, the judgment result indicating whether the currently judged image is a sample image or a reconstructed image; and training the target clustering model based on the judgment result.

[0165] Image discrimination networks, also known as discriminators, are components of target clustering models. Their main function is to discriminate between input images and reconstructed images. By learning the feature differences between sample and reconstructed images, image discrimination networks gain the ability to distinguish between the two types of images. For example, an image discrimination network consists of a series of neural network layers, such as convolutional layers and fully connected layers, which extract and analyze features from the input image to output a judgment result.

[0166] The judgment result is the conclusion output by the image judgment network after classifying the input sample image or reconstructed image. This result clearly indicates whether the currently judged image is a sample image or a reconstructed image. The judgment result acts as a supervisory signal during model training. By comparing the model's judgment result with the actual image category, i.e., the true label of the sample image or reconstructed image, the loss function can be calculated. This guides the training of the target clustering model, enabling the model to continuously optimize and improve the accuracy of the judgment. This training method helps the model better learn the feature differences between sample images and reconstructed images, further optimizing the model's feature extraction and processing capabilities. This indirectly improves the overall performance of the target clustering model, making it more accurate and reliable in clustering tasks.

[0167] It is understandable that the feature extraction network, clustering network, and image judgment network form a close collaborative relationship. The judgment results of the image judgment network provide feedback information to other networks. For example, the feature extraction network optimizes the representation of image features based on the judgment results, enabling the extracted image features to more effectively reflect the true attributes of the image. The clustering network, based on effective image features, makes the clustering results more accurate and detailed.

[0168] In this embodiment, the process by which the image judgment network judges the sample image and the reconstructed image helps the model capture more detailed and discriminative feature information in the image, such as subtle changes in texture and subtle differences in color. In this way, the model can mine deeper features in the image, improve its understanding and learning ability of image features, and thus improve the accuracy of clustering based on image features.

[0169] refer to Figure 2 , Figure 2This diagram illustrates a process for processing sample images using a target clustering model, as provided in some embodiments of this application. The target clustering model includes a feature extraction network, a clustering network, an image reconstruction network, and an image judgment network. The sample image is input to the feature extraction network, which extracts features and outputs the sample image features. These features are then input to the clustering network, which performs clustering processing and outputs a predicted clustering result. Additionally, the sample image features are also input to the image reconstruction network, which reconstructs the image and outputs a reconstructed image. The reconstructed image and the sample image are then input to the image judgment network, which outputs a judgment result.

[0170] In some embodiments, determining a reference pixel set that matches each source pixel set includes: determining the position coordinates of the center pixel in each source pixel set and each reference pixel set in the image coordinate system, wherein the center pixel in each pixel set is the cluster center of multiple pixels in each pixel set; and matching the source pixel set whose position coordinates are closest to the reference pixel set with the reference pixel set to obtain a reference pixel set that matches each source pixel set.

[0171] An image coordinate system is a coordinate system used to determine the position of each pixel in an image. Typically, the top-left corner of the image is the origin, the horizontal axis is the x-axis, and the vertical axis is the y-axis. Each pixel has a unique coordinate value in this system. This coordinate system allows for precise location and description of the positions of pixels in an image. In this scheme, determining the position coordinates of the center pixel in the image coordinate system provides a quantitative basis for subsequent matching operations.

[0172] When matching source and reference pixel sets, position coordinates are a crucial metric. By comparing the distances between the center pixel coordinates of different sets, such as Euclidean distance, we determine which source pixel set is most similar to which reference pixel set. This completes the matching process, yielding the reference pixel set corresponding to each source pixel set, thus laying the foundation for subsequent histogram matching.

[0173] In this embodiment, the position coordinates of the center pixels within the source and reference pixel sets in the image coordinate system are used as the matching basis. This eliminates the need for complex calculations, making the entire matching process easy to understand and implement, and improving matching efficiency. This is highly advantageous for defect detection scenarios with high real-time requirements, saving significant time and computational resources.

[0174] In some embodiments, histogram matching is performed on each source pixel set of the image to be detected and a reference pixel set that matches the source pixel set so that the pixel value distribution of the image to be detected and the reference image are the same. Before this, the defect detection device normalizes the pixel values ​​of multiple pixels in the same pixel set.

[0175] Normalization is a data preprocessing operation that normalizes the pixel values ​​of multiple pixels within the same pixel set. Its main purpose is to transform the pixel values ​​to a specific range to facilitate subsequent histogram matching. Since pixel values ​​in different images can vary significantly—for example, different lighting conditions and shooting equipment can lead to differences in overall brightness and contrast—normalized pixel values ​​can more objectively reflect the inherent characteristics of the image during histogram matching, unaffected by data biases.

[0176] Normalization refers to mapping the pixel value of each pixel in a pixel set to a new numerical range according to a certain mathematical transformation. For example, mapping it to [0, 1] or [-1, 1].

[0177] For example, a minimum-maximum value normalization process is performed on the pixels within each pixel set. Refer to the following formula, where x... i Let x represent the pixel value of the i-th pixel in the pixel set, max(x) represent the maximum pixel value in the pixel set, min(x) represent the minimum pixel value in the pixel set, and Norm represent the normalized pixel value of the i-th pixel.

[0178]

[0179] By performing the above maximum and minimum value normalization process, the pixel values ​​in each pixel set can be mapped to [0, 1].

[0180] In this embodiment, normalization is performed before histogram matching to prevent excessive changes to the internal relative structure of the image to be detected during the matching process, thus avoiding limitations on the image's usability in various subsequent application scenarios. For example, in image analysis in industrial production, subsequent operations such as feature extraction and defect detection are required on the image to be detected. The stability of the internal relative structure of the image is fundamental to ensuring the accuracy and effectiveness of these subsequent operations. Therefore, normalization can constrain pixel value variations to a certain extent, maintaining the internal relative structure of the image to be detected and ensuring that the image can meet various requirements in subsequent use.

[0181] In some embodiments, histogram matching is performed on each source pixel set of the image to be detected and a reference pixel set that matches the source pixel set to make the pixel value distribution of the image to be detected and the reference image the same. This includes: for any source pixel set, determining the cumulative distribution function of the source pixel set based on the pixel values ​​of the pixels in the source pixel set; determining the cumulative distribution function of the reference pixel set based on the pixel values ​​of the pixels in the reference pixel set that matches the source pixel set; and adjusting the pixel values ​​of multiple pixels in the source pixel set so that the cumulative distribution function of the adjusted source pixel set is the same as the cumulative distribution function of the reference pixel set.

[0182] For a given set of pixels, the cumulative distribution function (CDF) represents the proportion of pixels in that set whose values ​​are less than or equal to a specific pixel value. During histogram matching, the pixel values ​​of multiple pixels within the source pixel set are adjusted based on the difference between the CDFs of the source and reference pixel sets. The goal of this adjustment is to make the adjusted CDF of the source pixel set equal to the CDF of the reference pixel set, thereby achieving a consistent pixel value distribution between the target image and the reference image at the pixel set level.

[0183] refer to Figure 3 , Figure 3 This provides a schematic diagram of a histogram matching process. The left coordinate system's x-axis represents the pixel values ​​in the source pixel set, and the y-axis represents the cumulative distribution function (CDF). The solid line in the coordinate system represents the CDF1 of the source pixel set. The dashed line represents the CDF2 of the reference pixel set. The purpose of histogram matching is to adjust the pixel values ​​of the pixels in the source pixel set so that their CDF changes from the form shown by the solid line (CDF1) to the form shown by the dashed line (CDF2). The right coordinate system's x-axis represents the pixel values ​​in the reference pixel set, and the y-axis represents the CDF2 of the reference pixel set. For example, for a target pixel in the source pixel set, its normalized pixel value is X1, and the corresponding CDF value in the CDF is the target CDF. In the CDF of the reference pixel set matched with this source pixel set, the pixel value corresponding to the target CDF is X2. X2 is the pixel value mapped from pixel value X1 in the reference pixel set. Next, the pixel value of the pixel with a value of X1 in the source pixel set is adjusted to X2. By making this adjustment to each pixel value in the source pixel set, the cumulative distribution function of the source pixel set changes from CDF1 to CDF2. The pixel value distribution of the image region corresponding to this source pixel set then becomes the same as the pixel value distribution of the image region corresponding to this reference pixel set.

[0184] It's important to note that histogram matching aims to make the pixel value distribution of the image to be detected the same as that of the reference image. In achieving this, since the pixel values ​​of the image to be detected and the reference image may not correspond exactly, interpolation can be used to adjust the pixel values.

[0185] For example, after calculating the cumulative distribution functions of the source and reference pixel sets, for each pixel value in the source pixel set, it is necessary to find its corresponding pixel value in the reference pixel set. However, in practice, some pixel values ​​in the source pixel set may not have a direct, precise corresponding value in the cumulative distribution function of the reference image. In this case, interpolation methods can be used to estimate a suitable corresponding pixel value. This interpolation method can effectively map the pixel values ​​of the image to be detected to the pixel value distribution of the reference image in histogram matching, making the image to be detected and the reference image more similar in pixel value distribution, thereby completing the histogram matching process.

[0186] In this embodiment, by making the cumulative distribution functions of the source pixel set and the reference pixel set identical, the pixel value distribution of the image to be detected can be effectively adjusted to be consistent with the reference image. This eliminates the interference caused by the difference in pixel value distribution between the image to be detected and the reference image in defect detection. Moreover, since the cumulative distribution function mainly reflects the statistical characteristics of pixel values, rather than the specific position and arrangement of pixels, adjusting the pixel values ​​of pixels in the source pixel set to make the cumulative distribution function of the source pixel set identical with that of the reference pixel set will not cause significant damage to the structure of the image to be detected, thus facilitating defect detection based on the structure of the image to be detected.

[0187] In some embodiments, such as in a defect detection scenario, acquiring the image to be detected and the reference image used for training the defect detection model includes: taking a picture of the current detection target with a camera to obtain the image to be detected; determining any training image used by the defect detection model as the reference image, wherein the training image is obtained by taking pictures of other targets of the same type as the current detection target, and the training image is compressed.

[0188] It's important to note that industrial inspection algorithms use compressed images when training the defect detection model. However, in the defect detection process, the input is an uncompressed image captured by a camera in real-time. The pixel value distribution of these two types of images differs, leading to insufficient detection accuracy. Compressing the camera-captured image before defect detection is too slow, resulting in low efficiency throughout the process. The method presented in this application utilizes a target clustering model and histogram matching to quickly adjust the pixel value distribution of the real-time captured image to match that of the compressed image used during model training. This ensures the accuracy of defect detection.

[0189] refer to Figure 4 , Figure 4 A schematic diagram of a defect detection process is provided. The process involves a camera acquiring an image to be detected, i.e., capturing an image of the target object. Next, the pixel value distribution of the image to be detected is adjusted. The training image used to train the defect detection model is used as a reference image, and the pixel value distribution of the image to be detected is adjusted to be the same as that of the reference image using the method provided in this application. Then, the already deployed defect detection model is used to detect defects in the adjusted image. It should be noted that the image to be detected acquired by the camera is compressed using an image compression model, and this compressed image can be stored. A large number of compressed images are then used as training images to train the defect detection model. After the defect detection model is trained, it can be deployed into the defect detection process to perform the defect detection task.

[0190] In this embodiment, since the defect detection model is trained based on training images, when the pixel value distribution of the image to be detected is the same as that of the training image, the model can process the image to be detected more stably, reducing false positives and false negatives caused by differences in image features. Therefore, using any training image used by the defect detection model as a reference image and matching the pixel value distribution of the image to be detected and the reference image through histogram matching helps to improve the stability of the defect detection model and make the defect detection results more reliable.

[0191] This application also proposes a defect detection device, with reference to... Figure 5 The defect detection device includes:

[0192] The image acquisition module 10 is used to acquire the image to be detected and the reference image used for training the defect detection model. The pixel value distribution of the image to be detected and the reference image are different.

[0193] The image clustering module 20 is used to perform clustering processing on the pixels in the image to be detected and the reference image respectively, so as to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0194] Image matching module 30 is used to determine a set of reference pixels that match each source pixel set;

[0195] The distribution adjustment module 40 is used to perform histogram matching on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same.

[0196] The defect detection module 50 is used to perform defect detection on the target in the image to be detected after histogram matching using a defect detection model, and obtain the defect detection result.

[0197] In some embodiments, the image clustering module 20 is used to perform mean-shift clustering on the image to be detected and the reference image respectively using a target clustering model to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

[0198] In some embodiments, the training process of the target clustering model includes:

[0199] Obtain sample images and sample clustering results. The sample clustering results are obtained by performing mean-shift clustering on the sample images.

[0200] The feature extraction network in the target clustering model is used to extract features from the sample image to obtain the sample image features.

[0201] The clustering network in the target clustering model is used to cluster the features of the sample images to obtain the predicted clustering results.

[0202] The target clustering model is trained based on the predicted clustering results and the sample clustering results.

[0203] In some embodiments, the clustering network includes multiple upsampling layers and an output layer;

[0204] The clustering network in the target clustering model is used to cluster the features of the sample images to obtain the predicted clustering results, including:

[0205] By using multiple upsampling layers in the clustering network, upsampling is performed sequentially based on the sample image features to obtain the image features output by the last upsampling layer.

[0206] By using the output layer of the clustering network, the image features output by the last upsampling layer are transformed to obtain the predicted clustering results.

[0207] In some embodiments, the feature extraction network includes multiple downsampling layers, which are used to extract image features at different levels.

[0208] By using multiple upsampling layers in the clustering network, upsampling is performed sequentially based on the sample image features to obtain the image features output by the last upsampling layer, including:

[0209] For any upsampling layer in the clustering network, the input image features are upsampled through the upsampling layer. The upsampled image features are then fused with the image features of the corresponding layer in the feature extraction network to obtain the image features output by the upsampling layer.

[0210] In some embodiments, a target clustering model is trained based on predicted clustering results and sample clustering results, including:

[0211] Based on the predicted clustering results and the sample clustering results, determine the first clustering loss value;

[0212] Multiple second clustering loss values ​​are determined based on the image features output from multiple upsampling layers and the downsampling features of the corresponding layers of the predicted clustering results;

[0213] The target clustering model is trained based on the first clustering loss value and multiple second clustering loss values.

[0214] In some embodiments, the training process of the target clustering model further includes:

[0215] The image reconstruction network in the target clustering model is used to reconstruct the features of the sample image to obtain the reconstructed image.

[0216] A target clustering model is trained based on reconstructed images and sample images.

[0217] In some embodiments, the training process of the target clustering model further includes:

[0218] The image judgment network in the target clustering model judges the sample image and the reconstructed image separately, and obtains the judgment result, which indicates whether the currently judged image is a sample image or a reconstructed image.

[0219] Based on the judgment results, a target clustering model is trained.

[0220] In some embodiments, the image matching module 30 is used to determine the position coordinates of the center pixel in each source pixel set and each reference pixel set in the image coordinate system, wherein the center pixel in each pixel set is the cluster center of multiple pixels in each pixel set; and to match the source pixel set and the reference pixel set whose position coordinates are closest to the center pixel to obtain the reference pixel set that matches each source pixel set.

[0221] In some embodiments, the distribution adjustment module 40 is further configured to normalize the pixel values ​​of multiple pixels within the same pixel set before performing histogram matching on each source pixel set of the image to be detected and a reference pixel set that matches the source pixel set so that the pixel value distribution of the image to be detected and the reference image are the same.

[0222] In some embodiments, the distribution adjustment module 40 is configured to, for any source pixel set, determine the cumulative distribution function of the source pixel set based on the pixel values ​​of the pixels in the source pixel set; determine the cumulative distribution function of the reference pixel set based on the pixel values ​​of the pixels in the reference pixel set matched by the source pixel set; and adjust the pixel values ​​of multiple pixels in the source pixel set so that the cumulative distribution function of the adjusted source pixel set is the same as the cumulative distribution function of the reference pixel set.

[0223] In some embodiments, the image acquisition module 10 is used to capture the current detection target with a camera to obtain an image to be detected; any training image used by the defect detection model is determined as a reference image, wherein the training image is obtained by capturing other targets of the same type as the current detection target, and the training image is compressed.

[0224] The defect detection device provided in this application employs the defect detection method described in the above embodiments. Its beneficial effects are the same as those of the defect detection method provided in the above embodiments. Furthermore, other technical features of the defect detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0225] Furthermore, this application also proposes a defect detection device, which includes a memory, a processor, and a defect detection program stored in the memory and executable on the processor. The defect detection program is configured to implement the steps of the defect detection method described above.

[0226] The following is for reference. Figure 6 The diagram illustrates a structural schematic of a defect detection device suitable for implementing embodiments of this application. The defect detection device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6The defect detection device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0227] like Figure 6 As shown, the defect detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the defect detection device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the defect detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows defect detection equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0228] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0229] The defect detection device provided in this application employs the defect detection method described in the above embodiments. Its beneficial effects are the same as those of the defect detection method provided in the above embodiments. Furthermore, other technical features of the defect detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0230] This application also provides a storage medium storing a defect detection program, which, when executed by a processor, implements the steps of the defect detection method described above.

[0231] The beneficial effects of the storage medium provided in this application are the same as those of the defect detection method provided in the above embodiments, and will not be repeated here.

[0232] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the defect detection method described above.

[0233] The beneficial effects of the computer program product provided in this application are the same as those of the defect detection method provided in the above embodiments, and will not be repeated here.

[0234] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0235] In addition, for technical details not described in detail in this embodiment, please refer to the defect detection method provided in any embodiment of this application, which will not be repeated here.

[0236] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A defect detection method, characterized in that, The method includes: Obtain the image to be detected and the reference image used for training the defect detection model, wherein the pixel value distribution of the image to be detected and the reference image are different; Clustering is performed on the pixels in the image to be detected and the reference image respectively to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image; Determine the reference pixel set that matches each of the source pixel sets; Histogram matching is performed on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same; The defect detection model is used to detect defects in the target in the image to be detected after histogram matching, and the defect detection result is obtained.

2. The method as described in claim 1, characterized in that, The step of clustering the pixels in the image to be detected and the reference image to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image includes: Using a target clustering model, mean-shift clustering is performed on the image to be detected and the reference image respectively to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image.

3. The method as described in claim 2, characterized in that, The training process of the target clustering model includes: Obtain sample images and sample clustering results, wherein the sample clustering results are obtained by performing mean-shift clustering on the sample images; The feature extraction network in the target clustering model is used to extract features from the sample image to obtain sample image features; The clustering network in the target clustering model is used to perform clustering processing on the features of the sample image to obtain the predicted clustering results; The target clustering model is trained based on the predicted clustering results and the sample clustering results.

4. The method as described in claim 3, characterized in that, The clustering network includes multiple upsampling layers and an output layer; The step of clustering the sample image features through the clustering network in the target clustering model to obtain the predicted clustering results includes: Through multiple upsampling layers in the clustering network, upsampling processing is performed sequentially based on the sample image features to obtain the image features output by the last upsampling layer; The predicted clustering result is obtained by performing feature transformation on the image features output by the last upsampling layer through the output layer of the clustering network.

5. The method as described in claim 4, characterized in that, The feature extraction network includes multiple downsampling layers, which are used to extract image features at different levels. The process of sequentially upsampling the sample image features through multiple upsampling layers in the clustering network to obtain the image features output by the last upsampling layer includes: For any upsampling layer in the clustering network, the input image features are upsampled through the upsampling layer, and the upsampled image features are fused with the image features of the corresponding layer in the feature extraction network to obtain the image features output by the upsampling layer.

6. The method as described in claim 4, characterized in that, The step of training the target clustering model based on the predicted clustering results and the sample clustering results includes: Based on the predicted clustering results and the sample clustering results, a first clustering loss value is determined; Multiple second clustering loss values ​​are determined based on the image features output by the multiple upsampling layers and the downsampling features of the corresponding layers of the predicted clustering results; The target clustering model is trained based on the first clustering loss value and the plurality of second clustering loss values.

7. The method as described in claim 3, characterized in that, The training process of the target clustering model also includes: The image reconstruction network in the target clustering model is used to reconstruct the features of the sample image to obtain a reconstructed image; The target clustering model is trained based on the reconstructed image and the sample image.

8. The method as described in claim 7, characterized in that, The training process of the target clustering model also includes: The image judgment network in the target clustering model is used to judge the sample image and the reconstructed image respectively, and the judgment result is obtained. The judgment result indicates whether the currently judged image is a sample image or a reconstructed image. Based on the judgment result, the target clustering model is trained.

9. The method as described in claim 1, characterized in that, Determining the reference pixel set that matches each of the source pixel sets includes: Determine the position coordinates of the center pixel in each source pixel set and each reference pixel set in the image coordinate system, and the center pixel in each pixel set is the cluster center of multiple pixels in each pixel set; The source pixel set whose position coordinates are closest to the center pixel point and the reference pixel set are matched to obtain the reference pixel set that matches each of the source pixel sets.

10. The method as described in claim 1, characterized in that, Before performing histogram matching on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same, the method further includes: The pixel values ​​of multiple pixels within the same pixel set are normalized.

11. The method as described in claim 1, characterized in that, The step of performing histogram matching on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same, includes: For any source pixel set, the cumulative distribution function of the source pixel set is determined based on the pixel values ​​of the pixels within the source pixel set; Based on the pixel values ​​of the pixels in the reference pixel set matched by the source pixel set, the cumulative distribution function of the reference pixel set is determined; The pixel values ​​of multiple pixels within the source pixel set are adjusted so that the cumulative distribution function of the adjusted source pixel set is the same as the cumulative distribution function of the reference pixel set.

12. The method as described in claim 1, characterized in that, The acquisition of the image to be detected and the reference image used for training the defect detection model includes: The target to be detected is captured by a camera to obtain the image to be detected; The reference image is determined by any training image used by the defect detection model. The training image is obtained by taking pictures of other targets of the same type as the current detection target, and the training image is compressed.

13. A defect detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be detected and the reference image used for training the defect detection model, wherein the pixel value distribution of the image to be detected and the reference image are different; The image clustering module is used to perform clustering processing on the pixels in the image to be detected and the reference image respectively, to obtain multiple source pixel sets of the image to be detected and multiple reference pixel sets of the reference image. An image matching module is used to determine the reference pixel set that matches each of the source pixel sets; The distribution adjustment module is used to perform histogram matching on each source pixel set of the image to be detected and the reference pixel set that matches the source pixel set, so that the pixel value distribution of the image to be detected and the reference image are the same. The defect detection module is used to perform defect detection on the target in the image to be detected after histogram matching using the defect detection model, and obtain the defect detection result.

14. A defect detection device, characterized in that, The device includes: a memory, a processor, and a defect detection program stored in the memory and executable on the processor, the defect detection program being configured to implement the steps of the defect detection method as described in any one of claims 1 to 12.

15. A storage medium, characterized in that, The storage medium stores a defect detection program, which, when executed by a processor, implements the steps of the defect detection method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the defect detection method as described in any one of claims 1 to 12.

Images