A pattern defect detection and repair method, device, equipment and medium

Abstract

This invention provides a method, apparatus, device, and medium for pattern defect detection and repair. The method includes: preprocessing an input image to obtain a preprocessed image; extracting pattern units from the preprocessed image and calculating parameters for characterizing the pattern space and feature patterns; detecting and locating defects based on the feature vector of the standard unit and the average adjacent spacing; generating a repaired image block; and fusing the repaired image block with the original image. The aim is to completely solve the core pain points of traditional methods, such as strong reliance on manual intervention, high defect miss rate, poor repair effect, and weak scene adaptability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for detecting and repairing pattern defects. Background Technology

[0002] In the production and quality inspection of repetitive printed products such as textiles, wallpaper, and packaging printing, the consistency and integrity of repetitive printing are core quality evaluation indicators. However, current methods for detecting and repairing defects in repetitive printing used in this field still have many shortcomings, making it difficult to meet the core requirements of industrial production for efficiency, accuracy, and automation. These shortcomings are specifically reflected in the following aspects: Firstly, traditional inspection methods rely on manual visual inspection, which has inherent drawbacks such as low efficiency and high subjectivity. This method is easily affected by human factors such as the experience level and fatigue of the inspectors, which not only fails to meet the real-time quality inspection needs of mass production scenarios, but also has extremely low accuracy in identifying minor defects such as distortion of basic patterns and slight misalignment, making it difficult to achieve comprehensive quality inspection.

[0003] Secondly, existing automated detection algorithms have limitations, focusing on identifying local defects in single primitives while neglecting the topological relationships inherent in repeated printing. This deficiency prevents the algorithms from effectively identifying topological anomalies such as missing or misaligned primitives, resulting in a high rate of missed defects and failing to meet the requirements for accurate detection.

[0004] Third, existing restoration techniques are relatively crude, often employing basic methods such as simple replacement and interpolation filling, resulting in poor restoration effects. After restoration using these methods, the texture continuity and topological consistency between the restored area and the surrounding normal print are poor, easily leaving obvious restoration marks. Furthermore, they cannot adapt to repeating print patterns of different styles and complexities, making it difficult to meet diverse restoration needs.

[0005] Furthermore, existing methods are highly sensitive to interference factors in real-world industrial scenarios, such as uneven lighting and image noise, and lack robustness. This makes it difficult to balance detection accuracy and repair effectiveness, failing to meet the stringent quality control requirements of high-end printed products and hindering the high-quality development of the repeat printing industry. Summary of the Invention

[0006] The technical problem this invention aims to solve is to provide a method, apparatus, equipment, and medium for pattern defect detection and repair, aiming to completely resolve the core pain points of traditional methods, such as heavy reliance on manual labor, high defect false negative rates, poor repair effects, and weak scene adaptability. This solution employs a closed-loop design encompassing "image preprocessing - repeated primitive extraction - multi-level defect detection - generative repair - edge fusion," integrating key technologies such as adaptive denoising, connected component analysis, multi-scale attention fusion, and lightweight generative adversarial networks (CGAN) to achieve accurate detection and natural repair of both primitive-level and topological-level defects in repetitive printed products. This method can automatically extract the core features and topological distribution rules of repeated primitives, effectively improving the accuracy of defect identification in complex industrial scenarios. Simultaneously, it can accurately match the texture details and topological structure of the repaired area with the surrounding normal pattern, completely eliminating repair traces and truly achieving automation, efficiency, and high precision in defect detection and repair. This solution is highly versatile and can be widely applied to quality control and defect correction work for repetitive printed products in various fields such as textiles and printing, providing technical support for the high-quality development of the industry.

[0007] In a first aspect, the present invention provides a method for detecting and repairing pattern defects, comprising the following steps: Step S1: Preprocess the input image to obtain a preprocessed image; Step S2: Perform edge detection and connected component analysis on the preprocessed image. Determine a representative area based on the statistical distribution of connected component areas, and select connected components with areas similar to the representative area as candidate pattern units. For each candidate pattern unit, extract image data within its bounding rectangle region to form the feature vector of the candidate pattern unit. Calculate the average value of all candidate pattern unit feature vectors as the standard unit feature vector. Calculate the average adjacent spacing between all candidate pattern units. The average adjacent spacing is calculated as follows: First, select candidate unit pairs with a centroid distance less than a preset threshold; then, calculate the average distance between all selected unit pairs. Step S3: Calculate the feature similarity between the feature vector of each candidate pattern unit and the feature vector of the standard unit, and determine the candidate pattern units with feature similarity lower than a first preset threshold as first-type defective units; for each candidate pattern unit, determine its neighboring units as other units whose centroid distance from other units is within a predetermined floating range of the average adjacent spacing; count the number of neighboring units for each unit, and determine the units with fewer than a second preset threshold as second-type defective units; fuse the position information of the first-type defective units and the second-type defective units to obtain the defective region mask; Step S4: Select units in the candidate pattern units that do not belong to the first type of defect unit and the second type of defect unit as normal units; for each defect region, select reference units from the normal units based on the average adjacent spacing and feature similarity; input the defect region image and the average adjacent spacing into the pre-trained neural network model to generate repaired image blocks; fuse the repaired image blocks with the original image.

[0008] Secondly, the present invention provides a pattern defect detection and repair device, comprising: The preprocessing module preprocesses the input image to obtain a preprocessed image; The pattern parameter acquisition module performs edge detection and connected component analysis on the preprocessed image. Based on the statistical distribution of connected component areas, a representative area is determined, and connected components with areas close to the representative area are selected as candidate pattern units. For each candidate pattern unit, image data within its bounding rectangle is extracted to form the feature vector of the candidate pattern unit. The average value of the feature vectors of all candidate pattern units is calculated as the standard unit feature vector. The average adjacent spacing between all candidate pattern units is calculated as follows: first, candidate unit pairs with a centroid distance less than a preset threshold are selected; then, the average distance between all selected unit pairs is calculated. The defect area mask acquisition module calculates the feature similarity between the feature vector of each candidate pattern unit and the feature vector of the standard unit, and determines the candidate pattern units with feature similarity lower than a first preset threshold as first-type defect units; for each candidate pattern unit, it determines other units whose centroid distance from other units is within a predetermined floating range of the average adjacent spacing as its neighboring units; it counts the number of neighboring units for each unit, and determines units with fewer than a second preset threshold as second-type defect units; it fuses the position information of the first-type defect units and the second-type defect units to obtain the defect area mask; The image repair module identifies units in the candidate pattern units that do not belong to the first or second type of defect units as normal units; for each defect region, it selects reference units from the normal units based on the average adjacent spacing and feature similarity; it inputs the defect region image and the average adjacent spacing into the pre-trained neural network model to generate repaired image blocks; and it fuses the repaired image blocks with the original image.

[0009] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0010] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0011] One or more technical solutions provided by this invention have at least the following technical effects or advantages: (i) High detection accuracy and low false negative rate. This invention employs a dual detection mechanism at both the primitive level and the topology level. It identifies local defects such as distortion and color anomalies within primitives through feature similarity, while simultaneously identifying global anomalies such as missing or misaligned primitives through topological rule verification. Combined with multi-scale attention fusion to optimize defect localization, it effectively overcomes the limitations of traditional single-dimensional detection methods. Experimental results demonstrate that this method achieves an accuracy rate of over 96% in identifying defects in repetitive printing, with a false negative rate of less than 3%, and can accurately identify minute defects with an area smaller than 0.5 mm², significantly outperforming existing detection algorithms.

[0012] (ii) The repair effect is natural and the topological consistency is strong. Relying on lightweight CGAN and topological constraint technology, the repaired area can not only match the surrounding texture details, but also strictly follow the topological rules such as the spacing and rotation angle of the primitives in the repeated printing. Then, the boundary traces are eliminated by edge gradient fusion. The repaired image has no obvious traces of manual correction. The texture continuity and overall consistency between the repaired area and the original printing are better than traditional replacement and interpolation repair methods, which can meet the quality requirements of high-end printed products.

[0013] (iii) Strong robustness and wide adaptability to various scenarios Through adaptive denoising and brightness normalization, it effectively resists interference from uneven lighting and image noise, and can work stably in printed images with different lighting conditions and resolutions. The primitive extraction adopts adaptive area screening and topological feature calculation, which does not require manual parameter setting. It can be adapted to repeating printed patterns in different fields such as textile fabrics, wallpaper, and packaging printing, including various printing types such as regular arrangement and slight rotation arrangement, and has strong versatility.

[0014] (iv) High degree of automation and excellent efficiency The entire process requires no manual intervention, automatically executing from image input to repair completion. The detection and repair time for a single 1024×1024 resolution printed image can be controlled within 3 seconds, supporting real-time quality inspection and repair of large-volume printed products. The lightweight CGAN model simplifies the network structure, reduces computational complexity, and is adaptable to industrial embedded devices, balancing detection and repair accuracy with processing efficiency, significantly improving the automation level and productivity of production quality inspection.

[0015] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the device in Embodiment 2 of the present invention. Detailed Implementation

[0018] The overall concept of the technical solution in this application is as follows: This invention utilizes a four-stage core process—preprocessing and primitive extraction, multi-level defect detection, generative repair, and edge fusion—combining computer vision, deep learning, and image processing algorithms to achieve automatic detection and repair of defects in repetitive printing. Each stage is closely linked and progressively advanced, ensuring both comprehensive and accurate defect detection and natural and consistent repair results. Specific steps and terminology are explained below: (a) Terminology Explanation 1. Adaptive Denoising (fastNlMeansDenoisingColored): A color image denoising algorithm that can effectively remove Gaussian noise, salt-and-pepper noise, etc. while preserving image texture details, avoiding noise interference with subsequent primitive extraction and defect detection.

[0019] 2. Repeating pattern primitive: The smallest basic unit in repeating printing, which has consistent characteristics (area, shape, texture) and is arranged according to certain topological rules, is the core component of repeating printing. Its characteristics and arrangement rules determine the overall consistency of the printing.

[0020] 3. Connected component analysis: By marking adjacent regions in an image that have the same pixel value (or meet specific conditions), the target region can be segmented and its features extracted. This invention is used to filter repeating pattern primitives and extract their topological information.

[0021] 4. Lightweight Generative Adversarial Network (CGAN): A lightweight model based on the improvement of Generative Adversarial Network (GAN), consisting of a generator and a discriminator (this invention focuses on the generator). Combining the U-Net structure and topological constraint layer, it can reduce computational complexity while ensuring repair effect and quickly generate repair regions that conform to topological rules.

[0022] 5. Multi-scale attention fusion: By fusing defect detection results from different levels, defect areas are marked and merged, improving the accuracy of defect localization and avoiding missed or false detections caused by a single detection dimension.

[0023] (II) Image preprocessing and extraction of repeating pattern primitives This stage removes image noise and illumination interference through preprocessing, and then extracts repeating primitives and topological features through edge detection and connected component analysis, providing a foundation for subsequent defect detection. The specific steps are as follows: 1. Image Preprocessing: First, the input repeating print image is read, and the BGR color space is converted to RGB color space to adapt to subsequent algorithm processing. An adaptive denoising algorithm (fastNlMeansDenoisingColored) is used to denoise the image, and the parameters are set (filter strength 10, color filter strength 10, template window size 7, search window size 21) to remove noise while preserving the print texture details. The image pixel values ​​are mapped to the range of 0-255 by brightness / contrast normalization (cv2.NORM_MINMAX) to eliminate the influence of uneven lighting on subsequent processing and obtain a clear image after preprocessing.

[0024] 2. Repetitive Primitive Extraction: The preprocessed image is converted to grayscale, and the Canny edge detection algorithm (sigma=1) is used to extract the edges of the printed outline to enhance the primitive boundary features. Connected regions in the edge image are marked by connected component analysis (measure.label), and regions with an area greater than 50 pixels are selected to filter out small noise interference. The mode of the area of ​​the connected regions is used as the standard primitive area threshold, and connected regions whose area deviates from the threshold by ±20% are selected as candidate primitives. The core information of each candidate primitive is extracted, including centroid coordinates, rotation angle (orientation), bounding box (bbox), and grayscale feature fingerprint (flattened vector of grayscale value of bounding box region), forming a primitive information set.

[0025] 3. Topological feature calculation: Based on the centroid coordinates of the primitives, calculate the average distance between adjacent primitives (only count adjacent primitives whose distance is less than 5 times the square root of the primitive area, and exclude interference from non-adjacent primitives); take the average of the feature fingerprints of all primitives to obtain the standard primitive features, which serve as the benchmark for subsequent defect judgment; finally, output the topological feature data containing the primitive information set, average distance, and standard primitive features.

[0026] (III) Multi-level defect detection and regional fusion This stage uses both primitive-level and topology-level detection to locate defects, and then merges defect areas through attention fusion to achieve comprehensive and accurate defect localization. The specific steps are as follows: 1. Primitive-level residual detection: Calculate the cosine similarity between each primitive feature fingerprint and the standard primitive feature. If the similarity is lower than the preset threshold (1-0.15=0.85), it is judged as a primitive-level defect (such as primitive pattern distortion, color abnormality). Record the index, bounding box, centroid and defect degree (1-cosine similarity) of the defective primitive to form a primitive-level defect set.

[0027] 2. Topology-level anomaly detection: Based on the centroid coordinates and average spacing of the primitives, check whether the number of adjacent primitives of each primitive is abnormal. Primitives whose distance is within ±20% of the average spacing are identified as adjacent primitives. If a single primitive has fewer than 2 adjacent primitives, it is identified as a topology-level defect (such as missing primitives or misaligned positions). The index, centroid, and defect type of the defective primitive are recorded to form a topology-level defect set.

[0028] 3. Multi-scale attention fusion: Create a flaw mask with the same size as the original image (initially all 0s), and mark primitive-level flaw regions (by bounding box) as foreground (pixel value 255); for topological-level flaw regions, draw a circular region with the centroid as the center and the square root of the average spacing as the radius and mark it as foreground to expand the coverage of topological flaws; use 5×5 elliptical structuring elements to dilate the flaw mask and merge adjacent flaw regions; extract the final flaw region bounding box through contour detection (cv2.findContours), and filter regions with an area greater than 100 pixels to filter out false flaws, resulting in an accurate set of flaw regions and flaw mask.

[0029] (iv) Generative Flaw Repair and Edge Blending This stage involves selecting the optimal reference primitives, generating the repair region using a lightweight CGAN, and then eliminating repair traces through edge blending to ensure a natural repair effect. The specific steps are as follows: 1. Optimal Matching Primitive Selection: For each defective region, its centroid coordinates are calculated. After excluding defective primitives, the nearest normal primitives with the highest feature similarity are selected. A weighted score based on distance and feature similarity (weight = (1 - normalized distance) × feature similarity) is used, and the three primitives with the highest scores are selected as optimal reference primitives to provide texture and topological reference for repair.

[0030] 2. Lightweight CGAN Generation of Repair Regions: A pre-trained lightweight CGAN model is loaded. This generator is based on a U-Net architecture and includes an encoder, decoder, and a topological constraint layer. The encoder extracts blemish region features through two convolutions and max pooling operations, while the decoder upsamples features through transposed convolution. The topological constraint layer incorporates topological features such as the average spacing and rotation angle of primitives into the encoded features, forcing the generated region to conform to the topological rules of repeating patterns. The blemish region image is normalized and input into the model. Combining the topological features, a repair region consistent with the texture and topology of the surrounding pattern is generated. The generated result is then inversely normalized to the 0-255 pixel range.

[0031] 3. Edge Blending: To eliminate boundary traces between the repaired area and the original image, an overlap blending width of 10 pixels is set, generating a gradient mask (the weight of the boundary area linearly transitions from 0 to 1, and the weight of the center area is 1). The repaired area and the corresponding area of ​​the original image are then blended according to the mask weights to achieve a smooth boundary transition, ultimately resulting in a complete image without repair traces.

[0032] (V) Integration of the main algorithm process The above stages are integrated into a complete process: First, the input printed image is preprocessed and primitives are extracted; then, multi-level defect detection and region fusion are performed. If no defects are detected, the original image is directly output; if defects exist, a pre-trained CGAN model is loaded, and optimal primitive selection, generative repair, and edge fusion operations are performed sequentially for each defect region; finally, the repaired complete image and defect mask are output, completing the entire detection and repair process. Example 1

[0033] like Figure 1 As shown, this embodiment provides a method for detecting and repairing pattern defects, including the following steps: Step S1: Preprocess the input image to obtain a preprocessed image; Step S2: Perform edge detection and connected component analysis on the preprocessed image. Determine a representative area based on the statistical distribution of connected component areas, and select connected components with areas similar to the representative area as candidate pattern units. For each candidate pattern unit, extract image data within its bounding rectangle region to form the feature vector of the candidate pattern unit. Calculate the average value of all candidate pattern unit feature vectors as the standard unit feature vector. Calculate the average adjacent spacing between all candidate pattern units. The average adjacent spacing is calculated as follows: First, select candidate unit pairs with a centroid distance less than a preset threshold; then, calculate the average distance between all selected unit pairs. Step S3: Calculate the feature similarity between the feature vector of each candidate pattern unit and the feature vector of the standard unit, and determine the candidate pattern units with feature similarity lower than a first preset threshold as first-type defective units; for each candidate pattern unit, determine its neighboring units as other units whose centroid distance from other units is within a predetermined floating range of the average adjacent spacing; count the number of neighboring units for each unit, and determine the units with fewer than a second preset threshold as second-type defective units; fuse the position information of the first-type defective units and the second-type defective units to obtain the defective region mask; Step S4: Select units in the candidate pattern units that do not belong to the first type of defect unit and the second type of defect unit as normal units; for each defect region, select reference units from the normal units based on the average adjacent spacing and feature similarity; input the defect region image and the average adjacent spacing into the pre-trained neural network model to generate repaired image blocks; fuse the repaired image blocks with the original image.

[0034] In this embodiment, preferably, the preprocessing in step S1 includes: converting the image from the BGR color space to the RGB color space, performing noise reduction using a fast nonlocal mean algorithm, and normalizing the image pixel values.

[0035] In this embodiment, preferably, in step S2, determining the representative area includes: statistically analyzing the distribution of all connected domain areas, and determining the area value with the highest frequency of occurrence in the distribution, or the center value of a preset area interval with the largest number of connected domains, as the representative area; The term "screened connected regions with areas similar to the representative area" refers to connected regions whose screened areas deviate from the representative areas by less than ±20%. The step of extracting image data within its bounding rectangular region to form the feature vector of the unit specifically includes: converting the region into a grayscale image and flattening the grayscale pixel values ​​into a one-dimensional vector. The preset threshold is K times the square root of the representative area, where K is a predefined constant.

[0036] In this embodiment, preferably, in step S3, the similarity is cosine similarity, and the first preset threshold is 0.85; The predetermined floating range is ±20% of the average adjacent spacing, and / or the second preset threshold is 2; The specific steps for fusing the positional information of the first and second types of defective units to obtain the defective region mask are as follows: marking the circumscribed rectangular region corresponding to the first type of defective unit and the circular region around the centroid of the second type of defective unit with a radius equal to the square root of the average adjacent spacing as the foreground in the mask image; performing a morphological dilation operation on the mask image to merge adjacent regions; and extracting connected regions with an area greater than a preset threshold through contour detection as the final defective region.

[0037] In this embodiment, preferably, in step S4, the screening reference unit includes: calculating the distance between the centroid of the normal unit and the defective region, and the similarity between the feature vector of the normal unit and the feature vector of the standard unit; calculating a weighted score based on the distance and similarity, and selecting the top K normal units with the highest scores as reference units; The pre-trained neural network model is a conditional generative adversarial network, and its generator has an encoder-decoder structure.

[0038] In this embodiment, preferably, the generator further includes a topological constraint layer for converting the average adjacent spacing scalar into a feature vector and fusing it with the features extracted by the encoder.

[0039] In this embodiment, preferably, the fusion of the repaired image block and the original image specifically involves: at the boundary of the defective region, the repaired image block and the original image are weighted and mixed according to a preset gradient weight to achieve fusion; the weighted mixing according to the preset gradient weight includes: generating a weight mask that changes linearly from 0 to 1 in an overlapping area of ​​a preset width at the boundary of the defective region; and weighting and fusion of the repaired image block and the original image in the overlapping area according to the weight mask.

[0040] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2. Example 2

[0041] like Figure 2 As shown, this embodiment provides a pattern defect detection and repair device, including: The preprocessing module preprocesses the input image to obtain a preprocessed image; The pattern parameter acquisition module performs edge detection and connected component analysis on the preprocessed image. Based on the statistical distribution of connected component areas, a representative area is determined, and connected components with areas close to the representative area are selected as candidate pattern units. For each candidate pattern unit, image data within its bounding rectangle is extracted to form the feature vector of the candidate pattern unit. The average value of the feature vectors of all candidate pattern units is calculated as the standard unit feature vector. The average adjacent spacing between all candidate pattern units is calculated as follows: first, candidate unit pairs with a centroid distance less than a preset threshold are selected; then, the average distance between all selected unit pairs is calculated. The defect area mask acquisition module calculates the feature similarity between the feature vector of each candidate pattern unit and the feature vector of the standard unit, and determines the candidate pattern units with feature similarity lower than a first preset threshold as first-type defect units; for each candidate pattern unit, it determines other units whose centroid distance from other units is within a predetermined floating range of the average adjacent spacing as its neighboring units; it counts the number of neighboring units for each unit, and determines units with fewer than a second preset threshold as second-type defect units; it fuses the position information of the first-type defect units and the second-type defect units to obtain the defect area mask; The image repair module identifies units in the candidate pattern units that do not belong to the first or second type of defect units as normal units; for each defect region, it selects reference units from the normal units based on the average adjacent spacing and feature similarity; it inputs the defect region image and the average adjacent spacing into the pre-trained neural network model to generate repaired image blocks; and it fuses the repaired image blocks with the original image.

[0042] In this embodiment, preferably, the preprocessing in the preprocessing module includes: converting the image from the BGR color space to the RGB color space, performing noise reduction using a fast nonlocal mean algorithm, and normalizing the image pixel values.

[0043] In this embodiment, preferably, in the pattern parameter acquisition module, determining the representative area includes: statistically analyzing the distribution of all connected region areas, and determining the area value with the highest frequency of occurrence in the distribution, or the center value of a preset area interval with the largest number of connected regions, as the representative area; The term "screened connected regions with areas similar to the representative area" refers to connected regions whose screened areas deviate from the representative areas by less than ±20%. The step of extracting image data within its bounding rectangular region to form the feature vector of the unit specifically includes: converting the region into a grayscale image and flattening the grayscale pixel values ​​into a one-dimensional vector. The preset threshold is K times the square root of the representative area, where K is a predefined constant.

[0044] In this embodiment, preferably, in the defect area mask acquisition module, the similarity is cosine similarity, and the first preset threshold is 0.85; The predetermined floating range is ±20% of the average adjacent spacing, and / or the second preset threshold is 2; The specific steps for fusing the positional information of the first and second types of defective units to obtain the defective region mask are as follows: marking the circumscribed rectangular region corresponding to the first type of defective unit and the circular region around the centroid of the second type of defective unit with a radius equal to the square root of the average adjacent spacing as the foreground in the mask image; performing a morphological dilation operation on the mask image to merge adjacent regions; and extracting connected regions with an area greater than a preset threshold through contour detection as the final defective region.

[0045] In this embodiment, preferably, in the image restoration module, the selection reference unit includes: calculating the distance between the centroid of the normal unit and the defective region, and the similarity between the feature vector of the normal unit and the feature vector of the standard unit; calculating a weighted score based on the distance and similarity, and selecting the top K normal units with the highest scores as reference units; The pre-trained neural network model is a conditional generative adversarial network, and its generator has an encoder-decoder structure.

[0046] In this embodiment, preferably, the generator further includes a topological constraint layer for converting the average adjacent spacing scalar into a feature vector and fusing it with the features extracted by the encoder.

[0047] In this embodiment, preferably, the fusion of the repaired image block and the original image specifically involves: at the boundary of the defective region, the repaired image block and the original image are weighted and mixed according to a preset gradient weight to achieve fusion; the weighted mixing according to the preset gradient weight includes: generating a weight mask that changes linearly from 0 to 1 in an overlapping area of ​​a preset width at the boundary of the defective region; and weighting and fusion of the repaired image block and the original image in the overlapping area according to the weight mask.

[0048] Since the apparatus described in Embodiment 2 of the present invention is an apparatus used to implement the method of Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.

[0049] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to Embodiment 1, as detailed in Embodiment 3. Example 3

[0050] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement any of the implementation methods in Embodiment 1.

[0051] Since the electronic device described in this embodiment is the device used to implement the method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in Embodiment 1 of this application. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection of this application.

[0052] Based on the same inventive concept, this application provides a storage medium corresponding to Embodiment 1, as detailed in Embodiment 4. Example 4

[0053] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can implement any of the implementation methods in Embodiment 1.

[0054] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0055] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0056] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0057] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0058] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A pattern defect detection and repair method, characterized by, Includes the following steps: Step S1: Preprocess the input image to obtain a preprocessed image; Step S2: Perform edge detection and connected component analysis on the preprocessed image. Determine a representative area based on the statistical distribution of connected component areas, and select connected components with areas similar to the representative area as candidate pattern units. For each candidate pattern unit, extract image data within its bounding rectangle region to form the feature vector of the candidate pattern unit. Calculate the average value of all candidate pattern unit feature vectors as the standard unit feature vector. Calculate the average adjacent spacing between all candidate pattern units. The average adjacent spacing is calculated as follows: First, select candidate unit pairs with a centroid distance less than a preset threshold; then, calculate the average distance between all selected unit pairs. Step S3: Calculate the feature similarity between the feature vector of each candidate pattern unit and the feature vector of the standard unit, and determine the candidate pattern units with feature similarity lower than the first preset threshold as first-class defect units; for each candidate pattern unit, determine its neighboring units as other units whose centroid distance from other units is within the predetermined floating range of the average adjacent spacing. The number of adjacent units of each unit is counted, and units with fewer than a second preset threshold are identified as second-type defective units. By fusing the positional information of the first type of defective unit and the second type of defective unit, a defective region mask is obtained; Step S4: The units in the candidate pattern units that do not belong to the first type of defect unit and the second type of defect unit are regarded as normal units; For each defective region, reference units are selected from normal units based on the average adjacent spacing and feature similarity. The defective region image and the average adjacent spacing are input into the pre-trained neural network model to generate repaired image patches; the repaired image patches are then fused with the original image.

2. The method of claim 1, wherein, The preprocessing in step S1 includes: converting the image from the BGR color space to the RGB color space, using a fast nonlocal mean algorithm for denoising, and normalizing the image pixel values.

3. The method of claim 1, wherein, In step S2, determining the representative area includes: statistically analyzing the distribution of the areas of all connected components, and determining the area value with the highest frequency in the distribution, or the center value of a preset area interval with the largest number of connected components, as the representative area. The term "screened connected regions with areas similar to the representative area" refers to connected regions whose screened areas deviate from the representative areas by less than ±20%. The step of extracting image data within its bounding rectangular region to form the feature vector of the unit specifically includes: converting the region into a grayscale image and flattening the grayscale pixel values ​​into a one-dimensional vector. The preset threshold is K times the square root of the representative area, where K is a predefined constant.

4. The method according to claim 1, characterized in that, In step S3, the similarity is cosine similarity, and the first preset threshold is 0.

85. The predetermined floating range is ±20% of the average adjacent spacing, and / or the second preset threshold is 2; The specific steps for fusing the positional information of the first and second types of defective units to obtain the defective region mask are as follows: marking the circumscribed rectangular region corresponding to the first type of defective unit and the circular region around the centroid of the second type of defective unit with a radius equal to the square root of the average adjacent spacing as the foreground in the mask image; performing a morphological dilation operation on the mask image to merge adjacent regions; and extracting connected regions with an area greater than a preset threshold through contour detection as the final defective region.

5. The method according to claim 1, characterized in that, In step S4, the selection of reference units includes: calculating the distance between the centroid of the normal unit and the defective region, and the similarity between the feature vector of the normal unit and the feature vector of the standard unit; calculating a weighted score based on the distance and similarity, and selecting the top K normal units with the highest scores as reference units; The pre-trained neural network model is a conditional generative adversarial network, and its generator has an encoder-decoder structure.

6. The method according to claim 5, characterized in that, The generator also includes a topological constraint layer for converting the average adjacent spacing scalar into a feature vector and fusing it with the features extracted by the encoder.

7. The method according to claim 1, characterized in that, The specific steps of fusing the repaired image block with the original image are as follows: at the boundary of the defective region, the repaired image block and the original image are weighted and mixed according to a preset gradient weight to achieve fusion; the weighted mixing according to the preset gradient weight includes: generating a weight mask that changes linearly from 0 to 1 in the overlapping area of ​​a preset width at the boundary of the defective region; and weighting and fusing the repaired image block and the original image in the overlapping area according to the weight mask.

8. A pattern defect detection and repair device, characterized in that, include: The preprocessing module preprocesses the input image to obtain a preprocessed image; The pattern parameter acquisition module performs edge detection and connected component analysis on the preprocessed image. Based on the statistical distribution of connected component areas, a representative area is determined, and connected components with areas close to the representative area are selected as candidate pattern units. For each candidate pattern unit, image data within its bounding rectangle is extracted to form the feature vector of the candidate pattern unit. The average value of the feature vectors of all candidate pattern units is calculated as the standard unit feature vector. The average adjacent spacing between all candidate pattern units is calculated as follows: first, candidate unit pairs with a centroid distance less than a preset threshold are selected; then, the average distance between all selected unit pairs is calculated. The defect area mask module calculates the feature similarity between the feature vector of each candidate pattern unit and the feature vector of the standard unit, and determines the candidate pattern unit with a feature similarity lower than a first preset threshold as a first type of defect unit; for each candidate pattern unit, it determines other units whose centroid distance from other units is within a predetermined floating range of the average adjacent spacing as its adjacent units. The number of adjacent units of each unit is counted, and units with fewer than a second preset threshold are identified as second-type defective units. By fusing the positional information of the first type of defective unit and the second type of defective unit, a defective region mask is obtained; The image repair module classifies the candidate pattern units that do not belong to the first type of defect unit and the second type of defect unit as normal units. For each defective region, reference units are selected from normal units based on the average adjacent spacing and feature similarity. The defective region image and the average adjacent spacing are input into the pre-trained neural network model to generate repaired image patches; the repaired image patches are then fused with the original image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images