A LCD defect detection method of small normal sample learning

Abstract

This invention discloses an LCD defect detection method based on a small number of normal samples. First, the design semantic guidance branch of the CLIP model provides distribution constraints for the global feature space. Then, the image encoder of the CLIP model is used to construct a visual prototype branch to perform weak affine transformations on a small number of normal samples to generate enhanced views and expand the retrieval memory. A K-nearest neighbor local aggregation mechanism is then used to retrieve the Top-K features in the memory. Next, the anomaly scores of the images to be detected are calculated for both the CLIP model semantic branch and the visual prototype branch. Finally, a weighted fusion anomaly score threshold decision is used to output the defect detection results and their localization. This method solves the problem of weak model generalization ability and unstable detection caused by the extreme scarcity of normal reference samples in the multi-variety, small-batch trial production stage of LCDs, significantly improving the defect perception capability of LCDs under extremely small sample settings.

Description

Technical Field

[0001] This invention belongs to the field of industrial intelligent vision inspection technology, specifically relating to an LCD defect detection method based on learning from a small number of normal samples. Background Technology

[0002] Liquid Crystal Display (LCD) panels serve as the intuitive interface for various smart terminals, and their yield rate is a key indicator of manufacturing process quality. In highly automated production lines, due to complex physical vapor deposition and fine photolithography processes, Mura defects, such as weak bright spots, minor open circuits, and large areas of low contrast, inevitably occur on the display surface. Existing industrial inspection paradigms heavily rely on feature modeling of massive amounts of defect images. This "fully supervised" approach requires detailed annotation of various anomalous samples before model deployment. However, during actual production line ramp-up or new product replacement phases, the emergence of defect samples is highly outlier and unpredictable, resulting in a vacuum in the sample library for an extremely long time window. This severe data asymmetry makes existing algorithms exhibit extremely poor adaptability when dealing with "cold start" scenarios.

[0003] To achieve accurate quality inspection in data-scarce industrial environments, Few-shot Anomaly Detection (FSAD) technology has become a breakthrough. This technology attempts to define the boundaries of a "normal manifold" within the feature space by learning from a very small number (1-8) of normal reference samples. Existing FSAD research has primarily followed two paths in its technological evolution. The first is the non-parametric method represented by PatchCore and SPADE, which focuses on building a memory bank by extracting features from normal images and performing nearest-neighbor retrieval. However, under small-sample conditions, the limited features extracted by such methods lead to an extremely sparse memory bank, which not only fails to effectively characterize the high-dimensional feature manifold of normal samples but is also highly susceptible to local noise. This sparsity and fragility directly result in a severe lack of robustness in the detection reference benchmark. The second approach is the cross-modal paradigm of pre-trained visual language models such as WinCLIP and APRIL-GAN. While leveraging image-text semantic priors to achieve strong generalization capabilities, it also suffers from an inherent "cross-modal granularity mismatch." General-purpose large models focus on global semantic alignment, making it difficult to capture the local fine-grained deviations (such as tiny scratches) that industrial detection relies on, resulting in a sluggish response to subtle anomalies. In addition, existing methods overemphasize text cue optimization and fail to fully explore the specific visual texture distribution information inherent in a very small number of normal samples. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an LCD defect detection method based on a small number of normal samples, which solves the problem of unstable modeling caused by the extreme scarcity of reference samples during the multi-variety, small-batch trial production stage of LCDs.

[0005] The technical solution adopted in this invention is: a method for detecting LCD defects by learning from a small number of normal samples, the specific steps of which are as follows:

[0006] S1. Acquire and preprocess data of LCD images;

[0007] The data acquisition and preprocessing includes: acquiring, cropping, and selecting sample images.

[0008] First, the display module sample to be tested is assembled. Then, multi-color gamut environmental data acquisition is carried out. That is, in a controlled darkroom, the display screen is driven to cycle through R, G, B monochrome, high and low grayscale, black and white cross and gradient background, and image sequences are captured using a high-precision industrial camera.

[0009] Then, image preprocessing is performed on the captured image sequence. Moiré patterns are suppressed by adjusting the pose and focal length parameters of the optical imaging system. For residual imaging noise, frequency domain filtering is performed using Fourier transform or wavelet transform to remove non-target artifacts.

[0010] After image preprocessing, sample selection is performed. Normal samples are selected from the original library, and the original image is resampled and divided into image blocks of fixed resolution to construct a standard tensor. Then, fine annotation is performed, that is, the selected data is annotated with defects at the pixel level.

[0011] Finally, the labeled data is divided into datasets. Y reference samples are used as the training set, and the other data are used as the test set, depending on the actual situation.

[0012] Among them, Y reference samples are a small number of normal samples, and the data in the test set are the images to be detected.

[0013] S2. Based on step S1, construct semantic guidance branches using test images in the test set;

[0014] The CLIP model is used to construct semantically guided branches, injecting category-independent general anomaly semantics into the feature space and providing global distribution constraints.

[0015] First, the pre-trained image encoder E1 in the CLIP model is used to extract features from the images to be detected in the test set, obtaining image patch features. Then, a dual-path text prompt set is constructed for the target category of the image to be detected. Based on the similarity measure between image patch features and text features in the shared space, anomaly detection is achieved, generating an anomaly heatmap for the semantic guidance network. .

[0016] S3. Based on step S1, construct a visual prototype branch and perform multi-view enhancement on the reference samples in the training set, that is, perform weak affine transformation on a small number of normal samples to generate enhanced views.

[0017] The visual prototype branch is constructed using the image encoder E2 of the CLIP model. That is, the deep visual features of a small number of normal samples are fully extracted and modeled using the CLIP image encoder to construct a set of visual prototypes with representative features.

[0018] First, a normality-preserving composite weak augmentation function is designed. This function is then used to augment Y reference samples in the training set with multiple views, generating a multi-view reference image sequence. Finally, an image encoder E2 based on a visual Transformer architecture is used to augment each enhanced view. Extract the preset total level The Patch Tokens are denoted as a set. .

[0019] Each of the original normal reference images generate A weakly enhanced view .

[0020] Then, using the pre-trained image encoder E2, deep and multi-level normal visual features are extracted from a small number of normal samples in the training set. Then, through aggregation or clustering methods, the extracted normal visual features are summarized into a set of representative "visual prototypes" as the benchmark of normal patterns. The constructed visual prototypes are stored to form a reference library.

[0021] S4. Based on step S3, construct a multi-view memory library and expand the feature manifold to cover the distribution blind spot;

[0022] The originally isolated feature observation points are expanded into a continuous normal manifold by stitching together all reference samples and their enhanced views at a specific feature layer. ∈ Based on local features, construct the final multi-view memory. The expression is as follows:

[0023] ;

[0024] The size of the memory bank is And all feature vectors in the memory have been processed Normalization process.

[0025] S5. Based on step S4, design a K-nearest neighbor prototype aggregation mechanism, retrieve the Top-K features in the multi-view memory, perform local aggregation, smooth the feature density estimation and calibrate the metric bias, and obtain the anomaly score map of the corresponding feature level.

[0026] S6. Based on step S5, perform multi-dimensional feature fusion decision-making to achieve LCD defect detection with a small number of normal samples.

[0027] First, the hierarchical score map A of the visual prototype branch is aggregated to obtain the anomaly map. Then, pixel-level anomaly localization is performed, and the anomaly heatmap of the semantically guided network is used. With anomaly graph Pixel-by-pixel addition and fusion are used to construct the final pixel-level positioning heatmap. Finally, image-level classification is performed, and the global probability of the semantic guidance branch and the image anomaly score of the visual prototype branch are fused. A weighted fusion anomaly score threshold decision is used to output the defect detection result and its location.

[0028] Furthermore, step S2 is specifically as follows:

[0029] S21. Use the pre-trained image encoder in the CLIP model to extract features from the images to be detected in the test set.

[0030] First, the global feature vector of the top layer of the pre-trained image encoder E1 in the CLIP model is extracted. Then, the image patch features of multiple preset intermediate layers in the image encoder E1 are extracted. By fusing multi-scale spatial information, high-resolution feature support is provided for pixel-level defect localization.

[0031] The extracted image patch features are then passed through a learnable linear layer to map the image patch features to the linear space containing the text features.

[0032] In the CLIP model, the text encoder encodes the text to obtain text features. The linear layer is the Multilayer Perceptron (MLP).

[0033] S22. Construct a dual-path text prompt set for the target category of the image to be detected;

[0034] A set of normal text templates describing a target in a normal state and a set of abnormal text templates describing a target with defects are preset. The text encoder in the CLIP model maps the template sets to a high-dimensional semantic space.

[0035] Then, mean pooling is performed on the feature vectors corresponding to all text templates in the "normal" and "abnormal" template sets respectively to calculate their average vectors. Then, L2 normalization is performed on the two average vectors to transform them into unit vectors.

[0036] The two normalized vectors form the semantic center anchors representing the "normal state" and the "abnormal state," respectively.

[0037] S23. Anomaly detection is achieved based on the similarity measurement between image patch features and text features in the shared space;

[0038] For image-level anomaly detection, the cosine similarity between global image patch features and normal / abnormal semantic center anchors is calculated, and then normalized using the Softmax function to obtain a global score representing the degree of anomaly across the entire image. .

[0039] For pixel-level localization scoring, multi-level image patch features are projected onto the semantic space through a linear mapping layer. The similarity between each pixel's positional features and normal / abnormal semantic center anchor points is calculated to generate multi-level semantic score maps. The semantic score maps generated at each level are then accumulated and fused in the spatial dimension to generate an anomaly heatmap for the semantic guidance network. .

[0040] Furthermore, step S3 is specifically as follows:

[0041] S31. Design a composite weak enhancement function that preserves normality;

[0042] The composite weak enhancement function is derived from geometric transformation Color disturbance With random horizontal flip Combining the above, the expression is as follows:

[0043] ;

[0044] in, This represents a composite weak enhancement function; geometric transformations include random rotation, translation, and scaling; color perturbations include low-amplitude adjustments to brightness, contrast, and saturation.

[0045] S32. Based on step S31, generate a multi-view reference image sequence;

[0046] The training set contains Y reference samples, i.e., Y reference images. For each original normal reference image... ,generate A weakly enhanced view , where the view =1 represents the original image, the rest... -1 view by Random sampling parameters are generated using the following expression:

[0047] ;

[0048] S33. Utilize the image encoder E2 based on the visual Transformer architecture for each enhanced view. Extract the preset total level The Patch Tokens are denoted as a set. ;

[0049] S34. Construct the visual prototype branch;

[0050] Using a pre-trained image encoder E2, deep and multi-level normal visual features are extracted from a small number of normal samples in the training set (Y). Then, through aggregation or clustering methods, the extracted normal visual features are summarized into a set of representative "visual prototypes" as the benchmark for normal patterns. The constructed visual prototypes are stored to form a reference library.

[0051] Furthermore, step S5 is specifically as follows:

[0052] S51. Calculate the cross-modal feature similarity matrix;

[0053] Extract the image patch feature set Q of the image under test at feature layer l, and calculate the relationship between each query patch q in Q and the multi-view memory. Cosine similarity of all prototypes The expression is as follows:

[0054] ;

[0055] S52. Retrieve the local nearest neighbor feature set m, based on the similarity matrix. The K feature vectors with the highest similarity to the query block q are retrieved from the memory and used to form a local reference set.

[0056] S53. Construct a robust local prototype;

[0057] Using median aggregation function Perform dimensional independent computation on the features in set m to generate the original prototype features. The expression is as follows:

[0058] ;

[0059] Then to conduct After normalization, the final robust local prototype is obtained. The expression is as follows:

[0060] ;

[0061] S54. Calculate pixel-level anomaly distance score;

[0062] Define the query block q and its corresponding robust local prototype. The cosine distance between them is the anomaly distance score d, which reflects the degree of deviation of the feature point from the local normal distribution center.

[0063] S55. Spatial reshaping of the anomaly distance scores d of all query blocks, and upsampling to the original image size using bilinear interpolation algorithm to obtain the anomaly score map A of this feature level.

[0064] Furthermore, step S6 is specifically as follows:

[0065] S61, Hierarchical score graph aggregation of visual prototype branches;

[0066] The hierarchical score maps A generated by the visual prototype branch are summed to obtain the anomaly map. .

[0067] S62, pixel-level anomaly location;

[0068] A semantic guidance graph with global semantic discrimination capabilities Compared with the prototype branch anomaly graph that provides precise local distance information Use weights Pixel-by-pixel addition and fusion are used to construct the final pixel-level positioning heatmap. The expression is as follows:

[0069] ;

[0070] S63, Image-level classification determination;

[0071] Extracting the location heatmap The maximum response value in the pool is used as the local score, i.e., max pooling is employed. This yields a robust anomaly score. And an adjustment coefficient is introduced to correlate it with the global semantic score. Weighted fusion is performed to calculate the final image-level anomaly detection score. The expression is as follows:

[0072] ;

[0073] in, ∈ (0, 1) represents the adjustment coefficient, i.e., the weight.

[0074] S64. Based on the relationship between the final judgment score and the preset threshold, output the classification attributes of the image to be tested and the spatial distribution of the defect area, so as to realize LCD defect detection by learning from a small number of normal samples.

[0075] The beneficial effects of this invention are as follows: First, the method utilizes the semantic guidance branch of the CLIP model to provide global feature space distribution constraints. Then, using the image encoder of the CLIP model, a visual prototype branch is constructed to perform weak affine transformations on a small number of normal samples to generate enhanced views, thereby expanding the retrieval memory. A K-nearest neighbor local aggregation mechanism is then used to retrieve the Top-K features in the memory. Next, the anomaly scores of the images to be detected are calculated for both the CLIP model semantic branch and the visual prototype branch. Finally, a weighted fusion anomaly score threshold decision is used to output the defect detection results and their localization. This invention provides global distribution constraints by injecting prior general anomaly knowledge into the semantic guidance branch. Combined with deep feature modeling and multi-view enhancement of a small number of normal samples using the visual prototype branch, it constructs a robust local normal pattern benchmark. Finally, through an innovative K-nearest neighbor prototype aggregation and multi-feature fusion mechanism, it solves the problem of weak model generalization ability and unrobust detection caused by the extreme scarcity of normal reference samples in the multi-variety, small-batch trial production stage of LCDs. This significantly improves the accuracy, localization precision, and model robustness of LCD defect detection under sample scarcity conditions. Attached Figure Description

[0076] Figure 1 This is a flowchart of an LCD defect detection method based on learning from a small number of normal samples according to the present invention.

[0077] Figure 2 This is a schematic diagram of a deep learning network model (CLIP model) in an embodiment of the present invention.

[0078] Figure 3 This is a schematic diagram of the detection results of four sample learning in an embodiment of the present invention. Detailed Implementation

[0079] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0080] like Figure 1 The flowchart of an LCD defect detection method based on a small number of normal samples is shown below. The specific steps are as follows:

[0081] S1. Acquire and preprocess data of LCD images;

[0082] The data acquisition and preprocessing includes: acquiring, cropping, and selecting sample images.

[0083] First, the display module sample to be tested is assembled. Then, multi-color gamut environmental data acquisition is carried out. That is, in a controlled darkroom, the display screen is driven to cycle through R, G, B monochrome, high and low grayscale (32 / 64 / 128 / 255), black and white cross and gradient background, and image sequences are captured using a high-precision industrial camera.

[0084] Then, image preprocessing is performed on the captured image sequence. Moiré patterns are suppressed by adjusting the pose and focal length parameters of the optical imaging system. For residual imaging noise, frequency domain filtering is performed using Fourier transform or wavelet transform to remove non-target artifacts.

[0085] After image preprocessing, sample selection is performed. Normal samples are selected from the original library, and the original image is resampled and divided into image blocks with a fixed resolution (1000×1000 in this embodiment) to construct a standard tensor. Then, fine annotation is performed, that is, the selected data is annotated with defects at the pixel level.

[0086] Finally, the labeled data is divided into datasets. Y reference samples are used as the training set, and the other data are used as the test set, depending on the actual situation.

[0087] Here, Y reference samples are a small number of normal samples. The test set data consists of the images to be detected (test images). In this embodiment, Y∈{1, 2, 4, 8}.

[0088] S2. Based on step S1, construct semantic guidance branches using test images in the test set;

[0089] The CLIP model is used to construct semantically guided branches, injecting category-independent general anomaly semantics into the feature space to provide global distribution constraints. In this embodiment, the CLIP model is as follows: Figure 2 As shown.

[0090] First, the pre-trained image encoder E1 in the CLIP model is used to extract features from the test images (test pictures) in the test set to obtain image patch features. Then, a dual-path text prompt set is constructed for the target category of the image to be detected. Based on the similarity measure between the image patch features (visual features) and text features in the shared space, anomaly detection is achieved, generating an anomaly heatmap for the semantic guidance network. .

[0091] S3. Based on step S1, construct a visual prototype branch and perform multi-view enhancement on the reference samples in the training set, that is, perform weak affine transformation on a small number of normal samples to generate enhanced views.

[0092] The visual prototype branch is constructed using the image encoder E2 of the CLIP model. That is, the deep visual features of a small number of normal samples are fully extracted and modeled using the CLIP image encoder to construct a set of visual prototypes with representative features.

[0093] First, a normality-preserving composite weak augmentation function is designed. This function is then used to augment Y reference samples in the training set with multiple views, generating a multi-view reference image sequence. Finally, an image encoder E2 based on the Visual Transformer (VIT) architecture is used to augment each enhanced view. Extract the preset total level The Patch Tokens are denoted as a set. .

[0094] Each of the original normal reference images generate A weakly enhanced view .

[0095] Then, using the pre-trained image encoder E2, deep and multi-level normal visual features are extracted from a small number of normal samples in the training set (Y samples). These extracted normal visual features are then aggregated or clustered into a representative set of "visual prototypes" as a baseline for normal patterns. The constructed visual prototypes are stored to form a reference library, which is used for subsequent local similarity comparisons with test samples.

[0096] S4. Based on step S3, construct a multi-view memory library and expand the feature manifold to cover the distribution blind spot;

[0097] The multi-view memory is used to address the problem of incomplete feature distribution in small sample scenarios. It explicitly constructs multiple transformation views of normal samples and performs dense sampling of the normal distribution in the feature space.

[0098] The originally isolated feature observation points are expanded into a continuous normal manifold by stitching together all reference samples and their enhanced views at a specific feature layer. ∈ Based on local features, construct the final multi-view memory. The expression is as follows:

[0099] ;

[0100] The size of the memory bank is And all feature vectors in the memory have been processed Normalization is applied as a robust benchmark for subsequent anomaly detection.

[0101] S5. Based on step S4, design a K-nearest neighbor prototype aggregation mechanism, retrieve the Top-K features in the multi-view memory, perform local aggregation, smooth the feature density estimation and calibrate the metric bias, and obtain the anomaly score map of the corresponding feature level.

[0102] An anomaly metric based on K-nearest neighbor prototype aggregation is implemented to address the problem that traditional nearest neighbor search is susceptible to interference from outliers in the memory. A robust prototype is constructed through local feature fusion to establish an interference-resistant discrimination benchmark.

[0103] S6. Based on step S5, perform multi-dimensional feature fusion decision-making to achieve LCD defect detection with a small number of normal samples.

[0104] First, the hierarchical score map A of the visual prototype branch is aggregated to obtain the anomaly map. Then, pixel-level anomaly localization is performed, and the anomaly heatmap of the semantically guided network is used. With anomaly graph Pixel-by-pixel addition and fusion are used to construct the final pixel-level positioning heatmap. Finally, image-level classification is performed, and the global probability of the semantic guidance branch and the image anomaly score of the visual prototype branch are fused. A weighted fusion anomaly score threshold decision is used to output the defect detection result and its location.

[0105] In this embodiment, step S2 is specifically as follows:

[0106] S21. Use the pre-trained image encoder in the CLIP model to extract features from the images to be detected (test images) in the test set;

[0107] First, the global feature vector of the top layer of the pre-trained image encoder E1 (the CLIP image encoder of the ViT architecture) in the CLIP model is extracted to represent the overall semantic information of the image and achieve preliminary image-level classification.

[0108] In this embodiment, image block features from the preset intermediate layers {6, 12, 18, 24} in the image encoder E1 are extracted. By fusing multi-scale spatial information, high-resolution feature support is provided for pixel-level defect localization.

[0109] The extracted image patch features are then passed through a learnable linear layer to map the image patch features to the linear space containing the text features.

[0110] In the CLIP model, the text encoder encodes the text to obtain text features. The linear layer is the Multilayer Perceptron (MLP).

[0111] S22. Construct a dual-path text prompt set for the target category of the image to be detected;

[0112] A set of normal text templates describing a target in a normal state and a set of abnormal text templates describing a target with defects are preset. The text encoder in the CLIP model maps the template sets to a high-dimensional semantic space.

[0113] The normal text template set describing the target as being in a normal state includes: "Undamaged [Category]", "This is a good [Category]"; the abnormal text template set describing defects includes: "Damaged [Category]", "This is a scratched [Category]".

[0114] Then, mean pooling is performed on the feature vectors corresponding to all text templates in the "normal" and "abnormal" template sets respectively, and their average vectors are calculated to aggregate the overall semantic information of the category.

[0115] The two average vectors are then subjected to L2 normalization to transform them into unit vectors. These two normalized vectors constitute the semantic center anchors representing the "normal state" and "abnormal state," respectively, for subsequent similarity measurement.

[0116] S23. Anomaly detection is achieved based on the similarity measurement between image patch features (visual features) and text features in the shared space;

[0117] For image-level anomaly detection, the cosine similarity between global image patch features and normal / abnormal semantic center anchor points (text features) is calculated, and then normalized using the Softmax function to obtain a global score representing the degree of anomaly across the entire image. .

[0118] For pixel-level localization scoring, multi-level image patch features are projected onto the semantic space through a linear mapping layer. The similarity between each pixel location feature and the normal / abnormal semantic center anchor point (text feature) is calculated to generate a multi-level semantic score map. The semantic score maps generated at each level are then accumulated and fused in the spatial dimension to generate an anomaly heatmap for the semantic guidance network. It is used for visual prototype branch fusion decisions.

[0119] In this embodiment, step S3 is specifically as follows:

[0120] S31. Design a composite weak enhancement function that preserves normality;

[0121] The composite weak enhancement function is used to introduce diversity without disrupting the normal feature distribution; it is derived from geometric transformations. Color disturbance With random horizontal flip Combining the above, the expression is as follows:

[0122] ;

[0123] in, This represents a composite weak enhancement function; geometric transformations include random rotation (≤10°), translation (≤5%), and scaling (0.9 to 1.1); color perturbations include low-amplitude adjustments to brightness, contrast, and saturation.

[0124] S32. Based on step S31, generate a multi-view reference image sequence;

[0125] The training set contains Y reference samples, i.e., Y reference images. For each original normal reference image... ,generate A weakly enhanced view , where the view =1 represents the original image, the rest... -1 view by Random sampling parameters are generated using the following expression:

[0126] ;

[0127] S33. Using the image encoder E2 based on the Visual Transformer (VIT) architecture, for each enhanced view... Extract the preset total level The Patch Tokens are denoted as a set. ;

[0128] S34. Construct the visual prototype branch;

[0129] Using a pre-trained image encoder E2, deep and multi-level normal visual features are extracted from a small number of normal samples in the training set (Y). Then, through aggregation (including averaging) or clustering methods, the extracted normal visual features are summarized into a set of representative "visual prototypes" as the benchmark for normal patterns. The constructed visual prototypes are stored to form a reference library for subsequent local similarity comparison with test samples.

[0130] In this embodiment, step S5 is specifically as follows:

[0131] S51. Calculate the cross-modal feature similarity matrix;

[0132] Extract the image patch feature set Q of the image under test at feature layer l, and calculate the relationship between each query patch q in Q and the multi-view memory. Cosine similarity of all prototypes The expression is as follows:

[0133] ;

[0134] S52. Retrieve the local nearest neighbor feature set m, based on the similarity matrix. The K feature vectors with the highest similarity to the query block q are retrieved from the memory and used to form a local reference set.

[0135] S53. Construct a robust local prototype;

[0136] Using median aggregation function Dimensionality-independent computation is performed on the features in set m to suppress outlier interference and generate original prototype features. The expression is as follows:

[0137] ;

[0138] Then to conduct After normalization, the final robust local prototype is obtained. The expression is as follows:

[0139] ;

[0140] S54. Calculate pixel-level anomaly distance score;

[0141] Define the query block q and its corresponding robust local prototype. The cosine distance between them is the anomaly distance score d, which reflects the degree of deviation of the feature point from the local normal distribution center.

[0142] S55. Spatial reshaping of the anomaly distance scores d of all query blocks, and upsampling to the original image size using bilinear interpolation algorithm to obtain the anomaly score map A of this feature level.

[0143] In this embodiment, step S6 is specifically as follows:

[0144] S61, Hierarchical score graph aggregation of visual prototype branches;

[0145] The hierarchical score maps A generated by the visual prototype branch are summed to obtain the anomaly map. .

[0146] S62, pixel-level anomaly location;

[0147] A semantic guidance graph with global semantic discrimination capabilities Compared with the prototype branch anomaly graph that provides precise local distance information Use weights Pixel-by-pixel addition and fusion are used to construct the final pixel-level positioning heatmap. The expression is as follows:

[0148] ;

[0149] S63, Image-level classification determination;

[0150] Extracting the location heatmap The maximum response value in the pool is used as the local score, i.e., max pooling is employed. This yields a robust anomaly score. And an adjustment coefficient (i.e., weighting) is introduced. ∈ (0, 1) and compare it with the global semantic score Weighted fusion is performed to calculate the final image-level anomaly detection score. The expression is as follows:

[0151] ;

[0152] S64. Based on the relationship between the final judgment score and the preset threshold, output the classification attributes (normal or abnormal) of the image to be tested and the spatial distribution of the defect area, so as to realize LCD defect detection by learning from a small number of normal samples.

[0153] The detection results of the 4-sample learning in this embodiment are as follows: Figure 3 As shown, from Figure 3 It can be seen that the method of the present invention can still effectively detect display screen defects even when only 4 normal samples are used for learning.

[0154] In summary, the method of this invention provides global distribution constraints by injecting prior general anomaly knowledge through semantically guided branches, and combines visual prototype branches to perform deep feature modeling and multi-view enhancement on a small number of normal samples to build a robust local normal pattern benchmark. Finally, through an innovative K-nearest neighbor prototype aggregation and multi-feature fusion mechanism, it solves the problems of weak model generalization ability and unrobust detection caused by the extreme scarcity of normal reference samples in the multi-variety, small-batch trial production stage of LCDs. It significantly improves the accuracy of LCD defect detection, positioning accuracy and model robustness under the condition of scarce samples.

[0155] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A method for detecting LCD defects using a small number of normal samples, the specific steps of which are as follows: S1. Acquire and preprocess data of LCD images; in, The data acquisition and preprocessing includes: acquisition, cropping, and selection of sample images; First, assemble the display module sample to be tested, and then collect multi-color gamut environmental data. That is, in a controlled darkroom, drive the display screen to cycle through R, G, B monochrome, high and low grayscale, black and white cross and gradient background, and use a high-precision industrial camera to capture image sequences. Then, the captured image sequence is preprocessed. Moiré patterns are suppressed by adjusting the pose and focal length parameters of the optical imaging system. For residual imaging noise, frequency domain filtering is performed using Fourier transform or wavelet transform to remove non-target artifacts. After image preprocessing, sample selection is performed. Normal samples are selected from the original library, and the original image is resampled and divided into image blocks of fixed resolution to construct a standard tensor. Then, fine annotation is performed, that is, the defect is annotated at the pixel level on the selected data. Finally, the labeled data is divided into datasets. Y reference samples are used as the training set, and the other data are used as the test set, depending on the actual situation. Among them, Y reference samples are a small number of normal samples, and the data in the test set are the images to be detected; S2. Based on step S1, construct semantic guidance branches using test images in the test set; The CLIP model is used to construct semantic guidance branches, injecting category-independent general anomaly semantics into the feature space to provide global distribution constraints; First, the pre-trained image encoder E1 in the CLIP model is used to extract features from the images to be detected in the test set, obtaining image patch features. Then, a dual-path text prompt set is constructed for the target category of the image to be detected. Based on the similarity measure between image patch features and text features in the shared space, anomaly detection is achieved, generating an anomaly heatmap for the semantic guidance network. ; S3. Based on step S1, construct a visual prototype branch and perform multi-view enhancement on the reference samples in the training set, that is, perform weak affine transformation on a small number of normal samples to generate enhanced views. The visual prototype branch is constructed using the image encoder E2 of the CLIP model. That is, the deep visual features of a small number of normal samples are fully extracted and modeled using the CLIP image encoder to construct a set of visual prototypes with representative features. First, a normality-preserving composite weak augmentation function is designed. This function is then used to augment Y reference samples in the training set with multiple views, generating a multi-view reference image sequence. Finally, an image encoder E2 based on a visual Transformer architecture is used to augment each enhanced view. Extract the preset total level The Patch Tokens are denoted as a set. ; Each of the original normal reference images generate A weakly enhanced view ; Then, using the pre-trained image encoder E2, deep and multi-level normal visual features are extracted from a small number of normal samples in the training set. Then, through aggregation or clustering methods, the extracted normal visual features are summarized into a set of representative "visual prototypes" as the benchmark of normal patterns. The constructed visual prototypes are stored to form a reference library. S4. Based on step S3, construct a multi-view memory library and expand the feature manifold to cover the distribution blind spot; The originally isolated feature observation points are expanded into a continuous normal manifold by stitching together all reference samples and their enhanced views at a specific feature layer. ∈ Based on local features, construct the final multi-view memory. The expression is as follows: ； The size of the memory bank is And all feature vectors in the memory have been processed Normalization processing; S5. Based on step S4, design a K-nearest neighbor prototype aggregation mechanism, retrieve the Top-K features in the multi-view memory, perform local aggregation, smooth the feature density estimation and calibrate the metric bias, and obtain the anomaly score map of the corresponding feature level. S6. Based on step S5, perform multi-dimensional feature fusion decision-making to achieve LCD defect detection with a small number of normal samples. First, the hierarchical score map A of the visual prototype branch is aggregated to obtain the anomaly map. Then, pixel-level anomaly localization is performed, and the anomaly heatmap of the semantically guided network is used. With anomaly graph Pixel-by-pixel addition and fusion are used to construct the final pixel-level positioning heatmap. Finally, image-level classification is performed, and the global probability of the semantic guidance branch and the image anomaly score of the visual prototype branch are fused. A weighted fusion anomaly score threshold decision is used to output the defect detection result and its location.

2. The LCD defect detection method based on a small number of normal samples according to claim 1, characterized in that, Step S2 is as follows: S21. Use the pre-trained image encoder in the CLIP model to extract features from the images to be detected in the test set. First, the global feature vector of the top layer of the pre-trained image encoder E1 in the CLIP model is extracted. Then, the image patch features of multiple preset intermediate layers in the image encoder E1 are extracted. By fusing multi-scale spatial information, high-resolution feature support is provided for pixel-level defect localization. Then, the extracted image patch features are passed through a learnable linear layer to map the image patch features to the linear space where the text features reside. In the CLIP model, the text encoder encodes the text to obtain text features; the linear layer is the multilayer perceptron (MLP). S22. Construct a dual-path text prompt set for the target category of the image to be detected; A set of normal text templates describing a target in a normal state and a set of abnormal text templates describing a target with defects are preset; the template sets are mapped to a high-dimensional semantic space through the text encoder in the CLIP model; Then, mean pooling is performed on the feature vectors corresponding to all text templates in the "normal" and "abnormal" template sets respectively to calculate their average vectors. Then, L2 normalization is performed on the two average vectors to convert them into unit vectors. Among them, the two normalized vectors constitute the semantic center anchors representing the "normal state" and the "abnormal state", respectively. S23. Anomaly detection is achieved based on the similarity measurement between image patch features and text features in the shared space; For image-level anomaly detection, the cosine similarity between global image patch features and normal / abnormal semantic center anchors is calculated, and then normalized using the Softmax function to obtain a global score representing the degree of anomaly across the entire image. ; For pixel-level localization scoring, multi-level image patch features are projected onto the semantic space through a linear mapping layer. The similarity between each pixel's positional features and normal / abnormal semantic center anchor points is calculated to generate multi-level semantic score maps. The semantic score maps generated at each level are then accumulated and fused in the spatial dimension to generate an anomaly heatmap for the semantic guidance network. .

3. The LCD defect detection method based on a small number of normal samples according to claim 1, characterized in that, Step S3 is as follows: S31. Design a composite weak enhancement function that preserves normality; The composite weak enhancement function is derived from geometric transformation Color disturbance With random horizontal flip Combining the above, the expression is as follows: ； in, This represents a composite weak enhancement function; geometric transformations include random rotation, translation, and scaling; color perturbations include low-amplitude adjustments to brightness, contrast, and saturation. S32. Based on step S31, generate a multi-view reference image sequence; The training set contains Y reference samples, i.e., Y reference images. For each original normal reference image... ,generate A weakly enhanced view , where the view =1 represents the original image, the rest... -1 view by Random sampling parameters are generated using the following expression: ； S33. Utilize the image encoder E2 based on the visual Transformer architecture for each enhanced view. Extract the preset total level The Patch Tokens are denoted as a set. ; S34. Construct the visual prototype branch; Using a pre-trained image encoder E2, deep and multi-level normal visual features are extracted from a small number of normal samples in the training set (Y). Then, through aggregation or clustering methods, the extracted normal visual features are summarized into a set of representative "visual prototypes" as the benchmark for normal patterns. The constructed visual prototypes are stored to form a reference library.

4. The LCD defect detection method based on a small number of normal samples according to claim 1, characterized in that, Step S5 is as follows: S51. Calculate the cross-modal feature similarity matrix; Extract the image patch feature set Q of the image under test at feature layer l, and calculate the relationship between each query patch q in Q and the multi-view memory. Cosine similarity of all prototypes The expression is as follows: ； S52. Retrieve the local nearest neighbor feature set m, based on the similarity matrix. The K feature vectors with the highest similarity to the query block q are retrieved from the memory and used to form a local reference set. S53. Construct a robust local prototype; Using median aggregation function Perform dimensional independent computation on the features in set m to generate the original prototype features. The expression is as follows: ； Then to conduct After normalization, the final robust local prototype is obtained. The expression is as follows: ； S54. Calculate pixel-level anomaly distance score; Define the query block q and its corresponding robust local prototype. The cosine distance between them is the anomaly distance score d, which reflects the degree of deviation of the feature point from the local normal distribution center; S55. Spatial reshaping of the anomaly distance scores d of all query blocks, and upsampling to the original image size using bilinear interpolation algorithm to obtain the anomaly score map A of this feature level.

5. The LCD defect detection method based on a small number of normal samples according to claim 1, characterized in that, Step S6 is as follows: S61, Hierarchical score graph aggregation of visual prototype branches; The hierarchical score maps A generated by the visual prototype branch are summed to obtain the anomaly map. ; S62, pixel-level anomaly location; A semantic guidance graph with global semantic discrimination capabilities Compared with the prototype branch anomaly graph that provides precise local distance information Use weights Pixel-by-pixel addition and fusion are used to construct the final pixel-level positioning heatmap. The expression is as follows: ； S63, Image-level classification determination; Extracting the location heatmap The maximum response value in the pool is used as the local score, i.e., max pooling is employed. This yields a robust anomaly score. ; An adjustment coefficient is introduced and used in conjunction with the global semantic score. Weighted fusion is performed to calculate the final image-level anomaly detection score. The expression is as follows: ； in, ∈ (0, 1) represents the adjustment coefficient, i.e., the weight; S64. Based on the relationship between the final judgment score and the preset threshold, output the classification attributes of the image to be tested and the spatial distribution of the defect area, so as to realize LCD defect detection by learning from a small number of normal samples.

Images