An image-based screen defect detection method, system and apparatus

Abstract

The application discloses an image-based screen defect detection method, system and device, relates to the technical field of computer vision, and solves the technical problem that the prior art often adopts a general image enhancement or text image model to perform sample expansion, so that the expanded sample cannot reflect the physical form and distribution rule of defects under real production conditions, and the accuracy and generalization performance of a screen anomaly detection model are low; a screen image is subjected to an abnormal area identification model to obtain an area result; when the area label is an existing abnormal label, an abnormal area image corresponding to the existing abnormal label is subjected to a defect identification model to obtain a defect result, a process parameter of screen manufacturing is taken as a generation condition, and a data generation model is combined to expand defect samples, so that the abnormal area identification model and the defect identification model trained based on the data set can learn the internal correlation between defects and process parameters, and the identification accuracy and generalization performance of the model for screen defects are improved.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, specifically an image-based screen defect detection method, system, and device. Background Technology

[0002] Against the backdrop of rapid development in modern electronic manufacturing and display technology, the screen, as the core interface of human-computer interaction, directly impacts user experience and product reliability. Image-based screen defect detection technology refers to an intelligent quality inspection method that uses high-resolution imaging equipment to acquire images of the screen surface and combines computer vision and artificial intelligence algorithms to automatically identify various microscopic or macroscopic defects such as black screens and abnormal displays. It overcomes the problems of fatigue, subjectivity, and high missed detection rates associated with traditional manual visual inspection, ensuring high-quality standards for products leaving the factory. More importantly, it promotes the evolution of intelligent manufacturing towards high precision, high automation, and high intelligence, providing key technological support for the independent control and international competitiveness of the high-end display industry chain.

[0003] In screen manufacturing, the yield rate is usually very high, resulting in extremely scarce and unbalanced real defect samples available for training anomaly detection models. Existing technologies often use general image enhancement techniques or general text-based image models to expand the samples. Although they are visually diverse, they lack physical rationality and cannot reflect the physical morphology and distribution patterns of defects under real production conditions, resulting in low accuracy and generalization performance of screen anomaly detection models. Therefore, image-based screen defect detection methods still need further improvement. Summary of the Invention

[0004] This application aims to solve at least one of the technical problems existing in the prior art; to this end, this application proposes an image-based screen defect detection method, system and device to solve the technical problem that the prior art often uses general image enhancement or text image models to expand the samples, which makes the expanded samples lack physical rationality and cannot reflect the physical morphology and distribution law of defects under real production conditions, resulting in low accuracy and generalization performance of the screen anomaly detection model.

[0005] To achieve the above objectives, a first aspect of this application provides an image-based screen defect detection method, comprising:

[0006] Acquire the screen image to be detected;

[0007] The screen image is analyzed using an abnormal region identification model to obtain region results; the region results include region labels and region data; the region labels include labels indicating the presence of abnormalities and labels indicating the absence of abnormalities; the region data includes images of abnormal regions and images of normal regions;

[0008] When the region label is "existing abnormal label", extract the abnormal region image corresponding to the "existing abnormal label";

[0009] The defect results are obtained by using an abnormal region image to identify defects through a defect identification model.

[0010] The training data used to train the abnormal region identification model and the defect identification model are constructed from the screen defect dataset; the screen defect dataset consists of several historical screen images and expanded screen images that are data-enhanced by process parameters.

[0011] In constructing the training datasets for the abnormal area identification model and the defect identification model through the above steps, this application incorporates key process parameters from the screen manufacturing process as conditional information into the data generation model. The expanded defect samples generated in this way are highly consistent with the failure modes that occur in actual production in terms of physical mechanism and morphological distribution. The identification model trained based on such high-quality, high-fidelity synthetic data can effectively learn the inherent correlation between defect features and process parameters, thereby improving the accuracy and generalization ability of identifying various screen defects.

[0012] Furthermore, the training data for the abnormal region identification model and the defect identification model include:

[0013] Extract several screen images and their corresponding region results from the screen defect dataset, and use these screen images and their corresponding region results as training data for the abnormal region recognition model; the region results include region labels and region data; the region labels include labels indicating the presence of abnormalities and labels indicating the absence of abnormalities; the region data includes abnormal region images and normal region images;

[0014] Extract data from several regions and their corresponding defect results, and use the data from several regions and their corresponding defect results as training data for the defect identification model; the defect results include defect labels and defect types; the defect labels include labels indicating the presence of defects and labels indicating the absence of defects.

[0015] Furthermore, the screen defect dataset consists of several historical screen images and augmented screen images enhanced by process parameters, including:

[0016] Obtain the manufacturing process parameter set corresponding to the historical screen image with defective labels; the manufacturing process parameter set refers to the process parameters when producing the corresponding historical screen image, including substrate material type, etching accuracy, thin film deposition thickness, packaging temperature, packaging pressure and driving voltage range;

[0017] A process condition vector is obtained by standardizing and vectorizing several process parameters in the manufacturing process parameter set.

[0018] Vectorize the defect results with defect labels to obtain text condition vectors;

[0019] Historical screen images, process condition vectors, and text condition vectors are input into the data augmentation model to obtain augmented screen images and their corresponding defect results. The data augmentation model is constructed using an improved Stable-Diffusion model to generate screen images with defect labels, thereby improving the accuracy of screen anomaly detection and classification.

[0020] Several historical screen images and extended screen images, as well as the corresponding defect results of several historical screen images and extended screen images, are integrated into a screen defect dataset.

[0021] Furthermore, the data augmentation model includes:

[0022] Extract the U-Net denoising network from the Stable-Diffusion model;

[0023] A process condition fusion module is embedded before several residual blocks in the U-Net denoising network; the process condition fusion module is used to introduce process condition vectors and combine them with text condition vectors to guide the screen image in the denoising process.

[0024] The process condition fusion module consists of an input layer, a condition fusion unit, a feature modulation unit, and an output layer.

[0025] The input layer is used to receive historical screen images, process condition vectors, and text condition vectors.

[0026] The condition fusion unit is used to perform a fusion operation on the process condition vector and the text condition vector to obtain a fused condition vector; the fusion operation is implemented through a cross-attention mechanism, with the text condition vector as the query and the process condition vector as the key-value pair.

[0027] The feature modulation unit consists of a cross-attention layer, an adaptive feature modulation layer, and a conditional gating layer;

[0028] The cross-attention layer adopts a cross-attention mechanism, using the image features corresponding to the historical screen images as queries and the fusion condition vector as key-value pairs to perform a fusion operation to obtain the fused feature vector;

[0029] The adaptive feature modulation layer is used to transform the fusion condition vector into scaling and offset parameters; and to modulate the image features using the scaling and offset parameters to obtain modulated image features;

[0030] The conditional gating layer is used to convert the process condition vector into gating weights, and to use the gating weights to perform gating operations on the modulated image features to obtain gating image features.

[0031] The output layer is used to concatenate the gated image features with the image features corresponding to the historical screen images to obtain the image features after conditional modulation and gating.

[0032] In constructing the data augmentation model, this application encodes specific screen manufacturing process parameters into process condition vectors, which, together with text condition vectors describing defect semantics, serve as condition inputs, introducing an improved Stable Diffusion generation model. Simultaneously, a process condition fusion module is designed within the U-Net denoising network of this model, deeply fusing process and text conditions through a cross-attention mechanism. Furthermore, by combining adaptive feature modulation and process gating operations, refined and dynamic control of denoising behavior is achieved at each step of the diffusion process. The resulting augmented screen defect images are highly consistent with real-world process defects in terms of both physical morphology and failure mechanisms, fundamentally ensuring the physical rationality and process interpretability of the synthesized samples. Based on this high-fidelity, highly semantically consistent training data, the trained anomaly region recognition model and defect recognition model can effectively capture the inherent causal relationship between defects and manufacturing parameters, significantly improving the accuracy of identifying complex and diverse real defects and its generalization ability across different production lines.

[0033] Furthermore, the abnormal region identification model includes an input layer, a backbone network, a dynamic attention network guided by screen geometry priors, a neck network, and a detection head;

[0034] The input layer is used to receive the screen image to be detected;

[0035] The backbone network is used to extract multi-scale feature maps of screen images;

[0036] The screen geometry prior-guided dynamic attention network is used to receive multi-scale feature maps output by the backbone network and introduce the screen geometry prior knowledge to generate corresponding attention weight maps. The attention weight maps are then used to process the multi-scale feature maps to obtain several enhanced feature maps.

[0037] The neck network is used to fuse several enhanced feature maps to obtain a fused feature map;

[0038] The detection head is used to analyze the fused feature map to output the region result;

[0039] The screen geometry prior-guided dynamic attention network is located between the backbone network and the neck network.

[0040] Furthermore, the screen geometry prior-guided dynamic attention network includes a screen prior encoding module, a geometric attention weight generation module, and a dynamic feature recalibration module.

[0041] The screen prior encoding module is used to learn the geometric prior knowledge of the screen, rather than using a fixed template; the input data of the screen prior encoding module is the multi-scale feature map output by the backbone network, and the output data is the screen prior parameter vector, which is used to guide the generation of subsequent attention weight maps.

[0042] The geometric attention weight generation module is used to generate an attention weight map based on the screen prior parameter vector to enhance the feature response of possible screen regions; the input data of the geometric attention weight generation module is a multi-scale feature map and a screen prior parameter vector; the output data is the attention weight map.

[0043] The dynamic feature recalibration module is used to combine the attention weight map with the multi-scale feature map to achieve spatial and channel-level feature recalibration; the dynamic feature recalibration module is implemented through spatial attention modulation and channel attention modulation; the input data of the dynamic feature recalibration module is the attention weight map and the multi-scale feature map, and the output data is the enhanced feature map.

[0044] This application embeds a screen geometry prior-guided dynamic attention network between the backbone and neck network of the anomaly region identification model. This network consists of a screen prior encoding module, a geometric attention weight generation module, and a dynamic feature recalibration module. It can autonomously learn the geometric structure prior of the screen and generate an attention weight map accordingly. It performs adaptive recalibration of the feature map at the spatial and channel levels, thereby guiding the model to focus on the screen region and suppressing background interference. This enables the anomaly region identification model to have screen structure perception capabilities, improves the accuracy and robustness of locating screen regions in complex industrial scenarios, and reduces the risk of false positives and false negatives in subsequent defect classification.

[0045] Furthermore, the defect identification model is constructed using an improved Swing Transformer model;

[0046] The Swin Transformer model consists of several stages, a normalization layer, a global average pooling layer, and a classification head. The stages include stage one, stage two, stage three, and stage four.

[0047] The stage consists of a linear embedding layer, a downsampling layer, and several stacked Swing Transformer Blocks.

[0048] Add a CBAM module after the last Swin Transformer Block in Phase 3; the CBAM module includes a channel attention module and a spatial attention module; the channel attention module is located before the spatial attention module.

[0049] Increase the number of Swing Transformer Blocks in Phase 3 to N; where N is an integer, N>6;

[0050] The normalization layer is located before the global pooling layer and is used to normalize the features and stabilize the feature distribution.

[0051] Furthermore, the step of obtaining defect results by performing defect identification on the abnormal region image through a defect identification model includes:

[0052] S1: Extract the image of the abnormal region;

[0053] S2: The abnormal region map is subjected to high-resolution feature extraction in stage one to obtain high-resolution image features;

[0054] S3: The high-resolution image features are extracted into medium-resolution image features through stage two;

[0055] S4: The medium-resolution image features are processed through stage three to obtain image depth features;

[0056] S5: The image depth features are extracted using advanced semantic feature extraction in stage four to obtain image semantic features;

[0057] S6: The high-level semantic features are passed through a normalization layer, a global average pooling layer, and a classification head in sequence to perform feature normalization and defect type prediction operations to obtain the defect results.

[0058] A second aspect of the present invention provides an image-based screen defect detection system, comprising: a data acquisition module and a data analysis module; the data acquisition module and the data analysis module are connected together;

[0059] The data acquisition module acquires the screen image to be detected through a data acquisition device;

[0060] The data analysis module includes a region analysis unit and a defect analysis unit;

[0061] The region analysis unit: performs region analysis on the screen image using an abnormal region identification model to obtain region results;

[0062] The defect analysis unit: uses a defect recognition model to identify defects in abnormal region images to obtain defect results.

[0063] Another aspect of the present invention provides an image-based screen defect detection device, comprising: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the method as described in the first aspect and any possible implementation thereof; the image-based screen defect detection device may be an electronic device or a chip in an electronic device.

[0064] Compared with the prior art, the beneficial effects of this application are:

[0065] 1. This application obtains regional results by performing regional analysis on screen images through an anomaly region recognition model; when the region label is an anomaly label, the anomaly region image corresponding to the anomaly label is extracted, and the anomaly region image is used to perform defect recognition through a defect recognition model to obtain defect results. When constructing the training dataset for the anomaly region recognition model and the defect recognition model, key process parameters in the screen manufacturing process are used as generation conditions and input into the data generation model to generate expanded defect samples that are highly consistent with the actual production failure modes in terms of physical mechanism and morphological distribution. This enables the anomaly region recognition model and the defect recognition model trained based on this dataset to learn the inherent correlation between defects and process parameters, thereby improving the model's recognition accuracy and generalization performance for various screen defects.

[0066] 2. When constructing the data augmentation model in this application, specific screen manufacturing process parameters are encoded into process condition vectors, which are then input together with text condition vectors describing defects into an improved Stable-Diffusion model. A dedicated process condition fusion module is embedded in the U-Net denoising network of the model. Through a cross-attention mechanism, the process and text conditions are deeply fused, and adaptive feature modulation and process gating operations are used to precisely control each step of the diffusion process for denoising. This generates an augmented screen image that is highly consistent with the real process defects in terms of physical morphology and failure mechanism, fundamentally ensuring the physical rationality and process interpretability of the generated defect samples. This enables the abnormal region recognition model and defect recognition model trained on this dataset to learn the inherent causal relationship between defects and manufacturing parameters, thereby improving the model's accuracy in recognizing complex real defects and its generalization performance across production lines.

[0067] 3. When constructing the defect recognition model, this application makes targeted improvements to the standard Swing Transformer. Specifically, it significantly increases the number of Swing Transformer Blocks in its third stage to enhance deep feature interaction, introduces a CBAM module at the end of this stage to achieve collaborative calibration of channel attention and spatial attention, and adds a normalization layer before global average pooling to stabilize feature distribution. Through deep stacking and hybrid attention mechanisms, the defect recognition model can fully learn the subtle visual patterns and complex contextual relationships of screen defects, and adaptively enhance and filter key features, thereby improving the classification accuracy and robustness of various screen defects. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart of an image-based screen defect detection method according to this application;

[0070] Figure 2 This is a schematic diagram illustrating the principle of an image-based screen defect detection system according to this application. Detailed Implementation

[0071] The technical solutions of this application will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0072] Please see Figure 1 The first aspect of this application provides an image-based screen defect detection method, including:

[0073] Acquire the screen image to be detected;

[0074] The screen image is analyzed using an anomaly region identification model to obtain region results. The region results include region labels and region data. The region labels include labels indicating the presence or absence of anomaly labels. The region data includes images of anomaly regions and images of normal regions.

[0075] When the region label is "existing abnormal label", extract the abnormal region image corresponding to the "existing abnormal label";

[0076] The defect results are obtained by using an abnormal region image to identify defects through a defect identification model.

[0077] The training data used to train the abnormal region recognition model and the defect recognition model were constructed using the screen defect dataset; the screen defect dataset consists of several historical screen images and augmented screen images guided by process parameters.

[0078] The training data for the abnormal region identification model and the defect identification model in this embodiment include:

[0079] Several screen images and their corresponding region results are extracted from the screen defect dataset, and these screen images and their corresponding region results are used as training data for the abnormal region recognition model. The region results include region labels and region data. The region labels include labels indicating the presence of abnormalities and labels indicating the absence of abnormalities. The region data includes abnormal region images and normal region images. In this embodiment, the extracted screen images from the screen defect dataset include several historical screen images and several expanded screen images.

[0080] Several regions of data and their corresponding defect results are extracted, and these regions of data and their corresponding defect results are used as training data for the defect identification model. The defect results include defect labels and defect types. The defect labels include labels indicating the presence of defects and labels indicating the absence of defects. In this embodiment, the defect types include line defects, point defects, black screens, and abnormal displays.

[0081] The screen defect dataset in this embodiment consists of several historical screen images and augmented screen images enhanced by process parameters, including:

[0082] Obtain the manufacturing process parameter set corresponding to the historical screen image with defective labels; the manufacturing process parameter set refers to the process parameters when producing the corresponding historical screen image, including substrate material type, etching accuracy, thin film deposition thickness, packaging temperature, packaging pressure and driving voltage range;

[0083] A process condition vector is obtained by standardizing and vectorizing several process parameters in the manufacturing process parameter set.

[0084] Vectorize the defect results with defect labels to obtain text condition vectors;

[0085] Historical screen images, process condition vectors, and text condition vectors are input into the data augmentation model to obtain augmented screen images and their corresponding defect results. The data augmentation model is constructed using an improved Stable-Diffusion model to generate screen images with defect labels, thereby improving the accuracy of screen anomaly detection and classification.

[0086] Several historical screen images and extended screen images, as well as the corresponding defect results of several historical screen images and extended screen images, are integrated into a screen defect dataset.

[0087] The data augmentation model in this embodiment includes:

[0088] Extract the U-Net denoising network from the Stable-Diffusion model;

[0089] A process condition fusion module is embedded before several residual blocks in the U-Net denoising network; the process condition fusion module is used to introduce process condition vectors and, together with text condition vectors, guide the screen image to undergo denoising.

[0090] The process condition fusion module consists of an input layer, a condition fusion unit, a feature modulation unit, and an output layer;

[0091] The input layer is used to receive historical screen images, process condition vectors, and text condition vectors;

[0092] The conditional fusion unit is used to perform a fusion operation on the process condition vector and the text condition vector to obtain a fused condition vector. The fusion operation is implemented through a cross-attention mechanism, using the text condition vector as the query and the process condition vector as the key-value pair.

[0093] The feature modulation unit consists of a cross-attention layer, an adaptive feature modulation layer, and a conditional gating layer;

[0094] The cross-attention layer employs a cross-attention mechanism, using the image features corresponding to the historical screen images as queries and the fusion condition vector as key-value pairs to perform a fusion operation to obtain the fused feature vector;

[0095] An adaptive feature modulation layer is used to transform the fusion condition vector into scaling and shift parameters; and to modulate the image features using the scaling and shift parameters to obtain modulated image features. In this embodiment, the fusion condition vector is input into an MLP to generate scaling parameters (scale), shift parameters (shift), and image features are modulated using formulas. Modulation is performed to obtain modulated image features The formula satisfies:

[0096] In this embodiment, scale and shift need to be extended to the same spatial dimension as the image features.

[0097] The conditional gating layer is used to transform the process condition vector into gating weights, and to perform gating operations on the modulated image features using the gating weights to obtain gated image features. In this embodiment, the process condition vector is transformed into gating weights by inputting the process condition vector into a linear layer and a sigmoid layer.

[0098] The output layer is used to concatenate the gated image features with the image features corresponding to the historical screen images to obtain the image features after conditional modulation and gating. In this embodiment, the image features after conditional modulation and gating have the same size as the image features corresponding to the input historical screen images.

[0099] The abnormal region identification model in this embodiment includes an input layer, a backbone network, a dynamic attention network guided by screen geometry priors, a neck network, and a detection head; in this embodiment, the abnormal region identification model adopts an improved YOLOv11 model.

[0100] The input layer is used to receive the screen image to be detected;

[0101] The backbone network is used to extract multi-scale feature maps of the screen image; in this embodiment, the backbone network uses the CSPDarknet network, and the multi-scale feature map contains 3 scales.

[0102] A screen geometry prior-guided dynamic attention network is used to receive multi-scale feature maps output by the backbone network and introduce screen geometry prior knowledge to generate corresponding attention weight maps. The attention weight maps are then used to process the multi-scale feature maps to obtain several enhanced feature maps.

[0103] The neck network is used to fuse several enhanced feature maps to obtain a fused feature map; in this embodiment, the neck network uses the PANet network.

[0104] The detection head is used to analyze the fused feature map to output the region result;

[0105] The screen geometry prior-guided dynamic attention network is located between the backbone network and the neck network.

[0106] The screen geometry prior-guided dynamic attention network in this embodiment includes a screen prior encoding module, a geometric attention weight generation module, and a dynamic feature recalibration module.

[0107] The screen prior encoding module is used to learn the geometric prior knowledge of the screen, rather than using a fixed template; the input data of the screen prior encoding module is the multi-scale feature map output by the backbone network, and the output data is the screen prior parameter vector, which is used to guide the generation of the subsequent attention weight map; in this embodiment, the screen prior parameter vector includes screen center position preference, aspect ratio preference and edge importance weight.

[0108] The geometric attention weight generation module is used to generate an attention weight map based on the screen prior parameter vector, thereby enhancing the feature response of possible screen regions. The input data of the geometric attention weight generation module are multi-scale feature maps and screen prior parameter vectors; the output data is the attention weight map.

[0109] The dynamic feature recalibration module combines the attention weight map with the multi-scale feature map to achieve spatial and channel-level feature recalibration. The dynamic feature recalibration module is implemented through spatial attention modulation and channel attention modulation. The input data of the dynamic feature recalibration module is the attention weight map and the multi-scale feature map, and the output data is the enhanced feature map.

[0110] In this embodiment, an innovative dynamic attention network guided by screen geometry prior is embedded between the backbone and neck networks of the anomaly region identification model. This network consists of three parts: a screen prior encoding module, a geometric attention weight generation module, and a dynamic feature recalibration module. This dynamic attention network can autonomously learn the inherent geometric structure prior of the screen and generate an attention weight map accordingly, performing joint recalibration of the intermediate feature map in both spatial and channel dimensions. In this way, the model is effectively guided to focus on the real screen area, while significantly suppressing interference from the background, supports, or other irrelevant areas. This endows the anomaly region identification model with explicit screen structure perception capabilities, improving the model's localization accuracy and robustness in complex industrial scenarios. This lays a high-quality regional foundation for the subsequent defect classification stage and effectively reduces the risk of false positives and false negatives.

[0111] The defect identification model in this embodiment is constructed using an improved Swing Transformer model;

[0112] The Swin Transformer model consists of several stages, a normalization layer, a global average pooling layer, and a classification head. The stages include stage one, stage two, stage three, and stage four.

[0113] Each stage consists of a linear embedding layer, a downsampling layer, and a stack of several Swing Transformer Blocks. In this embodiment, stage one consists of one linear embedding layer and two Swing Transformer Blocks; stage two consists of one downsampling layer and two Swing Transformer Blocks; stage three consists of one downsampling layer and eighteen Swing Transformer Blocks; and stage four consists of one downsampling layer and two Swing Transformer Blocks.

[0114] Add a CBAM module after the last Swing Transformer Block in Phase 3; the CBAM module includes a channel attention module and a spatial attention module; the channel attention module is placed before the spatial attention module.

[0115] The number of Swing Transformer Blocks in Phase 3 is increased to N; where N is an integer, N>6; the specific value is set according to experience, and in this embodiment N is set to 18; N is set to be greater than 6 because the number of Swing Transformer Blocks in Phase 3 of the standard Swing Transformer model is 6;

[0116] The normalization layer is located before the global pooling layer and is used to normalize the features and stabilize the feature distribution. In this embodiment, the classification head uses a fully connected layer to output the defect results.

[0117] In constructing the defect recognition model, this embodiment makes several targeted improvements to the standard Swing Transformer to enhance the classification accuracy and robustness of screen defects: First, the number of Swing Transformer Blocks in the third stage is increased: By significantly increasing the number of Swing Transformer Blocks in the third stage of the Swing Transformer architecture, the model's ability to perform deep feature interactions is enhanced, helping the model to more fully capture and understand subtle visual patterns and complex contextual relationships in screen defects; Second, a convolutional block attention module is introduced at the end of the third stage to perform attention calibration in the channel dimension and spatial dimension respectively. By synergistically applying channel attention and spatial attention, key features can be adaptively enhanced and irrelevant information suppressed, thereby improving the model's performance; Finally, a normalization layer is added before global average pooling to stabilize the feature distribution, which helps ensure that data from different batches have similar scale and distribution, thereby accelerating the training process and improving the model's generalization ability; These improvements work together to enable the defect recognition model to learn and identify the features of various screen defects more accurately, while improving the model's robustness and stability.

[0118] In this embodiment, the defect identification process, which uses a defect identification model to identify defects in anomaly region images, includes:

[0119] S1: Extract the image of the abnormal region;

[0120] S2: High-resolution image features are obtained by performing high-resolution feature extraction on the abnormal region map through stage one;

[0121] S3: Extract medium-resolution image features from high-resolution image features through stage two;

[0122] S4: The medium-resolution image features are processed through a three-stage deep feature interaction to obtain the image depth features;

[0123] S5: The image depth features are extracted through advanced semantic feature extraction in stage four to obtain the image semantic features;

[0124] S6: The high-level semantic features are passed through a normalization layer, a global average pooling layer, and a classification head in sequence to perform feature normalization and defect type prediction operations to obtain the defect results.

[0125] Please see Figure 2 A second aspect of this application provides an image-based screen defect detection system, comprising: a data acquisition module and a data analysis module; the data acquisition module and the data analysis module are electrically and / or communicatively connected.

[0126] Data acquisition module: acquires the screen image to be detected through a data acquisition device; in this embodiment, the data acquisition device includes several sensors, etc.

[0127] The data analysis module includes a regional analysis unit and a defect analysis unit;

[0128] Region Analysis Unit: Performs region analysis on the screen image using an anomaly region identification model to obtain region results;

[0129] Defect Analysis Unit: This unit uses a defect recognition model to identify defects in images of abnormal regions and obtain the defect results.

[0130] Another embodiment of this application provides an image-based screen defect detection device, including: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the method described in the first aspect embodiment and any possible implementation of the first aspect embodiment; an image-based screen defect detection device may be an electronic device or a chip in an electronic device.

[0131] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

[0132] The working principle of this application is as follows: First, an image of the screen to be detected is acquired. Then, the screen image is analyzed using an anomaly region identification model to obtain the region results. When the region label indicates the presence of an anomaly label, the corresponding anomaly region image is extracted. This anomaly region image is then analyzed using a defect identification model to obtain the defect results. When constructing the training dataset for the anomaly region identification model and the defect identification model, key process parameters from the screen manufacturing process are used as generation conditions and input into the data generation model. This generates expanded defect samples that are highly consistent with real production failure modes in terms of physical mechanism and morphological distribution. This allows the anomaly region identification model and the defect identification model trained on this dataset to learn the inherent correlation between defects and process parameters, improving the model's accuracy and generalization performance in identifying various screen defects. This avoids the problem that existing technologies often use general image enhancement or text-based image models for sample expansion, resulting in expanded samples lacking physical rationality and failing to reflect the physical morphology and distribution patterns of defects under real production conditions, leading to low accuracy and generalization performance of the screen anomaly detection model.

[0133] The above embodiments are only used to illustrate the technical methods of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of this application without departing from the spirit and scope of the technical methods of this application.

Claims

1. An image-based screen defect detection method, characterized in that, include: Acquire the screen image to be detected; The screen image is analyzed using an abnormal region identification model to obtain region results; the region results include region labels and region data; the region labels include labels indicating the presence of abnormalities and labels indicating the absence of abnormalities; the region data includes images of abnormal regions and images of normal regions; When the region label is "existing abnormal label", extract the abnormal region image corresponding to the "existing abnormal label"; The defect results are obtained by using an abnormal region image to identify defects through a defect identification model. The training data used to train the abnormal region recognition model and the defect recognition model are both constructed from the screen defect dataset; the screen defect dataset consists of several historical screen images and augmented screen images guided by process parameters. The screen defect dataset consists of several historical screen images and augmented screen images enhanced by process parameters, including: Obtain the manufacturing process parameter set corresponding to the historical screen image with defective labels; the manufacturing process parameter set refers to the process parameters when producing the corresponding historical screen image, including substrate material type, etching accuracy, thin film deposition thickness, packaging temperature, packaging pressure and driving voltage range; A process condition vector is obtained by standardizing and vectorizing several process parameters in the manufacturing process parameter set. Vectorize the defect results with defect labels to obtain text condition vectors; Historical screen images, process condition vectors, and text condition vectors are input into a data augmentation model to obtain augmented screen images and their corresponding defect results. The data augmentation model is constructed using an improved Stable-Diffusion model. The U-Net denoising network is extracted from the Stable-Diffusion model. A process condition fusion module is embedded before several residual blocks in the U-Net denoising network. The process condition fusion module is used to introduce process condition vectors and, together with text condition vectors, guide the denoising process of the screen images. Several historical screen images and extended screen images, as well as the corresponding defect results of several historical screen images and extended screen images, are integrated into a screen defect dataset.

2. The image-based screen defect detection method according to claim 1, characterized in that, The data augmentation model includes: Extract the U-Net denoising network from the Stable-Diffusion model; A process condition fusion module is embedded before several residual blocks in the U-Net denoising network; the process condition fusion module is used to introduce process condition vectors and combine them with text condition vectors to guide the screen image in the denoising process. The process condition fusion module consists of an input layer, a condition fusion unit, a feature modulation unit, and an output layer. The input layer is used to receive historical screen images, process condition vectors, and text condition vectors. The condition fusion unit is used to perform a fusion operation on the process condition vector and the text condition vector to obtain a fused condition vector; the fusion operation is implemented through a cross-attention mechanism, with the text condition vector as the query and the process condition vector as the key-value pair. The feature modulation unit consists of a cross-attention layer, an adaptive feature modulation layer, and a conditional gating layer; The cross-attention layer adopts a cross-attention mechanism, using the image features corresponding to the historical screen images as queries and the fusion condition vector as key-value pairs to perform a fusion operation to obtain the fused feature vector; The adaptive feature modulation layer is used to transform the fusion condition vector into scaling and offset parameters; and to modulate the image features using the scaling and offset parameters to obtain modulated image features; The conditional gating layer is used to convert the process condition vector into gating weights, and to use the gating weights to perform gating operations on the modulated image features to obtain gating image features. The output layer is used to concatenate the gated image features with the image features corresponding to the historical screen images to obtain the image features after conditional modulation and gating.

3. The image-based screen defect detection method according to claim 1, characterized in that, The abnormal region identification model includes an input layer, a backbone network, a dynamic attention network guided by screen geometry priors, a neck network, and a detection head; The input layer is used to receive the screen image to be detected; The backbone network is used to extract multi-scale feature maps of screen images; The screen geometry prior-guided dynamic attention network is used to receive multi-scale feature maps output by the backbone network and introduce the screen geometry prior knowledge to generate corresponding attention weight maps. The attention weight maps are then used to process the multi-scale feature maps to obtain several enhanced feature maps. The neck network is used to fuse several enhanced feature maps to obtain a fused feature map; The detection head is used to analyze the fused feature map to output the region result; The screen geometry prior-guided dynamic attention network is located between the backbone network and the neck network.

4. The image-based screen defect detection method according to claim 3, characterized in that, The screen geometry prior-guided dynamic attention network includes a screen prior encoding module, a geometric attention weight generation module, and a dynamic feature recalibration module. The screen prior encoding module is used to learn the geometric prior knowledge of the screen; the input data of the screen prior encoding module is the multi-scale feature map output by the backbone network, and the output data is the screen prior parameter vector, which is used to guide the generation of the subsequent attention weight map. The geometric attention weight generation module is used to generate an attention weight map based on the screen prior parameter vector; the input data of the geometric attention weight generation module is a multi-scale feature map and a screen prior parameter vector; the output data is the attention weight map. The dynamic feature recalibration module is used to combine the attention weight map with the multi-scale feature map to achieve spatial and channel-level feature recalibration; the dynamic feature recalibration module is implemented through spatial attention modulation and channel attention modulation; the input data of the dynamic feature recalibration module is the attention weight map and the multi-scale feature map, and the output data is the enhanced feature map.

5. The image-based screen defect detection method according to claim 1, characterized in that, The defect identification model is constructed using an improved Swing Transformer model; The Swin Transformer model consists of several stages, a normalization layer, a global average pooling layer, and a classification head. The stages include stage one, stage two, stage three, and stage four. The stage consists of a linear embedding layer, a downsampling layer, and several stacked Swing Transformer Blocks. Add a CBAM module after the last Swin Transformer Block in Phase 3; the CBAM module includes a channel attention module and a spatial attention module; the channel attention module is located before the spatial attention module. Increase the number of Swing Transformer Blocks in Phase 3 to N; where N is an integer, N>6; The normalization layer is located before the global pooling layer and is used to normalize the features.

6. The image-based screen defect detection method according to claim 1, characterized in that, The step of obtaining defect results by identifying defects in abnormal region images using a defect recognition model includes: S1: Extract the image of the abnormal region; S2: The abnormal region map is subjected to high-resolution feature extraction in stage one to obtain high-resolution image features; S3: The high-resolution image features are extracted into medium-resolution image features through stage two; S4: The medium-resolution image features are processed through stage three to obtain image depth features; S5: The image depth features are extracted using advanced semantic feature extraction in stage four to obtain image semantic features; S6: The high-level semantic features are passed through a normalization layer, a global average pooling layer, and a classification head in sequence to perform feature normalization and defect type prediction operations to obtain the defect results.

7. The image-based screen defect detection method according to claim 1, characterized in that, The training data for the abnormal region identification model and the defect identification model include: Extract several screen images and their corresponding region results from the screen defect dataset, and use these screen images and their corresponding region results as training data for the abnormal region recognition model; the region results include region labels and region data; the region labels include labels indicating the presence of abnormalities and labels indicating the absence of abnormalities; the region data includes abnormal region images and normal region images; Extract data from several regions and their corresponding defect results, and use the data from several regions and their corresponding defect results as training data for the defect identification model; the defect results include defect labels and defect types; the defect labels include labels indicating the presence of defects and labels indicating the absence of defects.

8. An image-based screen defect detection system, applied to the image-based screen defect detection method according to any one of claims 1-7, characterized in that, include: A data acquisition module and a data analysis module; the data acquisition module and the data analysis module are connected to each other; The data acquisition module acquires the screen image to be detected through a data acquisition device; The data analysis module includes a region analysis unit and a defect analysis unit; The region analysis unit: performs region analysis on the screen image using an abnormal region identification model to obtain region results; The defect analysis unit: uses a defect recognition model to identify defects in abnormal region images to obtain defect results.

9. An image-based screen defect detection device, comprising: A processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement an image-based screen defect detection method as described in any one of claims 1-7.

Images