A panel defect self-adaptive detection method and system based on an abnormal synthesis strategy

Abstract

The application provides a panel defect adaptive detection method and system based on an abnormal synthesis strategy, and relates to the technical field of panel defect detection. The method flow is: performing primary detection on an original image based on a target detection model to obtain a preliminary detection result, wherein the preliminary detection result includes a defect category and a confidence; marking samples with a confidence lower than a confidence threshold in the preliminary detection result as normal samples to be reviewed; generating an adversarial abnormal sample using a gradient ascent abnormal synthesis strategy based on the normal samples to be reviewed; performing secondary detection on the normal samples to be reviewed and the adversarial abnormal sample based on a lightweight abnormality discrimination model to obtain a secondary detection result; and generating a final defect detection result based on the preliminary detection result and the secondary detection result. The application effectively solves the missed detection problem caused by confidence control in traditional methods by combining YOLOv8 primary detection and secondary detection based on abnormal synthesis.

Description

Technical Field

[0001] This invention relates to the field of panel defect detection technology, and more specifically, to an adaptive panel defect detection method and system based on an anomaly synthesis strategy. Background Technology

[0002] In the field of flat panel display manufacturing, with the rapid development of display technology and the continuous expansion of panel size, higher requirements are placed on the accuracy and efficiency of defect detection in the production process. Panel manufacturing typically includes three core process stages: array, cell, and module. The types, sizes, contrast ratios, and other characteristics of defects generated in each stage differ significantly, posing a huge challenge to automated defect detection.

[0003] Currently, the mainstream industrial practice uses deep learning-based automated optical inspection (AOI) systems for panel defect detection, but the following technical bottlenecks still exist: ① Poor adaptability to multi-process inspection: Defects in the array stage (such as TFT line breaks) are usually linear and have high contrast characteristics, with sizes on the order of 3-5μm; defects in the cell assembly stage (such as liquid crystal bubbles) are mostly circular and have low contrast characteristics, with sizes on the sub-pixel level; defects in the module stage (such as polarizer scratches) have edge-oriented characteristics. Traditional single inspection models are difficult to adapt to these significantly different defect characteristics simultaneously, resulting in poor inspection results in certain process stages.

[0004] ② High false negative rate for small-scale defects: Due to limitations in optical resolution, when the defect size is smaller than the system resolution, it occupies only a few pixels in the image. Traditional target detection models such as YOLOv8 easily lose feature information of these tiny defects during feature extraction. Actual tests show that for defects smaller than 5μm, the false negative rate of existing methods can reach over 20%.

[0005] ③ False negatives due to confidence level control: To ensure the reliability of detection results, industrial inspection systems typically set a threshold for the confidence level of the model output, only classifying detection results with a confidence level higher than the threshold as defects. However, for defects with low contrast and indistinct features, the confidence level of the model output is often too low, making them easily filtered out by the threshold and resulting in false negatives. This problem is particularly prominent in the detection of low-contrast defects in the cell assembly process.

[0006] ④ Scarcity of abnormal sample data: Deep learning model training relies on a large amount of labeled data, but defect samples in industrial production are relatively scarce, and samples of some rare defect types are even more difficult to obtain. The data imbalance problem seriously affects the model's generalization ability and the detection effect on rare defects.

[0007] ⑤ Conflicts and redundancy in multi-model architecture: To solve the problem of multi-process inspection, existing technologies often use multiple dedicated models to handle different process stages or different types of defects. This architecture not only increases system complexity and computational overhead, but may also lead to different models outputting contradictory results for the same inspection sample, requiring additional manual arbitration mechanisms and reducing inspection efficiency. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides an adaptive detection method and system for panel defects based on an anomaly synthesis strategy.

[0009] In a first aspect, embodiments of the present invention provide an adaptive panel defect detection method based on an anomaly synthesis strategy, comprising the following steps: Obtain the original image of the panel to be inspected; The original image is input into a pre-trained target detection model for primary detection to obtain preliminary detection results, which include defect category, confidence level, and bounding box location. Samples with a confidence level below the confidence threshold in the preliminary test results are marked as normal samples to be retested; Based on the normal samples to be reviewed, an adversarial anomaly samples are generated using a gradient ascent anomaly synthesis strategy. The normal samples to be re-examined and the corresponding adversarial abnormal samples are input into a pre-trained lightweight anomaly discrimination model for secondary detection to obtain secondary detection results; Based on the preliminary and secondary detection results, the final defect detection results are generated.

[0010] In the above embodiments, the present invention effectively reduces the missed detections caused by confidence threshold filtering through a two-stage detection mechanism, especially for secondary detection of low-confidence samples. Furthermore, during the two-stage detection process, most samples can be quickly determined through the YOLOv8 primary detection, and only a small number of low-confidence samples need to enter the computationally intensive secondary detection. This design optimizes the overall computational efficiency while ensuring detection accuracy.

[0011] As some optional embodiments of this application, the training process of the pre-trained target detection model is as follows: Historical images of each process stage in panel manufacturing are acquired, including the array stage, cell assembly stage, and module stage. The historical images are labeled with features to construct a training dataset, wherein the information of the feature labels includes defect categories and bounding box locations; Based on the training dataset, a multi-scale training strategy was used to train the YOLOv8 model, and the trained model parameters were saved to obtain a pre-trained object detection model.

[0012] As one of the optional implementations of this application, the process of generating adversarial anomaly samples using a gradient ascent anomaly synthesis strategy is as follows: Obtain the original image of a normal sample to be re-examined, and generate noise perturbation at random positions in the original image; The noise perturbation is iteratively optimized using the gradient ascent method to maximize the anomaly detection score of the lightweight anomaly detection model for the anomaly synthesized image. During the gradient ascent method iterative optimization process, the morphological characteristics of noise disturbances are constrained according to the current process stage of the detection panel. When the anomaly discrimination score reaches a preset threshold or the number of iterations reaches an upper limit, the optimization stops, and adversarial anomaly samples are obtained.

[0013] In the above embodiments, the invention employs an anomaly synthesis strategy based on gradient ascent, which can generate adversarial anomaly samples similar to the features of real minor defects. By allowing a lightweight anomaly discrimination model to learn the features of these synthetic anomalies, the sensitivity to actual minor defects is improved.

[0014] As some optional embodiments of this application, the morphological characteristics of the constrained noise disturbance specifically include: When the detection panel is in the array process stage, the noise disturbance is constrained to have a linear or quasi-linear morphological characteristic; When the detection panel is in the cell assembly process stage, the noise disturbance is constrained to have a circular or irregular low-contrast shape. When the detection panel is in the module process stage, the noise disturbance is constrained to have an edge-oriented morphological characteristic; The morphological constraint matrix is ​​associated with the process stage, and the morphological features are constrained through the morphological constraint matrix.

[0015] In the above embodiments, during the gradient ascent optimization process, the present invention imposes constraints on the morphological characteristics of noise disturbances according to the current process stage of the detection panel, so that the synthesized anomaly is more in line with the physical characteristics of the actual defect.

[0016] As some optional implementations of this application, the training process of the pre-trained lightweight anomaly detection model is as follows: Obtain defect-free images of each process stage in the panel manufacturing process; Based on the defect-free image, an anomaly synthesis strategy is used to generate diverse anomaly synthesized images; A training dataset is constructed based on defect-free images and anomalous synthetic images, wherein positive samples are the anomalous synthetic images and negative samples are the defect-free images; A binary classification model is trained using a convolutional neural network architecture. The binary classification model is used to determine whether an input image is normal or abnormal. The binary classification model is trained based on the training dataset. During the model training process, a hard sample mining strategy is adopted to optimize the model's anomaly discrimination ability for anomalous synthetic images.

[0017] In the above embodiments, the training of the lightweight anomaly discrimination model of the present invention only requires normal samples and synthetic anomaly samples, without the need for a large number of real defect samples, which solves the problem of scarce defect samples in industrial production and is particularly beneficial for the detection of rare defect types.

[0018] As one of the optional implementations of this application, the process of inputting the normal samples to be re-examined and the corresponding adversarial anomaly samples into a pre-trained lightweight anomaly discrimination model for secondary detection is as follows: The normal sample to be reviewed and the corresponding adversarial abnormal sample are input into the lightweight anomaly detection model to obtain two anomaly detection scores. Comparing the two anomaly discrimination scores, if the discrimination score of the adversarial anomaly sample is significantly higher than the discrimination score of the normal sample to be reviewed, then it is determined that the normal sample to be reviewed contains potential defective features similar to the synthetic anomaly. Based on the comparison results and the preset anomaly detection threshold, it is determined whether the normal sample to be re-examined has any missed detection defects.

[0019] In the above embodiments, the secondary detection stage of the present invention determines defects by comparing the difference in abnormal scores between the original sample and the adversarial abnormal sample. This relative comparison strategy is more robust than absolute threshold determination and helps to control the false alarm rate.

[0020] As some optional implementations of this application, the process for generating the final defect detection result based on the preliminary detection result and the secondary detection result is as follows: Samples with a confidence level higher than the confidence threshold in the preliminary detection results are directly marked as defective samples; Integrate the normal samples awaiting re-inspection that were identified as having missed defects from the secondary inspection results, and supplement the defect category and location information; Samples that are determined to be normal in both the initial and secondary tests are marked as normal samples. Generate a structured report containing defect statistics, location distribution, and process stage related information.

[0021] In the above embodiments, the present invention can generate a clearer and more comprehensive structured report by fusing preliminary detection results and secondary detection results.

[0022] As some optional implementations of this application, a model adaptive update step is also included: Collect the misjudged samples in the final defect detection report, including the falsely detected normal samples and the falsely detected defect samples; The misclassified samples are added to the training dataset to perform incremental training on the target detection model and / or the lightweight anomaly detection model, thereby achieving continuous optimization of the model.

[0023] In the above embodiments, the present invention achieves incremental learning and continuous optimization of the model by feeding back the collected misjudged samples to the training process.

[0024] As some optional implementations of this application, a dynamic adjustment step for the detection process is also included: Obtain the process stage identification information of the current inspection panel; Based on the process stage identification information, the confidence threshold, the morphological constraint parameters in the gradient ascent anomaly synthesis strategy, and the anomaly determination threshold of the lightweight anomaly discrimination model are dynamically adjusted to achieve adaptive detection of defect characteristics for different process stages.

[0025] In the above embodiments, the present invention adjusts the anomaly synthesis strategy and detection parameters according to the process stage, enabling a single model to adapt to different defect characteristics at each process stage of panel manufacturing. This eliminates the need to train and deploy dedicated models for each process stage, simplifying the model architecture and reducing maintenance costs.

[0026] In a second aspect, the present invention provides an adaptive panel defect detection system based on an anomaly synthesis strategy, the system comprising: An image acquisition unit is used to acquire the original image of the panel to be inspected; A primary detection unit is used to input the original image into a pre-trained target detection model for primary detection to obtain preliminary detection results, wherein the preliminary detection results include defect category, confidence level, and bounding box position. A confidence verification unit is used to mark samples with a confidence level lower than a confidence threshold in the preliminary detection results as normal samples to be re-examined. An anomaly generation unit generates adversarial anomaly samples based on the normal samples to be reviewed, using a gradient-ascending anomaly synthesis strategy. A secondary detection unit is used to input the normal sample to be re-examined and the corresponding adversarial abnormal sample into a pre-trained lightweight anomaly discrimination model for secondary detection to obtain secondary detection results. The result generation unit generates the final defect detection result based on the preliminary detection result and the secondary detection result.

[0027] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned adaptive panel defect detection method based on an anomaly synthesis strategy.

[0028] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned adaptive panel defect detection method based on an anomaly synthesis strategy.

[0029] The beneficial effects of this invention are as follows: 1. This invention effectively reduces missed detections caused by confidence threshold filtering by employing a two-stage detection mechanism, particularly for secondary detection of low-confidence samples.

[0030] 2. The invention employs a gradient-ascending-based anomaly synthesis strategy that can generate adversarial anomaly samples that are similar to the features of real minor defects. By allowing a lightweight anomaly discrimination model to learn the features of these synthetic anomalies, the sensitivity to actual minor defects is improved.

[0031] 3. The lightweight anomaly discrimination model of this invention only requires normal samples and synthetic anomaly samples for training, without the need for a large number of real defect samples. This solves the problem of scarce defect samples in industrial production and is particularly beneficial for the detection of rare defect types.

[0032] 4. This invention adjusts the anomaly synthesis strategy and detection parameters according to the process stage, enabling a single model to adapt to different defect characteristics at each process stage of panel manufacturing. It eliminates the need for separate training and deployment of dedicated models for each process stage, simplifying the model architecture and reducing maintenance costs. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments are briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1This is a flowchart of the adaptive panel defect detection method described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the abnormal synthesis and abnormal determination described in the embodiments of the present invention; Figure 3 This is a schematic diagram illustrating the generation of adversarial anomalous samples according to an embodiment of the present invention. Detailed Implementation

[0035] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0036] To address the shortcomings of existing detection technologies, such as poor adaptability to multi-process detection, high defect false negative rates, and scarcity of abnormal sample data, this invention provides an adaptive panel defect detection method and system based on an anomaly synthesis strategy. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 The flowchart of the adaptive detection method for panel defects is as follows: (1) Image acquisition and preprocessing.

[0037] In this embodiment of the invention, the first step is to obtain the original image of the panel to be detected.

[0038] Specifically, depending on the configuration of the detection equipment, the original image can be a bright-field image, a dark-field image, or both simultaneously. The bright-field image mainly reflects the reflectivity of the panel surface and is suitable for detecting defects such as metal lines and scratches; the dark-field image mainly reflects the scattering and fluorescence characteristics of the material and is suitable for detecting abnormalities and contamination in organic materials.

[0039] Furthermore, the image acquisition parameters are set according to the panel's process stage and inspection requirements. For the array process stage, high-resolution bright-field imaging is typically used, with a resolution of up to 0.5 μm / pixel; for the cell assembly process stage, it is usually necessary to combine bright-field and dark-field imaging to capture specific defects in the liquid crystal material.

[0040] Furthermore, after acquiring the image, necessary preprocessing operations are required, including but not limited to: illumination non-uniformity correction, image denoising, contrast enhancement, etc.; for cases where bright field images and dark field images are acquired simultaneously, image registration is also required to ensure that the two images are spatially aligned.

[0041] (2) Primary testing.

[0042] In this embodiment of the invention, the preprocessed original image needs to be input into a pre-trained target detection model for primary defect detection to obtain preliminary detection results; wherein, the preliminary detection results include defect category, confidence level and bounding box position, the confidence level reflects the model's confidence in the detection results, and the value range is [0,1].

[0043] Preferably, since the YOLOv8 model exhibits an excellent balance between accuracy and speed in object detection tasks, this embodiment of the invention uses the YOLOv8 model as the object detection model for primary defect detection. However, those skilled in the art should understand that other advanced object detection models, such as YOLOv5, YOLOv7, Faster R-CNN, RetinaNet, etc., can also be used as alternatives.

[0044] Furthermore, the training process of the pre-trained target detection model is as follows: (2.1) Obtain historical images of each process stage in panel manufacturing, wherein the process stages include the array stage, the cell assembly stage, and the module stage. Specifically, the historical images include normal samples and defective samples.

[0045] (2.2) Feature annotation is performed on the historical images to construct a training dataset; wherein, the information of the feature annotation includes defect category and bounding box location. Specifically, the defect category includes, but is not limited to, TFT broken line, TFT short circuit, metal residue, particle contamination, liquid crystal bubble, Mura unevenness, polarizer scratch, backlight foreign object, etc.

[0046] (2.3) Based on the training dataset, a multi-scale training strategy is adopted to train the YOLOv8 model, that is, the input image size is randomly selected within the pixel range, so that the model can adapt to defects of different scales.

[0047] Specifically, during training, higher loss weights are applied to defect categories with sizes smaller than a preset pixel threshold; and during training, an attention mechanism can be introduced to enable the model to focus on suspicious areas in the image, enhancing its sensitivity to low-contrast defects.

[0048] (2.4) Save the trained model parameters to obtain the pre-trained target detection model.

[0049] (3) Confidence threshold filtering.

[0050] In this embodiment of the invention, samples with a confidence level lower than the confidence threshold T1 in the preliminary detection results need to be marked as normal samples to be re-examined.

[0051] Specifically, the selection of the confidence threshold T1 needs to strike a balance between recall and precision. A higher threshold can reduce the false alarm rate but increase the risk of missed detections; a lower threshold can improve recall but increase false alarms. In practical applications, the confidence threshold T1 can be dynamically adjusted according to the importance of different defect categories. For critical defects (such as defects that cause panel malfunction), a lower threshold can be set to improve the detection rate; for non-critical defects, a higher threshold can be set to control false alarms.

[0052] (4) Abnormal synthesis strategy.

[0053] In this embodiment of the invention, based on the normal sample to be reviewed (OK sample) and its original image, adversarial anomaly samples are obtained using either global anomaly synthesis or local anomaly synthesis. Specifically, global anomaly synthesis employs a gradient-ascending anomaly synthesis strategy to generate corresponding adversarial anomaly samples. The aim is to generate anomaly samples that are similar to but not identical to the actual defect features, thereby enhancing the detection capability of the anomaly discrimination model. Furthermore, this invention can also use local anomaly synthesis to obtain adversarial anomaly samples; please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of anomaly synthesis and anomaly determination.

[0054] Further, please refer to Figure 3 , Figure 3 The diagram illustrates the generation of adversarial anomaly samples. The process for generating corresponding adversarial anomaly samples based on the gradient ascent anomaly synthesis strategy is as follows: (4.1) Obtain the original image of the normal sample to be reviewed, and generate noise perturbation at random positions in the original image.

[0055] Specifically, for the original image I of a normal sample to be reviewed, noise perturbation δ is generated at random locations in the image. t The noise can be random Gaussian noise, uniform noise, or other forms of noise. The magnitude and range of the noise can be set according to the expected size of the defect; for example, for small defects in the cassette assembly process, the noise range can be limited to a 10×10 pixel area.

[0056] (4.2) The noise perturbation is iteratively optimized by gradient ascent method to maximize the anomaly discrimination score of the lightweight anomaly discrimination model for the anomaly synthesis image.

[0057] Specifically, the noise perturbation δ is iteratively optimized using the gradient ascent method. The optimization objective is to maximize the impact of the lightweight anomaly detection model on the synthesized image I+δ. t The anomaly detection score.

[0058] The optimization process can be represented as: δt+1 =δ t +η·∇δ(fθ(I+δ t )); Where fθ is a lightweight anomaly detection model, fθ(I+δ) t ) represents the anomaly detection score of the model on the input image I+δ, η is the learning rate, and ∇δ represents the resolution of noise δ. t The gradient, δ t+1 This represents noise in gradient optimization.

[0059] (4.3) During the gradient ascent method iterative optimization process, the morphological characteristics of noise disturbance are constrained according to the current process stage of the detection panel.

[0060] Specifically, during the gradient ascent optimization process, constraints are imposed on the morphological characteristics of noise disturbances based on the current process stage of the detection panel. This is a significant innovation of the invention, enabling the synthesized anomalies to better reflect the physical characteristics of actual defects.

[0061] Furthermore, the constraints are implemented through a morphological constraint matrix M: δ constrained =M⊙δ t ; Where ⊙ denotes element-wise multiplication, δ constrained Indicates the noise δ t The shape constraint results. The shape constraint matrix M is designed according to the process stage: ① For the array process stage: M is designed as a filter to highlight linear characteristics, so that the synthesized anomalies appear as linear or near-linear shapes, simulating defects such as TFT broken lines and metal line short circuits.

[0062] ② For the cell assembly process: M is designed as a filter that highlights circular or elliptical features, so that the synthesis anomalies appear as dots or near-circular shapes, simulating defects such as liquid crystal bubbles and particulate contamination.

[0063] ③ For the module process stage: M is designed as a filter that highlights the edge orientation features, so that the synthesized anomalies are distributed along the edge of the image, simulating defects such as polarizer scratches and edge cracks.

[0064] (4.4) When the anomaly discrimination score reaches the preset threshold or the number of iterations reaches the upper limit, the optimization stops and an adversarial anomaly sample is obtained.

[0065] Specifically, optimization stops when any of the following conditions are met: (1) the anomaly discrimination score reaches the preset threshold S_thresh; (2) the number of iterations reaches the maximum number of iterations N_max; (3) the change in noise disturbance is less than the tolerance ε. After optimization, the adversarial anomaly sample I' = I + δ* is obtained, where δ* represents the noise from multiple iterations of optimization and morphological constraints.

[0066] The advantage of the gradient ascent method iterative optimization in this invention lies in its ability to generate adversarial examples targeting the "weaknesses" of specific anomaly detection models. These examples are often located near the decision boundary between normal and anomaly samples, and are the most difficult samples for the model to detect. By allowing the model to learn to detect these adversarial examples, the model's ability to detect actual defects can be effectively improved.

[0067] (5) Secondary testing.

[0068] In this embodiment of the invention, the normal sample to be reviewed and the corresponding adversarial anomaly sample are input into a pre-trained lightweight anomaly discrimination model for secondary detection to obtain secondary detection results. The lightweight anomaly discrimination model differs from the target detection model; it is a binary classification model that only needs to determine whether the input image is normal or abnormal, without needing to locate the defect location or classify the defect type. This simplification allows the model to be designed to be more lightweight and has a faster inference speed.

[0069] Specifically, the training process of the pre-trained lightweight anomaly detection model is as follows: (5.1) Obtain defect-free images of each process stage in the panel manufacturing process. The defect-free images are relatively easy to obtain and can be collected in large quantities, thus solving the problem of scarce defect samples.

[0070] (5.2 Based on the defect-free image, anomaly synthesis strategy is adopted to generate diverse anomalous synthetic images; specifically, in order to enhance the diversity of synthetic samples, the following strategies can be adopted: ① Initialize noise at different locations; ② Use different initial noise types; ③ Apply different morphological constraint matrices; ④ Control parameters such as the size and contrast of the synthesized anomaly.

[0071] (5.3) Construct a training dataset based on defect-free images and anomalous synthetic images; wherein, positive samples are the anomalous synthetic images and negative samples are the defect-free images.

[0072] (5.4) A binary classification model is trained using a convolutional neural network architecture. The binary classification model is used to determine whether the input image is normal or abnormal. The binary classification model can be a lightweight MobileNetV2, EfficientNet-B0, etc.

[0073] (5.5) The binary classification model is trained based on the training dataset; during the model training process, a hard sample mining strategy is adopted to optimize the model's anomaly discrimination ability for anomalous synthetic images. The training objective is to minimize the binary classification cross-entropy loss. During the training process, a hard sample mining strategy is adopted, focusing on samples that the model has difficulty in judging. These samples are often the most critical to improving the model performance.

[0074] Specifically, for each normal sample I to be reviewed, the lightweight anomaly detection model calculates its anomaly score s = f_θ(I); for the corresponding adversarial anomaly sample I', it calculates its anomaly score s' = f_θ(I'). By comparing s and s', it is determined whether the original sample has a missed defect. ① If s' is significantly higher than s (e.g., s' - s > Δ, where Δ is the difference threshold), it indicates that the abnormal features of the adversarial abnormal sample are significantly stronger than those of the original sample, and the original sample is likely to be a normal sample.

[0075] ② If s and s' are not significantly different, it means that the normal sample I to be re-examined already contains abnormal features similar to the antagonistic abnormal sample I', which is likely a missed detection defect; ③ If s > T2 (T2 is the anomaly detection threshold), it is directly determined to be an anomaly.

[0076] Based on the above comparisons, and combined with the anomaly detection threshold T2, it is finally determined whether a normal sample awaiting re-examination is a missed defect. This relative comparison strategy is more robust than absolute threshold determination because it considers the relative position of the sample in the feature space.

[0077] (6) The final defect detection results are generated.

[0078] Specifically, the results of the initial inspection and the secondary inspection are combined to generate the final defect inspection report: (6.1) Integrate the samples with confidence levels higher than the confidence threshold T1 in the preliminary detection results and directly mark them as defective samples.

[0079] (6.2) Integrate the normal samples awaiting re-inspection that were identified as missed defects in the secondary inspection results, and supplement the defect category and location information; that is, the normal samples awaiting re-inspection that were identified as missed defects in the secondary inspection are treated as new defect samples. For these samples, the defect category can refer to the category of similar samples in the primary inspection, or be marked as requiring manual review.

[0080] (6.3) Samples that are determined to be normal in both the preliminary test and the second test are marked as normal samples.

[0081] (6.4) Generate a structured report containing defect statistics, location distribution, and process stage correlation information. Specifically, the final structured report may include: defect category, location coordinates, confidence / anomaly score, image evidence, process stage information, inspection timestamp, etc.

[0082] In addition, to achieve continuous optimization of defect detection, an adaptive model update mechanism is established: ① Collection of misjudged samples: In the final test results, misjudged samples are collected, including two categories: false alarm samples (normal samples are misjudged as defects) and missed detection samples (defective samples are missed). These samples can be identified through manual sampling, customer feedback, and subsequent process testing.

[0083] ② Incremental learning: Regularly (e.g., weekly or monthly) incrementally train the model using collected misclassified samples. For the YOLOv8 model, a fine-tuning strategy can be adopted, using new samples for a small number of training rounds on the basis of the original model. To prevent new knowledge from overwriting old knowledge (catastrophic forgetting), continuous learning methods such as Elastic Weight Consolidation (EWC) can be used.

[0084] ③ Synthesis Strategy Optimization: Based on the characteristics of misclassified samples, optimize the gradient ascent anomaly synthesis strategy. For example, if a certain type of defect is found to have a high false negative rate, the anomaly synthesis strategy can be adjusted to generate more anomaly samples with similar characteristics, thereby enhancing the model's sensitivity to this type of defect.

[0085] ④ Dynamic Threshold Adjustment: Based on recent detection performance, the confidence threshold T1 and the anomaly detection threshold T2 are dynamically adjusted. If the false negative rate of a certain type of defect increases, the corresponding threshold can be appropriately lowered; if the false positive rate increases, the threshold can be appropriately raised.

[0086] In summary, this invention effectively solves the problem of missed detections caused by confidence level constraints in traditional methods by combining primary detection using the YOLOv8 model with secondary detection based on anomaly synthesis. Furthermore, the gradient-ascending anomaly synthesis technique can generate targeted adversarial examples, enhancing the detection sensitivity for minute and low-contrast defects. In addition, the process adaptation mechanism allows a single model to adapt to the detection needs of multiple stages in panel manufacturing. Therefore, this invention significantly outperforms existing technologies in terms of detection accuracy, missed detection rate, efficiency, and adaptability, possessing significant industrial application value and broad market prospects.

[0087] Furthermore, in one embodiment, based on the same inventive concept as the foregoing embodiments, this embodiment of the invention provides a panel defect adaptive detection system based on an anomaly synthesis strategy. The system corresponds one-to-one with the method described above, and the system includes: An image acquisition unit is used to acquire the original image of the panel to be inspected; A primary detection unit is used to input the original image into a pre-trained target detection model for primary detection to obtain preliminary detection results, wherein the preliminary detection results include defect category, confidence level, and bounding box position. A confidence verification unit is used to mark samples with a confidence level lower than a confidence threshold in the preliminary detection results as normal samples to be re-examined. An anomaly generation unit generates adversarial anomaly samples based on the normal samples to be reviewed, using a gradient-ascending anomaly synthesis strategy. A secondary detection unit is used to input the normal sample to be re-examined and the corresponding adversarial abnormal sample into a pre-trained lightweight anomaly discrimination model for secondary detection to obtain secondary detection results. The result generation unit generates the final defect detection result based on the preliminary detection result and the secondary detection result.

[0088] It should be noted that each unit in the adaptive panel defect detection system in this embodiment corresponds one-to-one with each step in the adaptive panel defect detection method in the aforementioned embodiment. Therefore, the specific implementation method and the technical effects achieved in this embodiment can be referred to the implementation method of the aforementioned adaptive panel defect detection method, and will not be repeated here.

[0089] Furthermore, in one embodiment, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory, the computer program being executed by the processor to implement the methods in the foregoing embodiments.

[0090] In addition, in one embodiment, this application also provides a computer storage medium storing a computer program that is executed by a processor to implement the methods described in the foregoing embodiments.

[0091] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a device including one or any combination of the above-mentioned memories. The computer may be a variety of computing devices, including smart terminals and servers.

[0092] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0093] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborative files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0094] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0095] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0096] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a multimedia terminal device (which may be a mobile phone, computer, television receiver, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0098] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An adaptive panel defect detection method based on an anomaly synthesis strategy, characterized in that, Includes the following steps: Obtain the original image of the panel to be inspected; The original image is input into a pre-trained target detection model for primary detection to obtain preliminary detection results, which include defect category, confidence level, and bounding box location. Samples with a confidence level below the confidence threshold in the preliminary test results are marked as normal samples to be retested; Based on the normal samples to be reviewed, an adversarial anomaly samples are generated using a gradient ascent anomaly synthesis strategy. The normal samples to be re-examined and the corresponding adversarial abnormal samples are input into a pre-trained lightweight anomaly discrimination model for secondary detection to obtain secondary detection results; Based on the preliminary and secondary detection results, the final defect detection results are generated.

2. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 1, characterized in that, The training process for the pre-trained target detection model is as follows: Historical images of each process stage in panel manufacturing are acquired, including the array stage, cell assembly stage, and module stage. The historical images are labeled with features to construct a training dataset, wherein the information of the feature labels includes defect categories and bounding box locations; Based on the training dataset, a multi-scale training strategy was used to train the YOLOv8 model, and the trained model parameters were saved to obtain a pre-trained object detection model.

3. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 1, characterized in that, The process of generating adversarial anomaly samples using a gradient ascent anomaly synthesis strategy is as follows: Obtain the original image of a normal sample to be re-examined, and generate noise perturbation at random positions in the original image; The noise perturbation is iteratively optimized using the gradient ascent method to maximize the anomaly detection score of the lightweight anomaly detection model for the anomaly synthesized image. During the gradient ascent method iterative optimization process, the morphological characteristics of noise disturbances are constrained according to the current process stage of the detection panel. When the anomaly discrimination score reaches a preset threshold or the number of iterations reaches an upper limit, the optimization stops, and adversarial anomaly samples are obtained.

4. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 3, characterized in that, The morphological characteristics of the constrained noise disturbance specifically include: When the detection panel is in the array process stage, the noise disturbance is constrained to have a linear or quasi-linear morphological characteristic; When the detection panel is in the cell assembly process stage, the noise disturbance is constrained to have a circular or irregular low-contrast shape. When the detection panel is in the module process stage, the noise disturbance is constrained to have an edge-oriented morphological characteristic; The morphological constraint matrix is ​​associated with the process stage, and the morphological features are constrained through the morphological constraint matrix.

5. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 1, characterized in that, The training process for the pre-trained lightweight anomaly detection model is as follows: Obtain defect-free images of each process stage in the panel manufacturing process; Based on the defect-free image, an anomaly synthesis strategy is used to generate diverse anomaly synthesized images; A training dataset is constructed based on defect-free images and anomalous synthetic images, wherein positive samples are the anomalous synthetic images and negative samples are the defect-free images; A binary classification model is trained using a convolutional neural network architecture. The binary classification model is used to determine whether an input image is normal or abnormal. The binary classification model is trained based on the training dataset. During the model training process, a hard sample mining strategy is adopted to optimize the model's anomaly discrimination ability for anomalous synthetic images.

6. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 1, characterized in that, The process of inputting the normal samples to be re-examined and the corresponding adversarial anomaly samples into a pre-trained lightweight anomaly detection model for secondary detection is as follows: The normal sample to be reviewed and the corresponding adversarial abnormal sample are input into the lightweight anomaly detection model to obtain two anomaly detection scores. Comparing the two anomaly discrimination scores, if the discrimination score of the adversarial anomaly sample is significantly higher than the discrimination score of the normal sample to be reviewed, then it is determined that the normal sample to be reviewed contains potential defective features similar to the synthetic anomaly. Based on the comparison results and the preset anomaly detection threshold, it is determined whether the normal sample to be re-examined has any missed detection defects.

7. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 1, characterized in that, Based on the preliminary and secondary inspection results, the process for generating the final defect inspection results is as follows: Samples with a confidence level higher than the confidence threshold in the preliminary detection results are directly marked as defective samples; Integrate the normal samples awaiting re-inspection that were identified as having missed defects from the secondary inspection results, and supplement the defect category and location information; Samples that are determined to be normal in both the initial and secondary tests are marked as normal samples. Generate a structured report containing defect statistics, location distribution, and process stage related information.

8. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 7, characterized in that, It also includes the model adaptive update step: Collect the misjudged samples in the final defect detection report, including the falsely detected normal samples and the falsely detected defect samples; The misclassified samples are added to the training dataset to perform incremental training on the target detection model and / or the lightweight anomaly detection model, thereby achieving continuous optimization of the model.

9. The adaptive panel defect detection method based on anomaly synthesis strategy according to claim 1, characterized in that, It also includes steps for dynamically adjusting the testing process: Obtain the process stage identification information of the current inspection panel; Based on the process stage identification information, dynamically adjust the confidence threshold, the morphological constraint parameters in the gradient ascent anomaly synthesis strategy, and the anomaly determination threshold of the lightweight anomaly discrimination model. To achieve adaptive detection of defect characteristics at different process stages.

10. A panel defect adaptive detection system based on anomaly synthesis strategy for implementing the method of claim 1, characterized in that, The system includes: An image acquisition unit is used to acquire the original image of the panel to be inspected; A primary detection unit is used to input the original image into a pre-trained target detection model for primary detection to obtain preliminary detection results, wherein the preliminary detection results include defect category, confidence level, and bounding box position. A confidence verification unit is used to mark samples with a confidence level lower than a confidence threshold in the preliminary detection results as normal samples to be re-examined. An anomaly generation unit generates adversarial anomaly samples based on the normal samples to be reviewed, using a gradient-ascending anomaly synthesis strategy. A secondary detection unit is used to input the normal sample to be re-examined and the corresponding adversarial abnormal sample into a pre-trained lightweight anomaly discrimination model for secondary detection to obtain secondary detection results. The result generation unit generates the final defect detection result based on the preliminary detection result and the secondary detection result.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes a computer program, it implements the panel defect adaptive detection method based on the anomaly synthesis strategy as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the panel defect adaptive detection method based on an anomaly synthesis strategy as described in any one of claims 1-9.

Images