Display panel defect detection model optimization method
By constructing a standard evaluation sample pool and training a benchmark teacher model, and by using knowledge distillation to optimize the parameters of the defect detection model, the problem of inconsistent optimization benchmarks among different algorithms was solved, and high-precision and highly adaptable defect detection was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHONGJIA MICROVISION (SHENZHEN) SEMICONDUCTOR TECHNOLOGY CO LTD
- Filing Date
- 2025-12-12
- Publication Date
- 2026-05-01
AI Technical Summary
The inconsistent optimization benchmarks of different defect detection algorithms lead to inconsistencies in indicators such as detection rate, false detection rate, and repeatability accuracy. They also lack robust evaluation benchmarks, closed-loop processes, and engineering improvements.
A standard evaluation sample pool is constructed to train a benchmark teacher model. The multimodal detection data output by the model is used as an optimization benchmark through knowledge distillation to optimize the parameters of the defect detection model, so that it has a consistent algorithm optimization benchmark under actual modalities.
The algorithm optimization benchmarks of different defect detection algorithms have been consistent, which has improved the adaptability and detection accuracy of the defect detection model and ensured a high detection rate and a low false detection rate under different working conditions.
Smart Images
Figure CN121962841A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of display technology, and specifically to an optimization method for a display panel defect detection model. Background Technology
[0002] With the mass production of new display technologies such as high-resolution OLED, Mini-LED, and Micro-LED, defect detection on the surface of display panels (including patterned circuit layers, thin film layers, and encapsulation layers) has been pushed to the micrometer or even sub-micrometer level of precision. Defect detection equipment already deployed in the industry (AOI / AVI type equipment) typically incorporates multiple defect detection algorithms, including traditional threshold segmentation algorithms, edge detection algorithms, and deep learning algorithms such as convolutional neural networks. However, the optimization benchmarks of different defect detection algorithms are inconsistent. Summary of the Invention
[0003] The main objective of this disclosure is to provide an optimization method for a display panel defect detection model, so as to improve the problem of inconsistent algorithm optimization benchmarks among different defect detection algorithms in related technologies.
[0004] To achieve the above objectives, a first aspect of this disclosure provides an optimization method for a display panel defect detection model, the method comprising: A standard evaluation sample pool is constructed, which includes multiple standard sample data. Each standard sample data includes: multimodal detection data collected under multimodal conditions for the same physical area of the display panel, and defect data labels that annotate the multimodal detection data. A baseline teacher model is obtained by training based on a standard evaluation sample pool; The multimodal detection data from the standard sample data is input into the benchmark teacher model, and the benchmark teacher model outputs the benchmark predicted defect data of the multimodal detection data. Obtain the actual modes among the multimodal modes that the defect detection model running on the defect detection equipment can access; Select actual modal detection data acquired under actual modal conditions from the standard sample data; The parameters of the defect detection model are optimized based on actual modal detection data and benchmark predicted defect data.
[0005] In some embodiments of this disclosure, the parameters of the defect detection model are optimized based on actual modal detection data and benchmark predicted defect data, including: The actual modal detection data is input into the defect detection model, and the defect detection model outputs the actual predicted defect data. The distillation loss data between the actual predicted defect data and the baseline predicted defect data is calculated based on the distillation loss function. Based on distillation loss data, the parameters of the defect detection model are optimized so that the distillation loss data meets the set distillation conditions.
[0006] In some embodiments of this disclosure, the baseline predicted defect data includes: baseline predicted defect category data, baseline predicted defect box location data, and baseline predicted defect physical size data; The actual predicted defect data includes: actual predicted defect category data, actual predicted defect box location data, and actual predicted defect physical size data.
[0007] In some embodiments of this disclosure, the distillation loss data between the actual predicted defect data and the baseline predicted defect data is calculated based on the distillation loss function, including: Based on the category probability distribution loss function in the distillation loss function, calculate the category probability distribution distillation loss value between the standard predicted defect category data and the actual predicted defect category data; Based on the defect box location loss function in the distillation loss function, calculate the defect box location distillation loss value between the standard predicted defect box location data and the actual predicted defect box location data. Based on the defect physical size loss function in the distillation loss function, calculate the defect physical size distillation loss value between the standard predicted defect physical size data and the actual predicted defect physical size data; Calculate the first distillation product of the category probability distribution distillation loss value and its preset category adjustment coefficient, the second distillation product of the defect box location distillation loss value and its preset location adjustment coefficient, and the third distillation product of the defect physical size distillation loss value and its preset size adjustment coefficient. The sum of the first distillation product, the second distillation product, and the third distillation product is used as the distillation loss data between the actual predicted defect data and the baseline predicted defect data.
[0008] In some embodiments of this disclosure, the parameters of the defect detection model are optimized based on distillation loss data to ensure that the distillation loss data meets set distillation conditions, including: Based on distillation loss data, the parameters of the defect detection model are optimized so that the sum of the first product, the second product, and the third product is less than or equal to the set distillation loss threshold.
[0009] In some embodiments of this disclosure, multimodality includes: Bright-field image acquisition mode that performs bright-field sampling on the same physical area of the display panel; Dark field image acquisition mode that performs dark field sampling on the same physical area of a display panel; Transmission image acquisition mode that performs transmission sampling on the same physical area of a display panel; and, The perturbation range mode of process characteristic quantities when preparing the same physical area of a display panel.
[0010] In some embodiments of this disclosure, a baseline teacher model is obtained by training based on a standard evaluation sample pool, including: The multimodal detection data is input into the baseline teacher initial model, and the baseline teacher initial model outputs the initial predicted defect data. Based on the training objective loss function, calculate the training objective loss data between the initial predicted defect data and the defect data labels; Based on the training target loss data, the parameters of the baseline teacher initial model are adjusted until the training target loss data meets the set training conditions, and the training ends. The baseline teacher initial model obtained after training is used as the baseline teacher model.
[0011] In some embodiments of this disclosure, the training objective loss data between the initial predicted defect data and the defect data labels is calculated based on the training objective loss function, including: Based on the defect judgment loss function in the training objective loss function, calculate the defect judgment training loss value between the initial predicted defect data and the defect data label; Based on the defect type classification loss function in the training objective loss function, calculate the defect classification training loss value between the initial predicted defect data and the defect data label; Based on the defect location box loss function in the training objective loss function, calculate the defect location box training loss value between the initial predicted defect data and the defect data label; Based on the defect physical size loss function in the training objective loss function, calculate the defect physical size training loss value between the initial predicted defect data and the defect data label; Calculate the defect judgment training loss value and the first training product value of its preset first weight coefficient, the defect classification training loss value and the second training product value of its preset second weight coefficient, the defect physical size training loss value and the third training product value of its preset third weight coefficient, and the defect physical size training loss value and the fourth training product value of its preset fourth weight coefficient. The sum of the first training product, the second training product, the third training product, and the fourth training product is used as the initial training target loss data between the predicted defect data and the defect data label.
[0012] In some embodiments of this disclosure, the method further includes: During the process of the defect detection model running on the defect detection equipment to detect defects in the display panel, the evaluation image is obtained by sampling the defect detection image input to the defect detection model; The evaluation images are input into the benchmark teacher model, and the benchmark teacher model outputs the benchmark evaluation defect data of the evaluation images. The defect detection model is evaluated based on the benchmark assessment defect data and the actual defect detection data output by the defect detection model in response to the assessment image input to it, and the evaluation result is obtained. Based on the evaluation results, the parameters of the defect detection model were optimized.
[0013] In some embodiments of this disclosure, an evaluation result is obtained by evaluating the defect detection model based on benchmark evaluation defect data and actual defect detection data output by the defect detection model in response to an input evaluation image, including: The defect detection model is applied to the defect detection equipment to obtain the actual throughput of the display panel defect detection process; Based on the benchmark assessment defect data and the actual defect detection data, the actual values of the assessment indicators are calculated. The actual values of the assessment indicators include at least the following: the actual defect detection rate of the defect detection model, the actual defect false detection rate, the actual defect box positioning accuracy, the actual repeatability positioning accuracy, and the actual minimum detectable defect size. The defect detection model is evaluated based on the preset target throughput and target values of the evaluation indicators, as well as the actual throughput and actual values of the evaluation indicators, to obtain the evaluation results.
[0014] A second aspect of this disclosure provides an optimization apparatus for a display panel defect detection model, the optimization apparatus comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the optimization method for the display panel defect detection model provided in any one of the first aspects.
[0015] A third aspect of this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to execute an optimization method for a display panel defect detection model provided in any of the first aspects.
[0016] A fourth aspect of this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform an optimization method for a display panel defect detection model provided in any of the first aspects.
[0017] The optimization method for the display panel defect detection model provided in this disclosure constructs a standard evaluation sample pool and trains a benchmark teacher model based on the standard evaluation sample pool. Then, based on the actual modal detection data collected by the defect detection model in the actual modal from the standard sample data and the benchmark predicted defect data output by the benchmark teacher model based on the multimodal detection data, the parameters of the defect detection model are optimized. This ensures that different defect detection models are all optimized based on the benchmark predicted defect data output by the benchmark teacher model, thereby making the algorithm optimization benchmark of different defect detection algorithms consistent and improving the problem of inconsistent algorithm optimization benchmarks of different defect detection algorithms in related technologies. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 A flowchart illustrating an optimization method for a display panel defect detection model provided in an embodiment of this disclosure; Figure 2 This is a schematic flowchart illustrating the optimization of parameters of a defect detection model according to an embodiment of the present disclosure. Figure 3 A flowchart illustrating the process of calculating distillation loss data between actual predicted defect data and baseline predicted defect data based on a distillation loss function, according to an embodiment of this disclosure. Figure 4 A schematic diagram illustrating the process of training a benchmark teacher model based on a standard evaluation sample pool, as provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of a process for calculating the training target loss data between the initial predicted defect data and the defect data labels, provided in an embodiment of the present disclosure. Figure 6 This is a schematic diagram illustrating the optimized process of a defect detection model provided in one embodiment of the present disclosure running on a defect detection device to detect defects in a display panel. Figure 7 This is a schematic diagram illustrating the process of evaluating a defect detection model to obtain evaluation results, provided as an embodiment of the present disclosure. Figure 8 This is a schematic diagram of the overall closed-loop process shown in one embodiment of the present disclosure; Figure 9This is a schematic diagram illustrating the structure and training process of a multimodal Teacher model according to an embodiment of the present disclosure; Figure 10 This is a schematic diagram of a knowledge distillation process according to an embodiment of the present disclosure; Figure 11 This is a flowchart illustrating the closed-loop monitoring process for edge inference and online data transmission according to an embodiment of this disclosure; Figure 12 This is a flowchart illustrating a unified algorithm evaluation metric and report generation process according to an embodiment of the present disclosure; Figure 13 A block diagram of an electronic device provided in an embodiment of this disclosure.
[0020] It should be noted that the elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation
[0021] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.
[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0023] In this disclosure, the terms “upper,” “lower,” “left,” “right,” “front,” “rear,” “top,” “bottom,” “inner,” “outer,” and “middle,” etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. These terms are primarily for the purpose of better describing this disclosure and its embodiments, and are not intended to limit the indicated devices, elements, or components to having a specific orientation, or to be constructed and operated in a specific orientation.
[0024] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in certain circumstances to indicate a dependency or connection. Those skilled in the art can understand the specific meaning of these terms in this disclosure according to the specific circumstances.
[0025] Furthermore, the terms "set up," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection via an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this disclosure according to the specific circumstances.
[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.
[0027] The optimization of different defect detection algorithms in related technologies has the following drawbacks: (1) The algorithm indicators of different equipment manufacturers are not comparable, and the indicators such as detection rate, false detection rate, and repeatability accuracy are inconsistent; (2) The performance of the same algorithm fluctuates greatly under different working conditions (bright field / dark field / transmission, multiple process residues, scratches, short circuits, open circuits and other defect types), and there is a lack of robust evaluation benchmarks; (3) A set of reusable closed-loop processes of "algorithm performance evaluation → gap diagnosis → optimization suggestions → iterative verification" has not been formed, which leads to the algorithm remaining in empirical fine-tuning for a long time and unable to be improved in an engineering manner.
[0028] Example 1 To address at least some of the aforementioned problems, embodiments of this disclosure provide an optimization method for a display panel defect detection model, aiming to improve the inconsistency in the optimization benchmarks of different defect detection algorithms. For example... Figure 1 As shown, the optimization method for the display panel defect detection model mainly includes the following steps: Step S110: Construct a standard evaluation sample pool. The standard evaluation sample pool includes multiple standard sample data. Each standard sample data includes: multimodal detection data collected under multimodal conditions for the same physical area of the display panel, and defect data labels that annotate the multimodal detection data. Step S120: Train the baseline teacher model based on the standard evaluation sample pool; Step S130: Input the multimodal detection data from the standard sample data into the benchmark teacher model, and output the benchmark predicted defect data of the multimodal detection data from the benchmark teacher model; Step S140: Obtain the actual modes among the multimodal modes that the defect detection model running on the defect detection equipment can access; Step S150: Select actual modal detection data acquired under actual modal conditions from the standard sample data; Step S160: Optimize the parameters of the defect detection model based on actual modal detection data and benchmark predicted defect data.
[0029] In the above scheme, a standard evaluation sample pool is constructed, and a benchmark teacher model is trained based on this pool. Then, the parameters of the defect detection model are optimized using actual modal detection data collected from the standard sample data under real-world conditions, and benchmark predicted defect data output by the benchmark teacher model based on multimodal detection data. This ensures that different defect detection models are optimized based on the benchmark predicted defect data output by the benchmark teacher model. This mechanism is equivalent to automatically finding the "most reliable information source" for each working condition, avoiding the manual fixing of weights in traditional methods, and improving the adaptability of the optimized defect detection model algorithm. This ensures that the algorithm optimization benchmarks of different defect detection algorithms are consistent, thus improving the problem of inconsistent algorithm optimization benchmarks among different defect detection algorithms in related technologies. The following section, in conjunction with the appendix... Figures 1 to 11 The method disclosed herein is described in detail.
[0030] First, refer to Figure 1 A standard evaluation sample pool is constructed. This standard evaluation sample pool includes multiple standard sample data, each of which includes: multimodal detection data collected under multimodal conditions for the same physical area of the display panel, and defect data labels that annotate the multimodal detection data.
[0031] It should be noted that multimodal refers to both different modes distinguished by different process characteristic parameters when preparing a certain physical area of the display panel and modes determined by different image acquisition methods when acquiring images of a certain physical area of the display panel.
[0032] For example, the multimodal mode may include at least the following: a bright-field image acquisition mode that performs bright-field sampling for the same physical area of the display panel, a dark-field image acquisition mode that performs dark-field sampling for the same physical area of the display panel, a transmission image acquisition mode that performs transmission sampling for the same physical area of the display panel, and a process feature perturbation range mode when fabricating the same physical area of the display panel.
[0033] For example, the bright field image acquisition mode emphasizes surface morphology, particles, scratches, and other appearance defects of the display panel; the dark field image acquisition mode can enhance the display panel's surface detail, highlighting fine scratches, pits, and other scattering features; and the transmission image acquisition mode can emphasize circuit breaks, short circuits, islands, holes, and other defects. Figure 1 Inconsistency defects; the perturbation range mode of process characteristic quantities can be obtained by the perturbation range of process characteristics such as film thickness, line width, line spacing, etc., which, for example, can include records within a perturbation range of ±10%.
[0034] In the process of acquiring multimodal detection data for the same physical area of a display panel under multimodal conditions, the following standardization process can be performed: For the same physical area of the display panel, establish synchronized timestamps for multimodal synchronous acquisition. Label each surface sample of the display panel with defect data. These defect data labels can include: defect category (short circuit / open circuit / island / hole, etc.), defect location (pixel coordinates), defect physical size (μm level), and defect severity level. Then, the multimodal detection data acquired for the same physical area of the display panel under multimodal conditions, along with the defect data labels annotated with the multimodal detection data, can be stored in a unified defect verification database to form a standard evaluation sample pool. The standard evaluation sample pool can contain at least 10 million standard sample data points, and at least 50 defect categories.
[0035] The significance of step S110 above is that it provides a unified input source for all subsequent "defect detection model algorithm evaluations", avoiding different defect detection devices from claiming high performance only on their own preferred datasets; at the same time, it supports cross-condition generalization evaluation, that is, the performance differences of the same defect detection model algorithm in different modalities / different processes / different cleanliness levels can be evaluated, thereby improving the evaluation capability.
[0036] Next, refer to Figure 1 A benchmark teacher model is obtained by training based on a standard evaluation sample pool. One of the key innovations of this disclosure is the introduction of a "benchmark teacher model," which does not run directly in the defect detection equipment on the production line, but rather in the high-computing environment of a pilot-scale platform. This benchmark teacher model is used to: learn the optimal feature representation after multimodal fusion, generate high-quality pseudo-labels / confidence scores, and provide a unified benchmark for defect detection model algorithms in various defect detection equipment.
[0037] For example, refer to Figure 4The benchmark teacher model, trained based on a standard evaluation sample pool, can include the following steps: inputting multimodal detection data into the initial benchmark teacher model, which then outputs initial predicted defect data; calculating the training objective loss data between the initial predicted defect data and the defect data labels based on the training objective loss function; adjusting the parameters of the initial benchmark teacher model based on the training objective loss data until the training objective loss data meets the set training conditions, and then using the resulting initial benchmark teacher model as the benchmark teacher model. This approach facilitates the acquisition of a more accurate benchmark teacher model.
[0038] There are several ways to calculate the training target loss data between the initial predicted defect data and the defect data label based on the training target loss function. Some of these methods are illustrated below.
[0039] For example, refer to Figure 5 Based on the training objective loss function, the training objective loss data between the initial predicted defect data and the defect data labels can be calculated, and may include: Based on the defect judgment loss function in the training objective loss function, calculate the defect judgment training loss value between the initial predicted defect data and the defect data label; Based on the defect type classification loss function in the training objective loss function, calculate the defect classification training loss value between the initial predicted defect data and the defect data label; Based on the defect location box loss function in the training objective loss function, calculate the defect location box training loss value between the initial predicted defect data and the defect data label; Based on the defect physical size loss function in the training objective loss function, calculate the defect physical size training loss value between the initial predicted defect data and the defect data label; Calculate the defect judgment training loss value and the first training product value of its preset first weight coefficient, the defect classification training loss value and the second training product value of its preset second weight coefficient, the defect physical size training loss value and the third training product value of its preset third weight coefficient, and the defect physical size training loss value and the fourth training product value of its preset fourth weight coefficient. The sum of the first training product, the second training product, the third training product, and the fourth training product is used as the initial training target loss data between the predicted defect data and the defect data label.
[0040] That is, in the above embodiment, the training target loss function includes four loss function branches: defect judgment loss function, defect type classification loss function, defect location box loss function, and defect physical size loss function, and corresponding weight coefficients are set for each branch. This makes the parameter settings of the baseline teacher model obtained after training more accurate, thereby improving the defect detection accuracy of the baseline teacher model.
[0041] There are several ways to set up the baseline teacher initial model. Some methods are illustrated below. For example, the structure of the baseline teacher initial model may include the following functions: Multi-branch feature extraction: For three image acquisition modes—bright field image acquisition mode, dark field image acquisition mode, and transmission image acquisition mode—one convolution / transformer branch is set for each mode to extract modal features F_bright, F_dark, and F_trans. Process parameter encoding: Encode continuous process parameters such as film thickness, line width, and line spacing of a certain physical area of the display panel into a feature vector F_proc (which can be done using MLP or Embedding). Fusion layer: The above features are concatenated or attention fusion (Cross-Attention / Multi-headAttention) is used to obtain a unified feature F_fused; Inspection head and measurement head: On top of F_fused, the target inspection head (predicting defect category + location box) and the size regression head (predicting actual defect size and shape contour) are respectively connected.
[0042] For example, a multimodal fusion mathematical description of the aforementioned benchmark teacher model may include: For each mode m∈{bright,dark,trans}, extract the feature F_m = Enc_m(I_m).
[0043] Perform a nonlinear mapping on the process parameter vector p (such as film thickness, linewidth, line spacing, etc.): F_proc = MLP(p).
[0044] Fusion: F_fused = Attn( [F_bright, F_dark, F_trans, F_proc] ), where Attn(·) can be multi-head self-attention or interactive attention. The core idea is to allow different modalities to perform weighted interactions in the semantic space and learn the most discriminative feature representation under the current technology conditions.
[0045] For example, the expression for the training target loss function can be as follows: L_total = λ1 L_det + λ2 L_cls + λ3 L_loc + λ4 L_size Where: L_total represents the training target loss data between the initial predicted defect data and the defect data label; L_det represents the defect judgment training loss value calculated by the defect type classification loss function based on the binary / multi-class loss (such as cross-entropy) of whether there is a defect; L_cls represents the defect classification training loss value calculated by the defect type classification loss function; L_loc represents the defect location box training loss value calculated by the defect location box regression loss (such as Smooth L1 or IoU Loss) calculated by the defect location box loss function; L_size represents the defect physical size training loss value calculated by the defect physical size loss function based on the regression loss of the true physical size of the defect (used for measurement accuracy evaluation).
[0046] The significance of the benchmark teacher model obtained through the aforementioned multimodal joint training is as follows: The benchmark teacher model can provide the "optimal judgment" while integrating multimodal features and process fluctuation parameters, which in practice often outperforms any single defect detection device's defect detection model algorithm. The high-quality labels and confidence scores output by the benchmark teacher model serve as a benchmark for subsequent comparison and evaluation between knowledge distillation and defect detection model algorithms of defect detection devices. Multimodal attention fusion can use different modal features and process parameter features as input sequences for multi-head attention, adaptively emphasizing the importance of a particular modality in the channel and spatial dimensions with learnable weights.
[0047] Next, refer to Figure 1 The multimodal detection data from the standard sample data is input into the benchmark teacher model, which outputs the benchmark predicted defect data from the multimodal detection data. Subsequently, during the optimization of the defect detection model's parameters, the benchmark predicted defect data from the multimodal detection data output by the benchmark teacher model is used as the optimization benchmark.
[0048] It should be noted that the defect detection model running on the defect detection equipment can be a lightweight Student model. In actual production line environments, defect detection equipment is usually limited by computing power, power consumption, and latency (it needs to keep up with the production pace). Therefore, this disclosure proposes to train a complex benchmark teacher model on a pilot-scale platform, and then transfer its capabilities to a lightweight Student model (the defect detection model) through knowledge distillation. The latter is actually deployed on an edge computing node or the industrial PC / GPU of the device itself.
[0049] Next, refer to Figure 1This involves obtaining the actual modalities from the multimodal datasets that the defect detection model running on the defect detection equipment can access. Since the defect detection model may not receive all modalities from the standard evaluation sample pool during optimization training (for example, the field defect detection equipment may only be able to obtain bright field + dark field), it is only necessary to obtain the actual modalities from the multimodal datasets that the defect detection model can access from the standard evaluation sample pool.
[0050] Next, refer to Figure 1 The actual modal detection data collected under actual modal conditions is selected from the standard sample data. The actual modal detection data is used as the input data source when optimizing the defect detection model, so that different defect detection models can use the same input data source and the prediction results of different defect detection models can be compared.
[0051] Next, refer to Figure 1 Based on actual modal detection data and benchmark predicted defect data, the parameters of the defect detection model are optimized. Since the defect detection model may not receive all modalities from the standard evaluation sample pool during optimization training (for example, on-site defect detection equipment may only acquire bright field + dark field data), the model learns to approximate the output distribution of the benchmark teacher model during optimization training. This ensures that different defect detection models optimize their parameters based on the benchmark predicted defect data output by the benchmark teacher model, thereby ensuring a consistent optimization benchmark for different defect detection algorithms and improving the problem of inconsistent optimization benchmarks among different defect detection algorithms in related technologies.
[0052] It should be noted that, considering the defect detection model runs on the production line, the requirements for the defect detection model as the Student model are as follows: low inference latency (meeting the requirement of an online detection cycle of ≥50 frames / second, specifically matching the resolution); low memory / GPU memory usage, adaptable to edge GPUs or high-performance CPUs; and accuracy close to the output of the benchmark Teacher model, especially in the detection capability of key defects (short circuits, open circuits, holes, islands, etc.).
[0053] The defect detection model that can be used as the Student model can include: lightweight convolutional neural networks (such as MobileNet, EfficientNet-lite, YOLO-tiny variants); low bit quantization (8-bit / 16-bit quantization inference) to accelerate matrix operations; CUDA kernel acceleration and TensorRT optimization (to achieve higher frame rates on GPU / edge computing modules).
[0054] When optimizing the parameters of a defect detection model based on actual modal detection data and benchmark predicted defect data, various methods can be used, some of which are exemplified below.
[0055] For example, refer to Figure 2 Based on actual modal detection data and baseline predicted defect data, the parameters of the defect detection model are optimized. This can include: inputting actual modal detection data into the defect detection model and having the defect detection model output actual predicted defect data; calculating the distillation loss data between the actual predicted defect data and the baseline predicted defect data based on the distillation loss function; and optimizing the parameters of the defect detection model based on the distillation loss data so that the distillation loss data meets the set distillation conditions.
[0056] For example, as described above, the baseline predicted defect data may include: baseline predicted defect category data, baseline predicted defect box location data, and baseline predicted defect physical size data; the actual predicted defect data may include: actual predicted defect category data, actual predicted defect box location data, and actual predicted defect physical size data. This improves the accuracy of defect detection results.
[0057] For example, there are several ways to calculate the distillation loss data between the actual predicted defect data and the baseline predicted defect data based on the distillation loss function. Some of these methods are described below.
[0058] For example, refer to Figure 3 Calculating the distillation loss data between the actual predicted defect data and the baseline predicted defect data based on the distillation loss function may include the following steps: Based on the category probability distribution loss function in the distillation loss function, calculate the category probability distribution distillation loss value between the standard predicted defect category data and the actual predicted defect category data; Based on the defect box location loss function in the distillation loss function, calculate the defect box location distillation loss value between the standard predicted defect box location data and the actual predicted defect box location data. Based on the defect physical size loss function in the distillation loss function, calculate the defect physical size distillation loss value between the standard predicted defect physical size data and the actual predicted defect physical size data; Calculate the first distillation product of the category probability distribution distillation loss value and its preset category adjustment coefficient, the second distillation product of the defect box location distillation loss value and its preset location adjustment coefficient, and the third distillation product of the defect physical size distillation loss value and its preset size adjustment coefficient. The sum of the first distillation product, the second distillation product, and the third distillation product is used as the distillation loss data between the actual predicted defect data and the baseline predicted defect data.
[0059] For example, the distillation loss function can be expressed as follows: L_distill = α L_cls_distill + β L_loc_distill + γ L_size_distill Where: L_distill represents the distillation loss between the actual predicted defect data and the baseline predicted defect data. L_cls_distill represents the class probability distribution distillation loss value, which physically means to make the class probability distribution of the defect detection model (Student model) approximate the soft label distribution (baseline predicted defect data) of the baseline teacher model (usually using KL divergence or temperature-weighted cross-entropy). L_loc_distill represents the defect box location distillation loss value, which physically means to make the defect box location regression result of the defect detection model (Student model) close to the baseline predicted defect box location data. L_size_distill represents the defect physical size distillation loss value, which physically means to make the defect physical size predicted by the defect detection model (Student model) approximate the defect physical size estimate of the baseline teacher model (Teacher model).
[0060] For example, the distillation loss function L_distill can also be expressed as follows.
[0061] L_distill = α KL(P_Teacher || P_Student) + β SmoothL1(Box_Teacher,Box_Student) + γ L_size_distill.
[0062] Specifically, the KL term promotes the distribution of Student in category determination to closely resemble that of Teacher; the SmoothL1 term promotes consistency between spatial localization and contour regression; and the L_size_distill term promotes the defect detection model, acting as Student, to learn the "actual physical size of defects" estimation ability of the benchmark teacher model, acting as Teacher, thus enabling the output of quantifiable size analysis at the device end. The L_size constraint model outputs quantifiable real defect sizes, achieving a minimum detectable capability assessment at the 0.4μm level.
[0063] refer to Figure 9The Teacher model shown integrates multimodal image information (bright field, dark field, transmission) and process context parameters (film thickness, linewidth, line spacing, etc.), and generates unified, highly discriminative features through an attention-based feature interaction layer. The Teacher model outputs not only defect categories and locations but also physical dimensions. This capability provides a high-precision benchmark for the subsequent evaluation of distillation and equipment measurement performance in the defect detection model, which acts as the Student.
[0064] Figure 10 A key point of this disclosure is that the method does not simply deploy a regular detection network on the device. Instead, it compresses the high-precision decision-making strategy of the Teacher model (which includes multimodal information understanding and process context awareness) into a lightweight Student (a defect detection model running on the defect detection device) that can run on an edge device through knowledge distillation. This enables it to maintain a high detection rate, a low false detection rate, and quantifiable size output even under conditions of low computing power and few modal inputs.
[0065] For example, optimizing the parameters of a defect detection model based on distillation loss data to ensure that the distillation loss data meets set distillation conditions can include: optimizing the parameters of the defect detection model based on the distillation loss data so that the sum of the first product, the second product, and the third product is less than or equal to a set distillation loss threshold. This reduces the loss value of knowledge distillation and improves the effectiveness of knowledge distillation.
[0066] In the above approach, knowledge distillation (Teacher→Student) is used as the core bridge in this disclosure to transfer the high-quality "judgment logic" of the benchmark teacher model obtained based on multimodal training to the on-site runnable defect detection model (Student model).
[0067] The inputs in the distillation process described above are: the same batch of multimodal data (bright field / dark field / transmission / process), and the distillation monitoring is based on: high-confidence predictions (category distribution, location box, size prediction, etc.) generated by the (benchmark teacher model).
[0068] The key innovation mentioned above lies in the fact that even if the defect detection model acting as the Student cannot simultaneously access all modalities (due to hardware limitations of defect detection equipment), it can still "see" the multimodal results reasoned by the baseline teacher model acting as the Teacher during the optimization training phase of the defect detection model, and learn the cross-modal discrimination boundary through distillation. This enables the defect detection model acting as the Student to still possess near-multimodal fusion judgment capabilities under single-modal conditions.
[0069] The industrial value of the above model lies in: bringing the capabilities of complex algorithms down to edge defect detection equipment, achieving high precision and high real-time performance; defect detection equipment manufacturers can periodically upgrade the pilot-scale platform disclosed herein to continuously improve the on-site algorithms without significantly modifying the hardware.
[0070] For example, refer to Figure 6 The method may further include: during the process of a defect detection model running on a defect detection device to detect defects in a display panel, sampling an evaluation image from a defect detection image input to the defect detection model; inputting the evaluation image into a benchmark teacher model, which outputs benchmark evaluation defect data of the evaluation image; evaluating the defect detection model based on the benchmark evaluation defect data and the actual defect detection data output by the defect detection model in response to the input evaluation image to obtain an evaluation result; and optimizing the parameters of the defect detection model based on the evaluation result. That is, this method can also be applied to the application of defect detection models for real-time monitoring and optimization, thereby ensuring that the parameters of the defect detection model can be updated in real time and ensuring the consistency of output results from different defect detection models.
[0071] For example, the evaluation results can be obtained by evaluating the defect detection model based on the benchmark evaluation defect data and the actual defect detection data output by the defect detection model in response to the evaluation image input to it. Some methods are described below as examples.
[0072] For example, refer to Figure 7 The evaluation of the defect detection model, based on benchmark defect data and actual defect detection data output by the model in response to the input evaluation image, may include: obtaining the actual throughput of the defect detection model applied to the defect detection equipment for the defect detection process of the display panel; calculating actual values of evaluation indicators based on the benchmark defect data and actual defect detection data, wherein the actual values of the evaluation indicators include at least the following: the actual defect detection rate, actual false detection rate, actual defect box positioning accuracy, actual repeatability positioning accuracy, and actual minimum detectable defect size of the defect detection model; and evaluating the defect detection model based on a preset target throughput and target values of the evaluation indicators, as well as the actual throughput and actual values of the evaluation indicators, to obtain the evaluation results.
[0073] For example, the edge computing deployment and online operation feedback function deployed on the defect detection device may include: a defect detection model (Student model) running on the defect detection device as an edge node, performing real-time inference on the defect detection device side / edge node, and outputting the defect location, type, and physical size; while recording the defect detection model's throughput, latency, frame loss rate, number of false alarms, and other operating indicators.
[0074] For example, during the operation of the defect detection model, data is reported. Key sampling frames (evaluation images) and operational metrics (actual defect detection data output by the defect detection model based on the evaluation images) are sent back to the pilot-scale platform, forming a "field performance curve." After receiving this data, the pilot-scale platform inputs it into the benchmark teacher model to obtain benchmark evaluation defect data. Subsequently, based on the benchmark evaluation defect data and the actual defect detection data, the actual values of evaluation metrics such as online detection rate trend, false positive rate drift, and repeatability degradation are calculated.
[0075] For example, this disclosure uses the defect detection model algorithm of the defect detection equipment as the Algorithm Under Test (AUT), defines a unified set of evaluation tasks and indicators for the AUT, and then calculates the actual values of the evaluation indicators. The evaluation tasks and indicators may include: Defect detection task to locate defect areas; Defect classification task to distinguish defect types (short circuit / open circuit / island / hole, etc.). The Defect Physical Size Measurement Task (Metrology) is used to estimate the physical size of defects (such as the smallest detectable size at the 0.4 μm level) and shape boundaries. Repeatability is a task used to evaluate the consistency of results from multiple scans of the same physical area of a display panel. Throughput is a real-time task used to evaluate frame rate / processing latency at a target production cycle time.
[0076] The calculation of the indicators may include the following: Defect detection rate Recall = TP / (TP + FN), for example, a defect detection rate of ≥98% can be required; False Alarm Rate = FP / (TP + FP), for example, the false alarm rate can be required to be ≤2%; Defect location accuracy can be achieved by, for example, requiring that the IoU (Intersection over Union) between the defect box / contour and the manually annotated area be greater than or equal to a certain threshold (e.g., 0.7). Repeatability accuracy, for example, the standard deviation of multiple measurements of the same defect center coordinates ≤ 1 μm; The smallest detectable defect size, the smallest equivalent diameter of the defect that the defect detection model algorithm can still stably detect (requiring an analysis capability at the 0.4 μm level). The throughput of the defect detection model: the processing speed (Frame / s) is not lower than the production line cycle time (e.g., ≥50 frames / second, depending on the panel resolution).
[0077] The mathematical form for calculating the above indicators can be taken as follows: IoU(predicted, labeled) = Area(predicted∩ labeled) / Area(predicted∪ labeled); Repeatability Where pi is the coordinate of the defect center in the i-th measurement. The mean; Multimodal robustness score R = Σ_k w_k Acc_k, where Acc_k is the accuracy of the defect detection model algorithm under the k-th operating condition / modality, and w_k is the corresponding weight. This score compresses the performance across operating conditions into a single score, which can be used to compare the stability and reliability of algorithms from different equipment manufacturers. For example, dynamic robustness scoring can also be performed. For instance, for a perturbation scenario of ±10% of process feature quantities such as film thickness, linewidth, and line spacing, the changes in various indicators of the defect detection model (ΔRecall, ΔFalseAlarm, ΔLatency) are analyzed; a robustness score R_online is generated and compared with the laboratory baseline R_lab to identify specific weak operating conditions.
[0078] Figure 8 This paper presents a closed-loop methodology, which involves collecting multimodal detection data from real-world near-production line conditions, forming a standardized benchmark dataset (standard evaluation sample pool) on a pilot-scale platform, and training a high-precision multimodal Teacher model (benchmark teacher model). Subsequently, knowledge distillation is used to obtain a Student model (defect detection model) that can run in real time on the equipment side. The Student model (defect detection model) performs online detection on edge hardware and transmits back operational metrics. The pilot-scale platform quantitatively evaluates the Student model (defect detection model) on the equipment and outputs optimization suggestions to continuously upgrade the algorithm performance.
[0079] Figure 11 The demonstration focuses on "what happens after deployment." The Student model (defect detection model) is not a static black box; while it runs in the field, a monitoring agent collects performance metrics and sample images. This data is then fed back to the pilot platform for quantitative analysis, generating structured evaluation reports and optimization suggestions for the next round. This means the algorithm can iterate continuously at the customer's site, rather than being a one-time delivery.
[0080] Figure 12This disclosure describes how the defect detection model algorithm outputs of various defect detection devices are aligned item by item with a unified Teacher benchmark, and a set of standardized metrics are calculated, including detection rate, false detection rate, location accuracy, repeatability, minimum detectable defect size (measuring capability), real-time throughput, and robustness score under ±10% process perturbation. The evaluation results are automatically compiled into a structured report and linked with improvement suggestions, which can be directly used by manufacturers to iterate algorithms or optimize hardware parameters.
[0081] The final step in this disclosure is to generate a structured evaluation report from all comprehensive metrics and automatically propose improvement suggestions. For example, the report may include: defect detection rate, defect false detection rate, repeatability accuracy, minimum detectable defect size (target 0.4 μm), real-time throughput, robustness score, etc. The report highlights the sources of discrepancies, such as "high false detection rate for minor scratch defects in dark-field mode" and "significantly increased missed detection of short-circuit defects in scenarios with a film thickness of +10%". The system can automatically recommend optimization paths based on the problem type. For example, whether it is necessary to expand the training samples for a certain type of defect; whether it is necessary to introduce additional modal or process context features; whether it is necessary to re-distill to improve the discriminative power of a specific modality; whether there is an inference latency bottleneck, and recommend CUDA / TensorRT-level optimization.
[0082] In summary, the optimization method for the defect detection algorithm in the display panel surface defect detection equipment described above is a systematic approach for objectively and quantitatively evaluating, jointly modeling and training, edge-deploying, and optimizing the performance of the defect detection model algorithm in the display panel surface defect detection equipment. It does not simply improve the defect detection model algorithm itself, but rather provides a systematic algorithmic process centered around "algorithm evaluation + algorithm improvement suggestions." Relying on standardized data from the pilot-scale platform, standardized operating condition simulation, and edge computing deployment system, it quantifies, diagnoses, and iteratively improves the performance of the equipment algorithm in real near-production line scenarios. This solves the problems of existing detection equipment algorithms being unable to be uniformly evaluated, difficult to compare horizontally, and unable to automatically provide improvement directions.
[0083] The above method mainly includes the following seven key steps: (1) Multimodal detection data acquisition and standardization; (2) Evaluation task definition and index calculation; (3) Multimodal joint training benchmark model (Teacher model); (4) Lightweight inference model deployed at the edge (Student model); (5) Knowledge distillation process (Teacher→Student); (6) On-site operation performance feedback and robustness analysis; (7) Automatically generate performance evaluation reports and optimization suggestions.
[0084] The above methods can be applied to the following scenarios: (1) When display panel manufacturers import / compare testing equipment, they can conduct objective horizontal evaluation of the testing algorithms of different suppliers; (2) Testing equipment manufacturers can perform performance regression verification before and after algorithm version iteration; (3) During the production line introduction stage (4.5 generation line and above OLED pilot line), the stability of the algorithm under dynamic cycle time, cleanliness disturbance, film thickness / line width / line spacing ±10% process disturbance is verified; (4) Online continuous optimization after on-site deployment: Data and indicators are collected through edge nodes, new improvement suggestions are generated periodically, and a continuous iteration algorithm maintenance process is formed.
[0085] As can be seen from the above description, this disclosure achieves the following technical effects: It constructs a unified multimodal detection data benchmark and indicator system; establishes an authoritative, unified, and traceable algorithm evaluation benchmark using a multimodal jointly trained Teacher model; performs multidimensional quantitative evaluation of the algorithms of detection equipment (detection rate, false detection rate, repeatability, minimum detectable defect size, etc.); and, through knowledge distillation and edge deployment, pushes high-precision inference capabilities down to the equipment site, balancing real-time performance and accuracy; generates executable performance improvement strategies (model structure optimization, inference acceleration, robustness enhancement, etc.) based on the evaluation results; quantifies the merits and demerits of algorithms from different equipment manufacturers through a unified indicator system and robustness scoring; and deploys optimization suggestions to the equipment side through knowledge distillation and edge compression to achieve practically implementable performance upgrades. Through automatically generated optimization suggestions, it provides a clear performance improvement path, realizing engineering implementation from pilot-scale platforms to mass production lines.
[0086] Example 2 This disclosure also provides an optimization apparatus for a display panel defect detection model, the optimization apparatus comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the optimization method for the display panel defect detection model provided in any of the first aspects.
[0087] The specific methods of execution of each unit in the above device embodiments have been described in detail in Embodiment 1 of the method, and will not be elaborated here.
[0088] Example 3 This disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to execute an optimization method for a display panel defect detection model provided in any of the first aspects.
[0089] Example 4 This disclosure provides an electronic device, such as... Figure 13 As shown, the electronic device includes: at least one processor 131; and a memory 132 communicatively connected to at least one processor 131; wherein the memory 132 stores a computer program executable by at least one processor 131, the computer program being executed by at least one processor 131 to cause at least one processor 131 to execute the optimization method of the display panel defect detection model provided in any of the first aspects.
[0090] refer to Figure 13 The electronic device includes one or more processors 131 and a memory 132. Figure 13 Taking a processor 131 as an example, the electronic device may also include an input device 133 and an output device 134. The processor 131, memory 132, input device 133, and output device 134 can be connected via a bus or other means. Figure 13 Taking the example of a connection between China and Israel via a bus.
[0091] Processor 131 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. The general-purpose processor can be a microprocessor or any conventional processor.
[0092] The memory 132, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the construction method in the embodiments of this disclosure. The processor 131 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 132, thereby implementing the optimization method of the display panel defect detection model in the above method embodiments.
[0093] Memory 132 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the processing device operated by the server. Furthermore, memory 132 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 132 may optionally include memory remotely located relative to processor 131, and these remote memories may be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0094] Input device 133 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the server's processing device. Output device 134 may include display devices such as a display screen.
[0095] One or more modules are stored in memory 132, and when executed by one or more processors 131, they perform actions such as... Figure 1 The method is shown. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in a different order than that shown here.
[0096] Those skilled in the art will understand that all or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory (FM), hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.
[0097] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.
Claims
1. An optimization method for a display panel defect detection model, characterized in that, include: A standard evaluation sample pool is constructed, which includes multiple standard sample data. Each standard sample data includes: multimodal detection data collected under multimodal conditions for the same physical area of the display panel, and defect data labels that annotate the multimodal detection data. A baseline teacher model is obtained by training based on the aforementioned standard evaluation sample pool; The multimodal detection data in the standard sample data is input into the benchmark teacher model, and the benchmark teacher model outputs the benchmark predicted defect data of the multimodal detection data. Obtain the actual modes among the multimodalities that the defect detection model running on the defect detection equipment can access; Select the actual modal detection data obtained under the actual modality from the standard sample data; Based on the actual modal detection data and the benchmark predicted defect data, the parameters of the defect detection model are optimized.
2. The method as described in claim 1, characterized in that, The optimization of the parameters of the defect detection model based on the actual modal detection data and the benchmark predicted defect data includes: The actual modal detection data is input into the defect detection model, and the defect detection model outputs the actual predicted defect data. The distillation loss data between the actual predicted defect data and the benchmark predicted defect data is calculated based on the distillation loss function. Based on the distillation loss data, the parameters of the defect detection model are optimized so that the distillation loss data meets the set distillation conditions.
3. The method as described in claim 2, characterized in that, The baseline predicted defect data includes: baseline predicted defect category data, baseline predicted defect box location data, and baseline predicted defect physical size data; The actual predicted defect data includes: actual predicted defect category data, actual predicted defect box location data, and actual predicted defect physical size data.
4. The method as described in claim 3, characterized in that, The calculation of distillation loss data between the actual predicted defect data and the baseline predicted defect data based on the distillation loss function includes: Based on the category probability distribution loss function in the distillation loss function, calculate the category probability distribution distillation loss value between the standard predicted defect category data and the actual predicted defect category data; Based on the defect box location loss function in the distillation loss function, calculate the defect box location distillation loss value between the standard predicted defect box location data and the actual predicted defect box location data. Based on the defect physical size loss function in the distillation loss function, calculate the defect physical size distillation loss value between the standard predicted defect physical size data and the actual predicted defect physical size data; Calculate the first distillation product of the category probability distribution distillation loss value and its preset category adjustment coefficient, the second distillation product of the defect box position distillation loss value and its preset position adjustment coefficient, and the third distillation product of the defect physical size distillation loss value and its preset size adjustment coefficient. The sum of the first distillation product, the second distillation product, and the third distillation product is used as the distillation loss data between the actual predicted defect data and the baseline predicted defect data.
5. The method as described in claim 4, characterized in that, The step of optimizing the parameters of the defect detection model based on the distillation loss data to ensure that the distillation loss data meets the set distillation conditions includes: Based on the distillation loss data, the parameters of the defect detection model are optimized so that the sum of the first product, the second product, and the third product is less than or equal to a set distillation loss threshold.
6. The method according to any one of claims 1-5, characterized in that, The multimodality includes: Bright-field image acquisition mode that performs bright-field sampling on the same physical area of the display panel; Dark field image acquisition mode that performs dark field sampling on the same physical area of the display panel; Transmission image acquisition mode that performs transmission sampling over the same physical area of the display panel; and, The perturbation range mode of process feature quantity when preparing the same physical area of the display panel.
7. The method according to any one of claims 1-5, characterized in that, The benchmark teacher model obtained by training based on the standard evaluation sample pool includes: The multimodal detection data is input into the benchmark teacher initial model, and the benchmark teacher initial model outputs the initial predicted defect data. Based on the training objective loss function, calculate the training objective loss data between the initial predicted defect data and the defect data labels; Based on the training target loss data, the parameters of the baseline teacher initial model are adjusted until the training target loss data meets the set training conditions and the training ends. The baseline teacher initial model obtained after the training is completed is then used as the baseline teacher model.
8. The method as described in claim 7, characterized in that, The step of calculating the training objective loss data between the initial predicted defect data and the defect data labels based on the training objective loss function includes: Based on the defect judgment loss function in the training objective loss function, calculate the defect judgment training loss value between the initial predicted defect data and the defect data label; Based on the defect type classification loss function in the training objective loss function, calculate the defect classification training loss value between the initial predicted defect data and the defect data label; Based on the defect location box loss function in the training objective loss function, calculate the defect location box training loss value between the initial predicted defect data and the defect data label; Based on the defect physical size loss function in the training objective loss function, calculate the defect physical size training loss value between the initial predicted defect data and the defect data label; Calculate the defect judgment training loss value and the first training product value of its preset first weight coefficient, the defect classification training loss value and the second training product value of its preset second weight coefficient, the defect physical size training loss value and the third training product value of its preset third weight coefficient, and the defect physical size training loss value and the fourth training product value of its preset fourth weight coefficient. The sum of the first training product value, the second training product value, the third training product value, and the fourth training product value is used as the training target loss data between the initial predicted defect data and the defect data label.
9. The method according to any one of claims 1-5, characterized in that, Also includes: During the process of the defect detection model running on the defect detection device to detect defects in the display panel, an evaluation image is obtained by sampling the defect detection image input to the defect detection model; The evaluation image is input into the benchmark teacher model, and the benchmark teacher model outputs the benchmark evaluation defect data of the evaluation image. Based on the benchmark assessment defect data and the actual defect detection data output by the defect detection model in response to the input assessment image, the defect detection model is evaluated to obtain an assessment result; Based on the evaluation results, the parameters of the defect detection model are optimized.
10. The method as described in claim 9, characterized in that, The evaluation of the defect detection model based on the benchmark evaluation defect data and the actual defect detection data output by the defect detection model in response to the input evaluation image, to obtain an evaluation result, includes: The actual throughput of the defect detection model applied to the defect detection equipment for the defect detection process of the display panel is obtained; Based on the benchmark assessment defect data and the actual defect detection data, the actual values of the assessment indicators are calculated. The actual values of the assessment indicators include at least the following: the actual defect detection rate, the actual defect false detection rate, the actual defect box positioning accuracy, the actual repeatability positioning accuracy, and the actual minimum detectable defect size of the defect detection model. The defect detection model is evaluated based on the preset target throughput and the target value of the evaluation index, as well as the actual throughput and the actual value of the evaluation index, to obtain the evaluation result.