Panel defect detection method, device, equipment, storage medium and program product

Abstract

The application discloses a panel defect detection method, device, equipment, storage medium and program product. The method comprises the following steps: acquiring a particle image patch, a pixel structure image patch and corresponding production line metadata of a panel to be detected; inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, calculating a risk score of the high-dimensional embedding vector, wherein the risk score is determined by a deviation degree item, a rarity degree item and a production line drift penalty item corresponding to the production line metadata; and determining a defect category of the panel to be detected based on the risk score and a preset threshold.

Description

Technical Field

[0001] This application relates to the field of automatic optical inspection technology, and more specifically, to a panel defect detection method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] In the context of Automated Optical Inspection (AOI) in panel manufacturing, precise detection and classification of particle defects and pixel structural anomalies during the production process are essential to ensure product yield and adapt to production demands arising from process fluctuations, material variations, and diverse pixel structures. This inspection process directly impacts production line efficiency and product quality traceability, making it one of the core application scenarios for industrial visual intelligent analysis.

[0003] Existing AOI systems mainly use template matching, threshold segmentation, rule engines, or a small number of supervised classifiers to determine defects. They complete the identification of anomalies in particle and pixel structures by comparing with fixed "standard templates" or filtering with a single threshold.

[0004] However, such solutions have significant drawbacks: First, they are difficult to adapt to process drift and environmental differences in production, and a single "standard template" or fixed threshold needs to be frequently manually adjusted, resulting in insufficient stability; Second, false defects are mixed with real defects, leading to a high false alarm rate and increasing the burden of re-judgment; Third, they are highly dependent on long-tail defect samples, requiring a large amount of manual annotation to train the model, resulting in low update efficiency and high cost. Summary of the Invention

[0005] The main purpose of this application is to provide a panel defect detection method, device, equipment, storage medium and program product to solve the problems of misjudgment of pixel structure variation, poor adaptation to process fluctuations and many false defects in the existing AOI detection technology. It can achieve the technical effects of accurately distinguishing normal fluctuations from real defects, reducing false detection rate and adapting to panel detection in multiple scenarios.

[0006] To achieve the above objectives, a first aspect of this application proposes a panel defect detection method, comprising: acquiring particle image patches, pixel structure image patches, and corresponding production line metadata of the panel to be inspected; inputting the particle image patches and the pixel structure image patches into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, calculating a risk score for the high-dimensional embedding vector, wherein the risk score is determined by a deviation term, a rarity term, and a production line drift penalty term corresponding to the production line metadata; determining the defect category of the panel to be inspected based on the risk score and a preset threshold; wherein the deviation term refers to the minimum Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structural family after aggregating structurally equivalent qualified pixel sample embedding vectors into structural families through family clustering; the rarity term refers to the local density of the high-dimensional embedding vector in the feature space of surrounding qualified samples or the outlier probability corresponding to the high-dimensional embedding vector.

[0007] According to the panel defect detection method provided in this application, before inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector, the method further includes: performing preprocessing operations on the particle image patch and the pixel structure image patch in sequence, the preprocessing operations including normalization processing, background correction, rotation enhancement, structure alignment, local contrast enhancement, and masking processing of invalid regions.

[0008] According to the panel defect detection method provided in this application, after inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector, the method further includes: determining whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster; if the high-dimensional embedding vector is within the tolerance boundary of at least one golden cluster, then the panel to be inspected is determined to be a qualified panel.

[0009] According to the panel defect detection method provided in this application, before determining whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster, the method further includes: performing cluster analysis using a clustering algorithm on the qualified sample embedding vectors corresponding to particles to form at least one particle golden cluster representing the normal characteristics of particles; performing family clustering based on geometric feature similarity, rotational equivalence, and fine structure consistency on the qualified sample embedding vectors corresponding to pixel structures to aggregate structurally equivalent qualified sample embedding vectors into at least one structure family, and constructing pixel structure golden clusters representing the normal structural characteristics of each structure family using a clustering algorithm; and determining the corresponding tolerance boundary using covariance analysis or the k-nearest neighbor radius method based on the distribution characteristics of the qualified sample embedding vectors within each golden cluster.

[0010] According to the panel defect detection method provided in this application, the calculation of the risk score of the high-dimensional embedding vector includes: According to the formula Calculate the risk score of the high-dimensional embedding vector. ; in, This indicates normalization processing. Indicates the degree of deviation. Items indicating rarity This indicates the penalty for production line drift. , , This represents the weighting coefficient.

[0011] According to the panel defect detection method provided in this application, the preset threshold includes a defect threshold and an interference threshold. The step of determining the defect category of the panel to be inspected based on the risk score and the preset threshold includes: if the risk score is greater than or equal to the defect threshold, then the panel to be inspected is determined to be a defective panel; if the risk score is greater than or equal to the interference threshold and less than the defect threshold, and satisfies the preset interference defect rule, then the panel to be inspected is determined to have an interference defect; if the risk score is less than the interference threshold, then the panel to be inspected is determined to be a qualified panel.

[0012] This application also provides a panel defect detection device, comprising the following modules: an acquisition module and a processing module; the acquisition module is used to acquire particle image patches, pixel structure image patches, and corresponding production line metadata of the panel to be inspected; the processing module is used to input the particle image patches and the pixel structure image patches into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, a risk score of the high-dimensional embedding vector is calculated, wherein the risk score is determined by a deviation degree term, a rarity term, and a production line drift penalty term corresponding to the production line metadata. The defect category of the panel to be inspected is determined based on the risk score and a preset threshold. The deviation term refers to the minimum Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters. The golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structural family after aggregating structurally equivalent qualified pixel sample embedding vectors into structural families through family clustering. The rarity term refers to the local density of the high-dimensional embedding vector in the feature space of surrounding qualified samples or the outlier probability corresponding to the high-dimensional embedding vector.

[0013] According to the panel defect detection device provided in this application, before inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector, the processing module is used to perform preprocessing operations on the particle image patch and the pixel structure image patch in sequence. The preprocessing operations include normalization processing, background correction, rotation enhancement, structure alignment, local contrast enhancement, and masking processing of invalid regions.

[0014] According to the panel defect detection device provided in this application, after inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector, the processing module is used to determine whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster; if the high-dimensional embedding vector is within the tolerance boundary of at least one golden cluster, the panel to be inspected is determined to be a qualified panel.

[0015] According to the panel defect detection device provided in this application, before determining whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster, the processing module is used to perform cluster analysis using a clustering algorithm on the qualified sample embedding vectors corresponding to particles to form at least one particle golden cluster representing the normal characteristics of particles; for the qualified sample embedding vectors corresponding to pixel structures, family clustering is performed based on geometric feature similarity, rotational equivalence, and fine structure consistency to aggregate structurally equivalent qualified sample embedding vectors into at least one structural family, and a pixel structure golden cluster representing the normal structural characteristics of each structural family is constructed using a clustering algorithm; based on the distribution characteristics of qualified sample embedding vectors within each golden cluster, covariance analysis or the k-nearest neighbor radius method is used to determine the corresponding tolerance boundary.

[0016] According to the panel defect detection device provided in this application, the calculation of the risk score of the high-dimensional embedding vector includes: according to the formula Calculate the risk score of the high-dimensional embedding vector. ;in, This indicates normalization processing. Indicates the degree of deviation. Items indicating rarity This indicates the penalty for production line drift. , , This represents the weighting coefficient.

[0017] According to the panel defect detection device provided in this application, the preset threshold includes a defect threshold and an interference threshold. The step of determining the defect category of the panel to be inspected based on the risk score and the preset threshold includes: if the risk score is greater than or equal to the defect threshold, then the panel to be inspected is determined to be a defective panel; if the risk score is greater than or equal to the interference threshold and less than the defect threshold, and satisfies the preset interference defect rule, then the panel to be inspected is determined to have an interference defect; if the risk score is less than the interference threshold, then the panel to be inspected is determined to be a qualified panel.

[0018] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the panel defect detection method as described above.

[0019] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the panel defect detection method as described above.

[0020] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the panel defect detection method as described above.

[0021] The technical solutions provided by the embodiments of this application may include the following beneficial effects: Because it adopts a dual-system gold cluster construction method of particle gold cluster and pixel structure multi-gold cluster, it can adapt to normal process fluctuations such as pixel rotation and geometric differences by aggregating equivalent pixel samples through family clustering. This avoids the rigid judgment of single template comparison and solves the problem of poor adaptability of traditional solutions. Because the risk score is calculated based on deviation, rarity, and production line drift penalty, and the defect judgment is completed in combination with preset thresholds, it can accurately distinguish defects, significantly reduce the false defect rate, and reduce the burden of manual review. Since anomaly identification can be achieved through the golden cluster tolerance boundary and risk scoring mechanism, without relying on a large number of labeled long-tail defect samples, the model training cost is reduced, and the update efficiency of the detection scheme and the flexibility of production line operation and maintenance are improved. Attached Figure Description

[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A schematic flowchart of the panel defect detection method provided in this application; Figure 2 This is a schematic diagram of the panel defect detection device provided by the present invention; Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0026] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0027] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0028] This application describes some exemplary embodiments for illustrative purposes. It should be understood that this application may be implemented in other ways not specifically shown in the accompanying drawings.

[0029] like Figure 1 As shown, this application provides a panel defect detection method, which can be applied to a panel defect detection device. The panel defect detection method may include steps S101-S103: S101, The panel defect detection device acquires the particle image patch, pixel structure image patch and corresponding production line metadata of the panel to be inspected.

[0030] Specifically, in the AOI online inspection scenario of panel manufacturing, in order to accurately distinguish whether the same type of structural difference is a normal process fluctuation of the pixel itself or a real defect, it is necessary to simultaneously acquire three types of data: particle image patch, pixel structure image patch, and corresponding production line metadata. Particle image patches are local areas precisely cropped from AOI online images, used to characterize the texture and morphological features of particles and foreign objects. For example, local image fragments corresponding to foreign objects such as tiny dust particles and metal fragments attached to the panel surface can intuitively reflect the size, shape and surface texture information of foreign objects. The pixel structure image patch is taken from the pixel unit area of ​​the panel and is used to characterize the pixel's geometry, rotation angle, symmetry characteristics and subtle structural changes, such as the local images corresponding to structural anomalies such as the slight deformation of liquid crystal pixels and the electrode arrangement deviation of OLED pixels. It can accurately capture subtle structural differences at the pixel level. Production line metadata, as key information linking defects to specific production scenarios, can cover panel number, production batch, production line identification, process formula parameters, etc., and can clearly restore the production environment and process background in which the defects occurred.

[0031] By simultaneously collecting the above three types of data, the lack of production context in single image data can be effectively compensated for, avoiding misjudgment caused by deviating from the process scenario. This provides comprehensive data support with scenario attributes for subsequent feature encoding, cluster analysis and other steps, making the defect detection process more in line with the actual production status of the production line, thereby improving the accuracy and reliability of the detection results.

[0032] S102, The panel defect detection device inputs the particle image patch and the pixel structure image patch into the shared feature encoder to generate a high-dimensional embedding vector.

[0033] Optionally, before inputting the particle image patch and the pixel structure image patch into the shared feature encoder to generate a high-dimensional embedding vector, the panel defect detection device may first perform preprocessing operations on the particle image patch and the pixel structure image patch in sequence. The preprocessing operations include normalization, background correction, rotation enhancement, structure alignment, local contrast enhancement, and masking of invalid regions.

[0034] Specifically, normalization processing employs brightness normalization, unifying the mean and variance of image brightness to eliminate interference from differences in production line lighting on image feature extraction, ensuring consistent brightness benchmarks for images from different batches and under different lighting conditions. Background correction addresses background noise and uneven brightness in images by using pixel grayscale value correction algorithms to remove background interference and highlight features of foreground targets such as particles and pixel structures. Rotation enhancement and structure alignment are key processes for adapting to differences in pixel structure rotation and deformation. By applying rotation transformations of different angles to the image and performing precise alignment based on the geometric symmetry of pixel structures, variable-structure pixel samples are incorporated into a unified feature analysis framework, rather than using a fixed template for forced comparison. Local contrast enhancement improves the feature recognition of minor defects by strengthening the grayscale differences between particle texture details and pixel structure edges, allowing subsequent encoding processes to capture defect information more accurately. Masking of invalid regions mainly targets black / low-signal areas in the image. By establishing a mask matrix to shield these areas without effective information, invalid data is avoided from interfering with feature extraction, ensuring that the encoder focuses on valuable image areas.

[0035] It should be noted that the above preprocessing operations can eliminate interference factors such as image noise, illumination differences, and structural deformation, providing high-quality input data for feature encoding.

[0036] After completing the preprocessing operation, the panel defect detection device can simultaneously input the preprocessed particle image patch and pixel structure image patch into the shared feature encoder. The shared feature encoder can convert the visual features of the two types of images into high-dimensional embedding vectors through iterative calculations of a multi-layer network. These high-dimensional embedding vectors can be used to measure the similarity between samples. The vector dimension of the high-dimensional embedding vector can be from 128 to 2048.

[0037] Optionally, to ensure the accuracy of subsequent feature space clustering and sample similarity measurement, the generated high-dimensional embedding vectors can be L2 normalized to ensure consistency in distance calculations within the high-dimensional space. For example, defect-free particle image patches and particle image patches with attached micro-metal fragments will form significantly different embedding vectors after encoding, exhibiting clear spatial separation in the high-dimensional space; similarly, the embedding vectors of normal pixel structures and pixel structures with electrode arrangement deviations will also be clearly distinguishable in the feature dimension.

[0038] Optionally, the shared feature encoder may include a convolutional neural network (CNN), a visual Transformer (ViT), or a combination of both. This allows it to simultaneously adapt to the feature extraction needs of two types of images. It can capture the local texture morphology of particles and the structural details of pixels through CNN, and also capture the global correlation features of the image with the help of ViT.

[0039] It should be noted that the shared feature encoder achieves unified feature representation of particle defects and pixel structural anomalies, breaking down the barrier between independent detection of the two types of defects. The generated high-dimensional embedding vector can accurately capture subtle defect features and structural differences, providing a highly recognizable feature foundation for subsequent steps such as golden cluster construction, risk scoring, and family clustering. At the same time, normalization processing and flexible encoder selection also improve the model's adaptability to production line process fluctuations and real-time inference efficiency.

[0040] Optionally, the panel defect detection device can also perform cluster analysis using a clustering algorithm on the qualified sample embedding vectors corresponding to particles to form at least one particle golden cluster representing the normal characteristics of particles; for the qualified sample embedding vectors corresponding to pixel structures, family clustering is performed based on geometric feature similarity, rotational equivalence, and fine structure consistency to aggregate structurally equivalent qualified sample embedding vectors into at least one structural family, and a pixel structure golden cluster representing the normal structural characteristics of each structural family is constructed using a clustering algorithm; based on the distribution characteristics of qualified sample embedding vectors within each golden cluster, covariance analysis or the k-nearest neighbor radius method is used to determine the corresponding tolerance boundary.

[0041] Specifically, for qualified particle sample embedding vectors, clustering algorithms can automatically aggregate qualified sample embedding vectors with similar features to form one or more stable particle clusters, each representing a set of normal particle features. For qualified pixel structure sample embedding vectors, the family clustering process prioritizes the similarity of geometric features of pixel structures, such as the matching degree of core geometric parameters like pixel shape and size ratio; it also considers rotational equivalence, classifying pixel samples with different rotation angles but identical structural essence as equivalence classes; and it supplements this with consistency in subtle structures to ensure that the aggregated samples are indistinguishable in minute structural details, ultimately forming multiple structural families. For qualified sample embedding vectors within each structural family, a secondary clustering algorithm is also used to construct a unique pixel structure cluster for each family, enabling the maintenance of multiple clusters under the same or multiple formulations. When determining the tolerance boundary, covariance analysis can be used, which involves calculating the covariance matrix of the embedding vectors of qualified samples within the golden cluster to quantify the dispersion of the sample distribution across each feature dimension, thereby defining the boundary range that can cover the distribution of normal samples. Alternatively, the k-nearest neighbor radius method can be used, which uses the center of the golden cluster as a reference, calculates the average distance from the center of the samples within the cluster, and combines it with a pre-set confidence level to determine a reasonable radius range as the tolerance boundary, ensuring that the embedding vectors of normal samples can all fall within the tolerance boundary, while effectively isolating abnormal samples.

[0042] Optionally, the clustering algorithm described above can be any one or more of K-Means clustering, HDBSCAN clustering, and Gaussian mixture model (GMM).

[0043] Optionally, after inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector, the panel defect detection device can determine whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster; if the high-dimensional embedding vector is within the tolerance boundary of at least one golden cluster, the panel to be inspected is determined to be a qualified panel. The golden cluster includes particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structure family after aggregating structurally equivalent qualified pixel sample embedding vectors into structure families through family clustering.

[0044] Specifically, the panel defect detection device first extracts the high-dimensional embedding vectors of particles and pixels corresponding to the panel under inspection. Then, for each particle embedding vector, it calculates the distance to the center of all particle golden clusters and verifies whether the vector falls within the tolerance boundary of any particle golden cluster. For each pixel structure embedding vector, it first matches the corresponding structure family based on its geometric features and rotation attributes, and then verifies whether the vector is within the tolerance boundary of any pixel structure golden cluster corresponding to that structure family. During the judgment process, the distance calculation of particle embedding vectors uses the same metric standard as when constructing golden clusters to ensure the consistency of the judgment logic. The family matching of pixel structure embedding vectors continues the judgment rules of geometric feature similarity, rotation equivalence, and fine structure consistency during family clustering to avoid cross-family misjudgment. Only when the particle embedding vector is within the tolerance boundary of a certain particle golden cluster, and the pixel structure embedding vector is within the tolerance boundary of a certain pixel structure golden cluster corresponding to its matched structure family, is it determined that the particle features and pixel structure features of the panel under inspection both meet the normal standards, and thus the panel under inspection is determined to be a qualified panel.

[0045] It should be noted that the above judgment method replaces the traditional single template comparison method with dual verification of gold clusters and tolerance boundaries. It can adapt to normal feature differences caused by production line process fluctuations. At the same time, through family clustering and collaborative management of multiple gold clusters, it can accurately identify normal variations and real defects in pixel structure, effectively improving the robustness and accuracy of detection.

[0046] S103. If the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, the panel defect detection device calculates the risk score of the high-dimensional embedding vector.

[0047] The risk score is determined by a deviation term, a rarity term, and a production line drift penalty term corresponding to the production line metadata. The deviation term refers to the minimum Euclidean distance from the high-dimensional embedding vector to the centers of all golden clusters; the rarity term refers to the local density of the high-dimensional embedding vector among the surrounding qualified samples in the feature space or the outlier probability corresponding to the high-dimensional embedding vector.

[0048] Specifically, judging the tolerance boundary can only initially screen out samples that "deviate from the normal distribution," but these samples may include various situations: they may be true defects (NG) affecting panel performance, pseudo-defects (Nuisance) caused by process fluctuations, or novel long-tail defects with scarce samples. Relying solely on boundary judgment would result in a uniform judgment of all deviation samples, leading to an increased false alarm rate or missed detection risk. Therefore, to avoid the above problems, panel defect detection devices can further distinguish different types of deviation samples by calculating risk scores, specifically including: (1) First, for the high-dimensional embedding vector of the particle, traverse all the golden clusters of particles and calculate the Euclidean distance from the vector to the center of each cluster, and select the minimum value as the particle deviation term; (2) For a high-dimensional embedding vector of a pixel structure, first match its corresponding structure family, then calculate the Euclidean distance from the vector to the center of the golden cluster of all pixel structures in the family, and take the minimum value as the pixel structure deviation term.

[0049] (3) The rarity term can be calculated by using k-nearest neighbor density estimation or kernel density estimation. By statistically analyzing the distribution of a preset number of qualified samples around the high-dimensional embedding vector in the feature space, the local density value can be obtained. Alternatively, based on Gaussian mixture model or HDBSCAN clustering outlier detection logic, the outlier probability corresponding to the vector can be calculated. The lower the local density or the higher the outlier probability, the larger the value of the rarity term.

[0050] (4) Production line drift penalty item combines the production batch, production line, process formula and other information in the production line metadata, and associates the production line drift index (such as the characteristic distribution offset between the current batch and the historical normal batch). If there is obvious process drift in the production line, the corresponding penalty coefficient is assigned according to the severity of the drift. The more severe the drift, the larger the penalty item value.

[0051] Optionally, the panel defect detection device calculates a risk score for the high-dimensional embedding vector, including: According to the formula Calculate the risk score of the high-dimensional embedding vector. ; in, This indicates normalization processing. Indicates the degree of deviation. Items indicating rarity This indicates the penalty for production line drift. , , This represents the weighting coefficient.

[0052] It should be noted that the three-dimensional evaluation of "deviation degree", "rarity degree" and "production line drift penalty" can not only accurately capture the abnormal characteristics of the sample itself, but also dynamically adjust the evaluation weight in combination with the production line process background, thereby effectively improving the ability to identify process fluctuations and long-tail defects, while reducing the false alarm rate caused by false defects, and providing comprehensive and production line-appropriate quantitative support for defect classification.

[0053] S104. The panel defect detection device determines the defect category of the panel to be inspected based on the risk score and the preset threshold.

[0054] Optionally, the preset threshold includes a defect threshold and an interference threshold. The panel defect detection device determines the defect category of the panel to be inspected based on the risk score and the preset threshold, including: if the risk score is greater than or equal to the defect threshold, the panel to be inspected is determined to be a defective panel; if the risk score is greater than or equal to the interference threshold and less than the defect threshold, and satisfies the preset interference defect rule, the panel to be inspected is determined to have an interference defect; if the risk score is less than the interference threshold, the panel to be inspected is determined to be a qualified panel.

[0055] Specifically, after the risk score is calculated, the panel defect detection device can compare the risk score with the preset threshold. The defect threshold and interference threshold are both from the strategy version library. They are generated by a large language model combined with cluster statistics, drift indicators and engineering rule text, and are determined after verification. They also support adaptive updates as the production line process changes.

[0056] If the risk score of the sample to be inspected is greater than or equal to the defect threshold, it means that the sample’s deviation, rarity and the comprehensive indicators affected by production line drift have exceeded the normal acceptable range. The corresponding defects will affect the normal performance of the panel. Therefore, the panel to be inspected is identified as a defective panel. Optionally, after determining that the panel to be inspected is a defective panel, the severity level and confidence level corresponding to the defect can also be output. The severity level can be determined based on the extent to which the risk score exceeds the defect threshold and the quality requirements associated with the production line metadata. The confidence level can be calculated based on the outlier degree of the samples in the feature space.

[0057] If the risk score is in the range of greater than or equal to the interference threshold and less than the defect threshold, and meets the preset interference defect rules, then the panel under inspection is determined to have interference defects. These defects will not affect the core performance of the panel and are only pseudo-defects caused by process fluctuations or environmental interference. They do not need to be included in the defect statistics, but relevant data will be recorded to optimize the tolerance boundary and threshold strategy.

[0058] Optionally, the preset interference defect rules can be such as the sample having a high local density in the feature space or belonging to a common deviation type in production line process fluctuations.

[0059] If the risk score is less than the interference threshold, it indicates that although the sample is not within the tolerance boundary of the golden cluster, the overall degree of abnormality is extremely low and belongs to the acceptable normal fluctuation. Therefore, the panel to be tested is determined to be a qualified panel.

[0060] It should be noted that by using dual threshold division and interference rule verification, the system can accurately distinguish between real defects, interference defects, and qualified samples, effectively suppressing false alarms caused by pseudo-defects and reducing the burden of re-judgment on the production line.

[0061] In this embodiment, the dual-system gold cluster construction method of particle gold clusters and pixel structure multi-gold clusters is adopted. By aggregating equivalent pixel samples through family clustering, it can adapt to normal process fluctuations such as pixel rotation and geometric differences, avoiding rigid judgment of single template comparison and solving the problem of poor adaptability of traditional solutions. Since the risk score is calculated based on the deviation degree, rarity, and production line drift penalty, and the defect judgment is completed in combination with the preset threshold, the defects can be accurately distinguished, the false defect false alarm rate is greatly reduced, and the burden of manual review is reduced. Since the anomaly identification can be achieved through the gold cluster tolerance boundary and risk scoring mechanism, there is no need to rely on a large number of labeled long-tail defect samples, thus reducing the model training cost and improving the update efficiency of the detection scheme and the flexibility of production line operation and maintenance.

[0062] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0063] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0064] The panel defect detection method provided in this application can be executed by a panel defect detection device or a control module for panel defect detection within that device. This application uses the panel defect detection device executing the panel defect detection method as an example to illustrate the panel defect detection device provided in this application.

[0065] It should be noted that the embodiments of this application can divide the panel defect detection device into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. Optionally, the module division in the embodiments of this application is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0066] like Figure 2 As shown in the figure, this application embodiment provides a panel defect detection device 200. The panel defect detection device 200 includes: an acquisition module 201 and a processing module 202.

[0067] The acquisition module 201 is used to acquire the particle image patch, pixel structure image patch and corresponding production line metadata of the panel to be inspected. The processing module 202 is used to input the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, then a risk score of the high-dimensional embedding vector is calculated, the risk score being determined by a deviation degree term, a rarity term, and a production line drift penalty term corresponding to the production line metadata; and the defect category of the panel to be inspected is determined based on the risk score and a preset threshold. Wherein, the deviation term refers to the minimum value of the Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structure family after aggregating qualified pixel sample embedding vectors with equivalent structure through family clustering; the rarity term refers to the local density of the high-dimensional embedding vector in the feature space of the surrounding qualified samples or the outlier probability corresponding to the high-dimensional embedding vector.

[0068] Optionally, before inputting the particle image patch and the pixel structure image patch into the shared feature encoder to generate a high-dimensional embedding vector, the processing module 202 is used to perform preprocessing operations on the particle image patch and the pixel structure image patch in sequence. The preprocessing operations include normalization, background correction, rotation enhancement, structure alignment, local contrast enhancement, and masking of invalid regions.

[0069] Optionally, after inputting the particle image patch and the pixel structure image patch into the shared feature encoder to generate a high-dimensional embedding vector, the processing module 202 is used to determine whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster; if the high-dimensional embedding vector is within the tolerance boundary of at least one golden cluster, then the panel to be inspected is determined to be a qualified panel.

[0070] Optionally, before determining whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster, the processing module 202 is used to perform cluster analysis using a clustering algorithm on the qualified sample embedding vectors corresponding to the particles to form at least one particle golden cluster representing the normal characteristics of the particles; for the qualified sample embedding vectors corresponding to the pixel structures, family clustering is performed based on geometric feature similarity, rotational equivalence, and fine structure consistency to aggregate structurally equivalent qualified sample embedding vectors into at least one structural family, and a pixel structure golden cluster representing the normal structural characteristics of the family is constructed using a clustering algorithm for each structural family; based on the distribution characteristics of the qualified sample embedding vectors within each golden cluster, covariance analysis or the k-nearest neighbor radius method is used to determine the corresponding tolerance boundary.

[0071] Optionally, calculating the risk score of the high-dimensional embedding vector includes: according to the formula Calculate the risk score of the high-dimensional embedding vector. ;in, This indicates normalization processing. Indicates the degree of deviation. Items indicating rarity This indicates the penalty for production line drift. , , This represents the weighting coefficient.

[0072] Optionally, the preset threshold includes a defect threshold and an interference threshold. Determining the defect category of the panel to be inspected based on the risk score and the preset threshold includes: if the risk score is greater than or equal to the defect threshold, then the panel to be inspected is determined to be a defective panel; if the risk score is greater than or equal to the interference threshold and less than the defect threshold, and satisfies a preset interference defect rule, then the panel to be inspected is determined to have an interference defect; if the risk score is less than the interference threshold, then the panel to be inspected is determined to be a qualified panel.

[0073] In this embodiment, the dual-system gold cluster construction method of particle gold clusters and pixel structure multi-gold clusters is adopted. By aggregating equivalent pixel samples through family clustering, it can adapt to normal process fluctuations such as pixel rotation and geometric differences, avoiding rigid judgment of single template comparison and solving the problem of poor adaptability of traditional solutions. Since the risk score is calculated based on the deviation degree, rarity, and production line drift penalty, and the defect judgment is completed in combination with the preset threshold, the defects can be accurately distinguished, the false defect false alarm rate is greatly reduced, and the burden of manual review is reduced. Since the anomaly identification can be achieved through the gold cluster tolerance boundary and risk scoring mechanism, there is no need to rely on a large number of labeled long-tail defect samples, thus reducing the model training cost and improving the update efficiency of the detection scheme and the flexibility of production line operation and maintenance.

[0074] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communications bus 340. The processor 310 can call logic instructions in the memory 330 to execute a panel defect detection method. This method includes: acquiring particle image patches, pixel structure image patches, and corresponding production line metadata of the panel to be inspected; inputting the particle image patches and pixel structure image patches into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, calculating a risk score for the high-dimensional embedding vector, the risk score being determined by a deviation term, a rarity term, and a production line drift penalty term corresponding to the production line metadata; determining the defect category of the panel to be inspected based on the risk score and a preset threshold; wherein, the deviation term refers to the minimum Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structural family after aggregating structurally equivalent qualified pixel sample embedding vectors into structural families through family clustering; the rarity term refers to the local density of the high-dimensional embedding vector in the feature space of surrounding qualified samples or the outlier probability corresponding to the high-dimensional embedding vector.

[0075] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the panel defect detection method provided by the above methods. The method includes: acquiring particle image patches, pixel structure image patches, and corresponding production line metadata of the panel to be inspected; inputting the particle image patches and the pixel structure image patches into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, calculating a risk score for the high-dimensional embedding vector, wherein the risk score consists of a deviation degree term, The rarity term and the production line drift penalty term corresponding to the production line metadata are determined; the defect category of the panel to be inspected is determined based on the risk score and a preset threshold; wherein, the deviation term refers to the minimum value of the Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structure family after aggregating the structurally equivalent qualified pixel sample embedding vectors into structure families through family clustering; the rarity term refers to the local density of the high-dimensional embedding vector in the feature space of the surrounding qualified samples or the outlier probability corresponding to the high-dimensional embedding vector.

[0077] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the panel defect detection method provided by the methods described above. This method includes: acquiring particle image patches, pixel structure image patches, and corresponding production line metadata of the panel to be inspected; inputting the particle image patches and the pixel structure image patches into a shared feature encoder to generate a high-dimensional embedding vector; if the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, calculating a risk score for the high-dimensional embedding vector, the risk score consisting of a deviation term, a rarity term, and a relation to the production line metadata. The production line drift penalty term corresponding to the data is determined; the defect category of the panel to be inspected is determined based on the risk score and the preset threshold; wherein, the deviation term refers to the minimum value of the Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structure family after aggregating the structurally equivalent qualified pixel sample embedding vectors into structure families through family clustering; the rarity term refers to the local density of the high-dimensional embedding vector in the feature space of the surrounding qualified samples or the outlier probability corresponding to the high-dimensional embedding vector.

[0078] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0079] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0080] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting panel defects, characterized in that, include: Obtain the particle image patch, pixel structure image patch, and corresponding production line metadata of the panel to be inspected; The particle image patch and the pixel structure image patch are input into a shared feature encoder to generate a high-dimensional embedding vector. If the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, then the risk score of the high-dimensional embedding vector is calculated. The risk score is determined by the deviation degree term, the rarity term, and the production line drift penalty term corresponding to the production line metadata. The defect category of the panel to be inspected is determined based on the risk score and the preset threshold. Wherein, the deviation term refers to the minimum value of the Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structure family after aggregating qualified pixel sample embedding vectors with equivalent structure through family clustering; the rarity term refers to the local density of qualified samples surrounding the high-dimensional embedding vector in the feature space or the outlier probability corresponding to the high-dimensional embedding vector.

2. The panel defect detection method according to claim 1, characterized in that, Before inputting the particle image patch and the pixel structure image patch into the shared feature encoder to generate a high-dimensional embedding vector, the method further includes: The particle image patch and the pixel structure image patch are preprocessed sequentially. The preprocessing operations include normalization, background correction, rotation enhancement, structure alignment, local contrast enhancement, and masking of invalid regions.

3. The panel defect detection method according to claim 1, characterized in that, After inputting the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector, the method further includes: Determine whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster; If the high-dimensional embedding vector is within the tolerance boundary of at least one golden cluster, then the panel to be inspected is determined to be a qualified panel.

4. The panel defect detection method according to claim 3, characterized in that, Before determining whether the high-dimensional embedding vector is within the tolerance boundary of any golden cluster, the method further includes: For the qualified sample embedding vectors corresponding to the particles, a clustering algorithm is used to perform cluster analysis to form at least one particle golden cluster that represents the normal characteristics of the particles. For qualified sample embedding vectors corresponding to pixel structures, family clustering is performed based on geometric feature similarity, rotation equivalence and fine structure consistency. Qualified sample embedding vectors with structural equivalence are aggregated into at least one structural family. For each structural family, a clustering algorithm is used to construct a pixel structure golden cluster that represents the normal structural features of the family. Based on the distribution characteristics of the embedding vectors of qualified samples within each gold cluster, the corresponding tolerance boundaries are determined using covariance analysis or the k-nearest neighbor radius method.

5. The panel defect detection method according to claim 1, characterized in that, The calculation of the risk score for the high-dimensional embedding vector includes: According to the formula Calculate the risk score of the high-dimensional embedding vector. ; in, This indicates normalization processing. Indicates the degree of deviation. Items indicating rarity This indicates the penalty for production line drift. , , This represents the weighting coefficient.

6. The panel defect detection method according to claim 1, characterized in that, The preset threshold includes a defect threshold and an interference threshold. Determining the defect category of the panel to be inspected based on the risk score and the preset threshold includes: If the risk score is greater than or equal to the defect threshold, then the panel to be inspected is determined to be a defect panel. If the risk score is greater than or equal to the interference threshold and less than the defect threshold, and meets the preset interference defect rules, then it is determined that the panel to be inspected has an interference defect. If the risk score is less than the interference threshold, the panel to be inspected is determined to be a qualified panel.

7. A panel defect detection device, characterized in that, include: Acquisition module and processing module; The acquisition module is used to acquire the particle image patch, pixel structure image patch and corresponding production line metadata of the panel to be inspected. The processing module is used to input the particle image patch and the pixel structure image patch into a shared feature encoder to generate a high-dimensional embedding vector; If the high-dimensional embedding vector is not within the tolerance boundary of any golden cluster, a risk score for the high-dimensional embedding vector is calculated. The risk score is determined by a deviation term, a rarity term, and a production line drift penalty term corresponding to the production line metadata. The defect category of the panel to be inspected is determined based on the risk score and a preset threshold. Wherein, the deviation term refers to the minimum value of the Euclidean distance from the high-dimensional embedding vector to the center of all golden clusters; the golden clusters include particle golden clusters formed by clustering qualified sample embedding vectors without particle defects, and pixel structure golden clusters constructed for each structure family after aggregating qualified pixel sample embedding vectors with equivalent structure through family clustering; the rarity term refers to the local density of qualified samples surrounding the high-dimensional embedding vector in the feature space or the outlier probability corresponding to the high-dimensional embedding vector.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the panel defect detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the panel defect detection method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the panel defect detection method as described in any one of claims 1 to 6.

Images