A substrate defect detection method and apparatus involving unsupervised learning

Abstract

The application discloses a substrate defect detection method and device related to unsupervised learning, and the method comprises the following steps: performing sliding segmentation on a substrate image with a preset size to obtain a plurality of sub-regions; adjacent two sub-regions do not overlap; a pixel point graph of a first sub-region is constructed, the pixel point graph is obtained after a preset unsupervised learning model performs defect detection on the first sub-region; the first sub-region is any one of the plurality of sub-regions, and the pixel point graph comprises a confidence value of any one pixel point being a substrate defect and a position coordinate of the pixel point in the first sub-region; a defect pixel point graph corresponding to the first sub-region is constructed to complete substrate defect detection; and the defect pixel point graph is constructed from pixel points with a confidence value greater than or equal to a preset confidence value in the pixel point graph. The application can perform substrate defect detection without using defect samples in the case that the number of substrate defect samples is small.

Description

Technical Field

[0001] This application relates to the field of defect detection technology for displays during inkjet printing, specifically to a substrate defect detection method and device involving unsupervised learning. Background Technology

[0002] Currently, in the field of inkjet printing of substrates, substrates need to undergo optical vision inspection in inkjet printing scenarios (including before and after inkjet printing) to identify defects. Generally, machine learning models are used to detect defects in substrate images, but a large number of defect samples need to be collected for machine learning.

[0003] However, collecting defect samples is difficult, resulting in a small number of defect samples on the substrate. With a small number of defect samples, there are insufficient negative samples for machine learning, resulting in a low recognition rate for defect identification through machine learning and significant missed defects.

[0004] Therefore, to address the issue of how to detect substrate defects when the number of defect samples is small, a substrate defect detection method and device involving unsupervised learning are needed. Summary of the Invention

[0005] This application provides a substrate defect detection method and apparatus involving unsupervised learning, which can perform substrate defect detection without using defect samples when the number of substrate defect samples is small.

[0006] The first aspect of this application discloses a substrate defect detection method involving unsupervised learning. The substrate defect detection method includes: performing sliding segmentation of a substrate image by a preset size to obtain multiple sub-regions; ensuring that adjacent sub-regions do not overlap; constructing a pixel map of a first sub-region, wherein the pixel map is obtained by performing defect detection on the first sub-region using a preset unsupervised learning model; the first sub-region is any one of the multiple sub-regions, and the pixel map includes a confidence value of any pixel as a substrate defect and the position coordinates of the pixel in the first region; constructing a defect pixel map corresponding to the first sub-region to complete the substrate defect detection; the defect pixel map is constructed from pixels in the pixel map whose confidence value is greater than or equal to a preset confidence value.

[0007] In the above scheme, defect detection of sub-regions is performed using an unsupervised learning model, enabling defect detection of the substrate even in the presence of completely defect-free samples. Unsupervised learning models include EfficientAD, Glass, and PatchCore models, which will not be listed individually. The preset size for sliding segmentation is determined by the input size of the unsupervised learning model and is not restricted. Furthermore, when performing sliding segmentation on the substrate image, since the unsupervised learning model outputs the probability value (confidence value) of whether a pixel is a defect, there is no need to set an overlapping region.

[0008] In one possible implementation, after constructing the defect pixel map corresponding to the first sub-region, the substrate defect detection method further includes: merging multiple defect pixel maps corresponding to multiple sub-regions into a defect substrate image; extracting the region of interest in the defect substrate image, wherein the region of interest is the defect region to be detected in the defect substrate image; the defect region to be detected includes the region after merging two adjacent sub-regions and the region after merging four adjacent sub-regions; if the average gray value of the defect region to be detected is less than a gray value threshold and the area of ​​the defect region is less than an area threshold, then it is determined that the defect region to be detected is not a defect region.

[0009] In the above scheme, multiple sub-regions are merged into a defective substrate image (retaining only defective pixels, filtering out most normal background pixels (i.e., non-defective pixels)). The region of interest (ROI) is then extracted and further filtered based on its grayscale and area thresholds. By setting the grayscale and area thresholds for the ROI (i.e., the defective area to be detected), some noise (such as light noise in the camera-captured substrate image, focusing issues in the camera-captured substrate image, etc.) can be filtered out; this noise typically has low grayscale values ​​and small areas. This specification does not impose specific values ​​on the grayscale and area thresholds for the ROI; they can be set according to actual needs.

[0010] In one possible implementation, the substrate defect detection method further includes: if the average gray value of the defect area to be detected is greater than or equal to a gray value threshold, or the area of ​​the defect area to be detected is greater than or equal to an area threshold, then the defect area to be detected is determined to be a defect area.

[0011] The above scheme discloses the conditions under which the area to be detected is a defective area. The area to be detected can be determined as a defective area if either of two conditions is met. One condition is that the average grayscale value of the area to be detected is greater than or equal to a grayscale threshold; in this case, the area of ​​the area to be detected is not considered. The other condition is that the area of ​​the area to be detected is greater than or equal to an area threshold; in this case, the average grayscale value of the area to be detected is not considered.

[0012] In one possible implementation, before extracting the region of interest (ROI) from the defective substrate image, where the ROI is the defective region to be detected in the defective substrate image, the substrate defect detection method further includes: converting the defect confidence of pixels in the defective substrate image into grayscale values ​​of pixels using a preset method; the preset method is: Gray = S((x-μ) / σ)*255; where Gray is the grayscale value of the pixel, x is the confidence value of the pixel being a defect, μ is the average value of the confidence values ​​of pixels in the defective region to be detected, σ is the standard deviation of the confidence values ​​of pixels in the defective region to be detected, and S is the sigmoid function.

[0013] The above solution discloses a method to convert the confidence value of pixels in a defective substrate image into the gray value of the pixels. This not only facilitates subsequent image processing of the defective substrate image, but also widens the differences between pixels after conversion to gray values ​​(the probability of most pixels is concentrated in a small range, and the gray value difference between pixels can be amplified after conversion to gray values), which facilitates subsequent defect screening.

[0014] In one possible implementation, merging multiple defect pixel maps corresponding to multiple sub-regions into a defective substrate image specifically includes: obtaining the numbers of a first sub-region and a second sub-region; the numbers include row numbers and column numbers, the second sub-region is any sub-region other than the first sub-region among the multiple sub-regions, and the number is pre-configured for the sub-region when sliding the substrate image; and merging the defect pixel maps corresponding to the first sub-region and the second sub-region respectively into the defective substrate image according to the numbers of the first sub-region and the second sub-region.

[0015] In the above scheme, the merging of sub-regions can be achieved by assigning row and column numbers to each sub-region when sliding and segmenting the substrate image, and then merging them using these numbers. This method is highly efficient. When locating defects subsequently, the position coordinates of the defect within the sub-region can be determined first using the pixel at the top left corner of the sub-region as the origin. Then, after merging into a defective substrate image, a mapping relationship can be established between the pixel at the top left corner of the sub-region and the pixel at the top left corner of the defective substrate image to determine the position coordinates of the defect in the defective substrate image.

[0016] In one possible implementation, multiple defect pixel maps corresponding to multiple sub-regions are merged into a defective substrate image. Specifically, this includes: obtaining the coordinates of a first sub-region and a second sub-region, the coordinates including the coordinates of the upper left corner and the lower right corner of the sub-region, the coordinates being determined when the substrate image is slide-segmented; and merging the defect pixel maps corresponding to the first sub-region and the second sub-region respectively into the defective substrate image based on the coordinates of the first sub-region and the second sub-region.

[0017] The above scheme discloses another method for merging sub-regions. By determining the coordinates of sub-regions in the substrate image, merging can be performed based on the coordinates of each sub-region. Subsequently, during defect localization, the defect coordinates of the sub-regions become the defect coordinates of the defective substrate image, facilitating defect localization.

[0018] In one possible implementation, the substrate defect detection method further includes: inputting the defect region into multiple binary classification models, including a point Mura defect binary classification model, a line Mura defect binary classification model, and a particle defect binary classification model; if the confidence values ​​output by the multiple binary classification models are all less than their respective binary classification model confidence thresholds, then the defect region is confirmed as a false alarm defect, so as not to report the defect region as a defect.

[0019] In the above scheme, multiple binary classification models can further classify and verify defective regions. Each binary classification model includes positive and negative samples, and each model has a corresponding confidence threshold. If all defective regions pass verification, then each defective region corresponds to a defect type (i.e., defect classification is complete). If a defective region is a false positive, then that false positive defect fails verification and is not classified. Both situations can be confirmed by setting the confidence threshold corresponding to the binary classification model.

[0020] In one possible implementation, the substrate defect detection method further includes: if multiple binary classification models have output confidence values ​​greater than or equal to the corresponding binary classification model confidence threshold, then the defect type of the defect area is confirmed so as to report the defect type of the defect area.

[0021] In the above scheme, if one of the multiple binary classification models outputs a confidence value for the defect area that is greater than or equal to the confidence threshold of the corresponding binary classification model, then the defect area can be determined to have passed the verification, and the defect type can be determined.

[0022] In this specification, an unsupervised learning model is used to detect defect pixels in the substrate image, that is, to identify and locate defects. By setting a preset confidence value, missed detections are avoided and some over-detections are allowed. Finally, the defect area can be verified and classified by setting a confidence threshold for a binary classification model (supervised).

[0023] In one possible implementation, extracting the region of interest (ROI) from the defective substrate image specifically includes: performing a binarization operation on the defective substrate image according to a preset ROI grayscale threshold; and obtaining the ROI from the defective substrate image through morphological processing and connected component analysis.

[0024] In the above scheme, the region of interest is extracted by setting a preset grayscale threshold. Morphological processing and connected component analysis are common methods, which will not be described here. The setting of the preset grayscale threshold is not limited and can be set according to actual needs; and the preset grayscale threshold can be used to further filter out non-defect pixels in the defect substrate image (further filtering on the positive samples (i.e., background pixels) filtered out by the pixel image through a preset confidence value).

[0025] The second aspect of this application discloses a substrate defect detection device, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and the user interface and network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the substrate defect detection device performs a substrate defect detection method involving unsupervised learning as described in any of the above.

[0026] The beneficial effects of this application include: By using an unsupervised learning model to perform defect detection on sub-regions, defect detection on the substrate can be achieved even in the case of completely defect-free samples. At this time, most pixels of normal background can be filtered out. The substrate image is segmented in a sliding manner without overlap, which, together with the unsupervised learning model, improves detection efficiency. Extract the region of interest (ROI) and then filter it again based on its grayscale and area thresholds. By setting the grayscale and area thresholds for the ROI, some noise can be further filtered out. Sub-regions can be merged using either their number or their coordinates, whichever you prefer. By setting the confidence threshold for the binary classification model, multiple binary classification models can be used to further classify and verify the defective area; When extracting the region of interest, non-defect pixels (i.e. normal background pixels) in the defect substrate image are further filtered out by setting a grayscale threshold of interest. The detection of defect pixels in the substrate image is accomplished by using an unsupervised learning model, which means the identification and localization of defects are completed. By setting a preset confidence value, missed detections are avoided and some over-detections are allowed. Finally, the verification and classification of defect areas can be completed by setting a confidence threshold for a binary classification model (supervised). Attached Figure Description

[0027] Figure 1 This application discloses a schematic flowchart of a substrate defect detection method involving unsupervised learning; Figure 2 This application specification discloses a schematic flowchart of another substrate defect detection method involving unsupervised learning; Figure 3This is a schematic diagram illustrating the relationship between multiple thresholds disclosed in this application. Figure 4 This is a schematic diagram of the substrate sliding division and merging structure disclosed in this application specification; Figure 5 This is a flowchart illustrating the two-category classification of defect areas disclosed in this application. Figure 6 This is a schematic diagram of the structure of a substrate defect detection device disclosed in this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0029] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0030] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0031] This specification discloses a substrate defect detection method involving unsupervised learning, such as... Figure 1 As shown. The substrate defect detection method includes steps S100-S300.

[0032] Step S100: Perform sliding segmentation of the substrate image at a preset size to obtain multiple sub-regions; adjacent sub-regions do not overlap.

[0033] Step S200: Construct a pixel map of the first sub-region. The pixel map is obtained by performing defect detection on the first sub-region using a preset unsupervised learning model. The first sub-region is any one of the multiple sub-regions. The pixel map includes the confidence value of any pixel as a substrate defect and the position coordinates of the pixel in the first region.

[0034] Step S300: Construct a defect pixel map corresponding to the first sub-region to complete the substrate defect detection; the defect pixel map is constructed from pixels in the pixel map whose confidence value is greater than or equal to a preset confidence value.

[0035] At this point, by using an unsupervised learning model to detect defects in sub-regions, defect detection of the substrate can be achieved even in the absence of defective samples. Unsupervised learning models include the EfficientAD model, the Glass model, and the PatchCore model, which will not be listed individually. The preset size for performing sliding segmentation is determined by the input size of the unsupervised learning model and is not restricted; furthermore, when performing sliding segmentation on the substrate image, since the unsupervised learning model outputs the probability value (confidence value) of whether a pixel is a defect, there is no need to set an overlapping region.

[0036] Specifically, sliding segmentation does not set overlapping areas, which allows each pixel to belong to only one sub-region. Furthermore, if multiple sub-regions are merged in subsequent operations, no fusion strategy is required (if sliding segmentation overlap areas are set, the defects in the overlapping areas need to be fused to determine whether the defects are the same and of the same type), reducing redundant calculations and improving detection efficiency.

[0037] In the above process, the unsupervised learning model detects each sub-region, obtaining a pixel map corresponding to each sub-region. Furthermore, the confidence value of each pixel in the pixel map is compared with a preset confidence value, and non-defective pixels are filtered out to obtain a defective pixel map. The preset confidence value is set as an empirical value and can be set according to actual needs; in the pixel-level filtering, a larger number of pixels can be filtered out, or a smaller number can be filtered out, without restriction.

[0038] Both the pixel map and the defect pixel map include the position coordinates of each pixel within its corresponding sub-region. These position coordinates facilitate the determination of the pixel's location. For example, the pixel located at the top left corner of the sub-region can be used as a reference point to construct a coordinate system, thereby obtaining the position coordinates of other pixels within the entire sub-region.

[0039] In addition, it should be noted that when sliding to segment the substrate image, if there are missing parts in the segmented sub-regions, they are filled in with the background image.

[0040] In one example, after constructing the defect pixel map corresponding to the first sub-region, as follows: Figure 2 As shown. The substrate defect detection method further includes steps S301-S303.

[0041] Step S301: Merge multiple defect pixel images corresponding to multiple sub-regions into a defect substrate image.

[0042] Step S302: Extract the region of interest from the defective substrate image. The region of interest is the defective region to be detected in the defective substrate image. The defective region to be detected includes the region after merging two adjacent sub-regions and the region after merging four adjacent sub-regions.

[0043] Step S303: If the average gray value of the defect area to be detected is less than the gray value threshold and the area of ​​the defect area is less than the area threshold, then the defect area to be detected is determined to be a defect area.

[0044] At this point, by merging multiple sub-regions into a defective substrate image (retaining only defective pixels while filtering out most normal background pixels (i.e., non-defective pixels)), the region of interest (ROI) is extracted, and further filtering is performed based on the ROI's grayscale and area thresholds. By setting the grayscale and area thresholds for the ROI (i.e., the defective area to be detected), some noise (such as light noise in the camera-captured substrate image, focusing issues in the camera-captured substrate image, etc.) can be filtered out; this noise typically has low grayscale values ​​and small areas. This manual does not impose specific values ​​on the grayscale and area thresholds for the ROI; they can be set according to actual needs.

[0045] The above scheme addresses two special cases after sub-region merging: when the region of interest (ROI) lies between two adjacent sub-regions in a sliding partition (without overlap) in the same row or column, and part of the ROI is in one sub-region while the other part is in an adjacent sub-region. Similarly, there are also cases where the ROI lies between four adjacent sub-regions.

[0046] It should be noted that before inkjet printing, the substrate background image is a clean, smooth plate; after inkjet printing, the substrate background image is an image of a defect-free film layer. In both cases, the substrate background image is completely black, and the grayscale value can be 0.

[0047] In one example, extracting the region of interest (ROI) from a defective substrate image specifically includes: performing a binarization operation on the defective substrate image according to a preset ROI grayscale threshold; and obtaining the ROI from the defective substrate image through morphological processing and connected component analysis.

[0048] In this example, the region of interest is extracted by setting a preset grayscale threshold. Morphological processing and connected component analysis are common methods and will not be described here. The setting of the preset grayscale threshold is not limited and can be set according to actual needs; and the preset grayscale threshold can be used to further filter out non-defect pixels in the defect substrate image (further filtering on the positive samples (i.e., background pixels) filtered out by the pixel image through a preset confidence value).

[0049] In one example, the substrate defect detection method further includes: if the average gray value of the defect area to be detected is greater than or equal to a gray value threshold, or the area of ​​the defect area to be detected is greater than or equal to an area threshold, then the defect area to be detected is determined to be a defect area.

[0050] At this point, as long as either of the two conditions is met, the area to be detected can be identified as a defect area. One condition is that the average gray value of the area to be detected is greater than or equal to the gray value threshold, in which case the area of ​​the area to be detected does not need to be considered; the other condition is that the area of ​​the area to be detected is greater than or equal to the area threshold, in which case the average gray value of the area to be detected does not need to be considered.

[0051] At this point, the average grayscale value is the average grayscale value of the pixels in the defect area to be detected, and the area can be determined based on the number of pixels in the defect area to be detected.

[0052] In one example, before extracting the region of interest (ROI) from the defective substrate image, which is the defective region to be detected in the defective substrate image, the substrate defect detection method further includes: converting the defect confidence of pixels in the defective substrate image into grayscale values ​​of pixels using a preset method; the preset method is: Gray = S((x-μ) / σ)*255; where Gray is the grayscale value of the pixel, x is the confidence value of the pixel as a defect, μ is the average value of the confidence values ​​of pixels in the defective region to be detected, σ is the standard deviation of the confidence values ​​of pixels in the defective region to be detected, and S is the sigmoid function.

[0053] This example discloses a method to convert the confidence values ​​of pixels in a defective substrate image into grayscale values ​​of pixels. This not only facilitates subsequent image processing of the defective substrate image, but also widens the differences between pixels after conversion to grayscale values ​​(the probability of most pixels is concentrated in a small range, and the grayscale differences between pixels can be amplified after conversion to grayscale values), which is convenient for subsequent defect screening.

[0054] Furthermore, by mapping any real number to the range 0-1 using the sigmoid function (excluding endpoints, which are asymptotic values), and then multiplying by 255 to obtain the grayscale value, the endpoint values ​​correspond to dark gray approaching 0 and bright white approaching 255, respectively. Moreover, μ and σ are involved in the grayscale conversion of each pixel, enhancing the contrast of each pixel. For example, a normal pixel (x≈μ) has a sigmoid(0)=0.5, and its grayscale value is approximately 128 (medium gray background); a defective pixel (x significantly greater than μ) has a sigmoid value that rapidly rises to near 1, and its grayscale value is close to 255 (bright white), making the defective area very noticeable on the grayscale image. Especially compared to methods that do not involve μ and σ in the pixel grayscale conversion, the contrast is significantly enhanced.

[0055] In one example, merging multiple defect pixel maps corresponding to multiple sub-regions into a defective substrate image specifically includes: obtaining the numbers of the first sub-region and the second sub-region; the numbers include row numbers and column numbers, the second sub-region is any sub-region other than the first sub-region among the multiple sub-regions, and the number is pre-configured for the sub-region when performing sliding segmentation on the substrate image; and merging the defect pixel maps corresponding to the first sub-region and the second sub-region respectively into the defective substrate image according to the numbers of the first sub-region and the second sub-region.

[0056] At this point, merging sub-regions can be achieved by assigning row and column numbers to each sub-region during the sliding segmentation of the substrate image, and then merging them using these numbers. This method is highly efficient. For subsequent defect localization, the defect's position coordinates within the sub-region can be determined first using the pixel at the top left corner of the sub-region as the origin. Then, after merging into a defective substrate image, a mapping relationship can be established between the pixel at the top left corner of the sub-region and the pixel at the top left corner of the defective substrate image to determine the defect's position coordinates within the defective substrate image.

[0057] In one example, merging multiple defect pixel maps corresponding to multiple sub-regions into a defective substrate image specifically includes: obtaining the coordinates of a first sub-region and a second sub-region, the coordinates including the coordinates of the upper left corner and the lower right corner of the sub-region, the coordinates being determined when performing sliding segmentation on the substrate image; and merging the defect pixel maps corresponding to the first sub-region and the second sub-region respectively into the defective substrate image based on the coordinates of the first sub-region and the second sub-region.

[0058] At this point, another method for merging sub-regions is discussed. By determining the coordinates of the sub-regions in the substrate image, merging can be performed based on these coordinates. Subsequently, when locating defects, the defect coordinates of the sub-regions become the defect coordinates of the defective substrate image, facilitating defect localization.

[0059] In addition, a defect is generally a defect frame of a certain size, and its coordinates can be composed of the coordinates of the upper left corner and the lower right corner of the defect.

[0060] like Figure 4 As shown, two methods of merging sub-regions are illustrated. The upper left corner 401 of the substrate 400 is the origin of the substrate's coordinates; the image segmentation example in the first row illustrates two sub-regions, and the image sliding segmentation example in the second row illustrates two sub-regions.

[0061] One method for merging sub-regions is as follows: when each sub-region is slide-divided without overlap, a corresponding number mn is assigned, where m is the row number and n is the column number. Figure 4 The first sub-region 410 is numbered 21, and the second sub-region 420 is numbered 22. Merging can proceed smoothly based on the numbering. If it is necessary to confirm the location coordinates of the defect later, the coordinate relationship between the defect and the origin 401 of the substrate coordinate system can be determined after merging (and only after merging), thereby obtaining the coordinates of the defect.

[0062] Another way to merge sub-regions is to determine the coordinates of each sub-region when it is being slid-divided without overlap, that is, to determine the pixel coordinates of the top left and bottom right corners of the sub-region. Figure 4 The image shows the coordinates of the top-left pixel 402 and bottom-right pixel 403 of the first sub-region 410, and the top-left pixel 404 and bottom-right pixel 405 of the second sub-region 420. The coordinate relationship between the coordinates of the first and second sub-regions and the origin 401 of the substrate coordinate system is known; that is, the coordinates of the defect in the sub-region are also the coordinates of the defect in the substrate image. The defect coordinates are determined by the coordinate relationship between the defect and the sub-region coordinates.

[0063] In one example, the substrate defect detection method further includes: inputting the defect region into multiple binary classification models, including a point Mura defect binary classification model, a line Mura defect binary classification model, and a particle defect binary classification model; if the confidence values ​​output by the multiple binary classification models are all less than their respective binary classification model confidence thresholds, then the defect region is confirmed as a false alarm defect, so as not to report the defect region as a defect.

[0064] Furthermore, multiple binary classification models can be used to further classify and validate defective regions. Each binary classification model includes positive and negative samples, and each model has a corresponding confidence threshold. If all defective regions pass validation, then each defective region corresponds to a defect type (i.e., defect classification is complete). If any defective region is a false positive, then that false positive defect fails validation and is not classified. Both of these situations can be confirmed by setting the confidence threshold corresponding to the binary classification model.

[0065] Furthermore, the types of binary classification models mentioned above are merely examples of common defect types and are not intended to limit the scope; binary classification models include ResNet, YOLO-CLS, etc., which will not be specifically described here. The binary classification model validates the defect region, which also serves as a further filter (i.e., filtering out some false positives). In practice, a small number of non-defect regions are allowed in the final defect region (i.e., over-detection), but under-detection is not permitted. The confidence threshold for the binary classification model can be set as needed.

[0066] In one example, the substrate defect detection method further includes: if multiple binary classification models have output confidence values ​​greater than or equal to the corresponding binary classification model confidence threshold, then the defect type of the defect area is confirmed so as to report the defect type of the defect area.

[0067] At this point, if one of the multiple binary classification models outputs a confidence value for the defective region that is greater than or equal to the corresponding binary classification model confidence threshold, then the defective region can be determined to have passed the verification, and the defect type can be determined.

[0068] like Figure 5 As shown, three binary classification models are illustrated: the 501 dot mura defect model, the 502 line mura defect model, and the 503 particle defect model. Particles generally refer to defects in the substrate before inkjet printing (e.g., particles or scratches caused by unclean substrate), while dot mura and line mura generally refer to defects in the substrate after inkjet printing.

[0069] When the confidence level of defect area 501a is greater than or equal to the preset point Mura defect confidence threshold, the defect type of the defect area can be determined to be point Mura, and the defect area is determined to be a real defect rather than a false alarm. At the same time, when the confidence level of defect area 501b is less than the preset point Mura defect confidence threshold, the defect type of the defect area cannot be determined, and the defect area is a false alarm (which can be confirmed by manual re-inspection later).

[0070] Similarly, when the confidence level of defect area 502a is greater than or equal to the preset line Mura defect confidence level threshold, the defect type of the defect area can be determined to be line Mura, and the defect area is determined to be a real defect rather than a false alarm. At the same time, when the confidence level of defect area 502b is less than the preset line Mura defect confidence level threshold, the defect type of the defect area cannot be determined, and the defect area is a false alarm (which can be confirmed by manual re-inspection later).

[0071] Furthermore, when the confidence level of the 503a defect area is greater than or equal to the preset particle defect confidence level threshold, the defect type of the defect area can be determined to be a particle defect, and it can be determined that the defect area is not a false alarm defect, but a real defect; at the same time, when the confidence level of the 503b defect area is less than the preset particle defect confidence level threshold, the defect type of the defect area cannot be determined, and the defect area is a false alarm defect (which can also be confirmed by manual re-inspection later).

[0072] The confidence thresholds for the three binary classification models mentioned above can be set according to their respective needs. Generally, factors to consider include allowing a certain number of defects to pass the inspection in the final true defect test.

[0073] In this specification, an unsupervised learning model is used to detect defect pixels in the substrate image, that is, to identify and locate defects. By setting a preset confidence value, missed detections are avoided and some over-detections are allowed. Finally, the defect area can be verified and classified by setting a confidence threshold for a binary classification model (supervised).

[0074] like Figure 3 As shown, steps S110-S140 illustrate the temporal relationship of various thresholds in this specification. Overall, the thresholds in steps S110-S140 show a gradually tightening trend, continuously filtering out non-defect cases to complete the identification, location, and classification of defects (S140).

[0075] The preset confidence level of pixels in the sub-region in step S110, i.e. Figure 1 In step S300, it is determined whether a pixel in the pixel image is a defective pixel. At this time, the preset confidence level is set to filter out most of the normal substrate background images.

[0076] Step S120 extracts the preset grayscale threshold of the region of interest, i.e. Figure 2 In step S302 (in Figure 1 Following step S300, during the extraction of the region of interest, a preset grayscale threshold is set to further filter out a small number of normal substrate background images. Before this, the pixels need to be converted to grayscale values.

[0077] The grayscale threshold and area threshold of the region to be detected in step 130, i.e. Figure 2 In step S303, it is determined whether the area to be detected is a defect area. At this time, the grayscale threshold and area threshold are set to filter out noise, which includes issues such as lighting or focus problems caused by the camera. For example, a defect is generally 10*10 pixels wide. If a 3*3 pixel wide area appears in the area to be detected, it will be filtered out.

[0078] It should be noted that steps S120 and S130 are further optimizations of the detection results in step S110. Steps S120 and S130 are not necessarily required and can be selected according to actual needs.

[0079] The confidence thresholds for multiple binary classification models in step S140 are used to classify and verify the defect areas obtained in step S130. S140 is optional and can be selected based on actual needs. Defect areas identified as false positives in S140 can also be manually re-inspected.

[0080] This specification also discloses a substrate defect detection device, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the substrate defect detection device performs a substrate defect detection method involving unsupervised learning as described above.

[0081] Specifically: A substrate defect detection method involving unsupervised learning, the substrate defect detection method comprising: performing sliding segmentation of a substrate image at a preset size to obtain multiple sub-regions; ensuring that adjacent sub-regions do not overlap; constructing a pixel map of a first sub-region, the pixel map being obtained by performing defect detection on the first sub-region using a preset unsupervised learning model; the first sub-region being any one of the multiple sub-regions, the pixel map including the confidence value of any pixel as a substrate defect and the position coordinates of the pixel in the first region; constructing a defect pixel map corresponding to the first sub-region to complete the substrate defect detection; the defect pixel map being constructed from pixels in the pixel map whose confidence value is greater than or equal to a preset confidence value.

[0082] In one example, after constructing the defect pixel map corresponding to the first sub-region, the substrate defect detection method further includes: merging the multiple defect pixel maps corresponding to multiple sub-regions into a defect substrate image; extracting the region of interest in the defect substrate image, wherein the region of interest is the defect region to be detected in the defect substrate image; the defect region to be detected includes the region after merging two adjacent sub-regions and the region after merging four adjacent sub-regions; if the average gray value of the defect region to be detected is less than the gray value threshold and the area of ​​the defect region is less than the area threshold, then it is determined that the defect region to be detected is not a defect region.

[0083] In one example, the substrate defect detection method further includes: if the average gray value of the defect area to be detected is greater than or equal to a gray value threshold, or the area of ​​the defect area to be detected is greater than or equal to an area threshold, then the defect area to be detected is determined to be a defect area.

[0084] In one example, before extracting the region of interest (ROI) from the defective substrate image, which is the defective region to be detected in the defective substrate image, the substrate defect detection method further includes: converting the defect confidence of pixels in the defective substrate image into grayscale values ​​of pixels using a preset method; the preset method is: Gray = S((x-μ) / σ)*255; where Gray is the grayscale value of the pixel, x is the confidence value of the pixel as a defect, μ is the average value of the confidence values ​​of pixels in the defective region to be detected, σ is the standard deviation of the confidence values ​​of pixels in the defective region to be detected, and S is the sigmoid function.

[0085] In one example, merging multiple defect pixel maps corresponding to multiple sub-regions into a defective substrate image specifically includes: obtaining the numbers of the first sub-region and the second sub-region; the numbers include row numbers and column numbers, the second sub-region is any sub-region other than the first sub-region among the multiple sub-regions, and the number is pre-configured for the sub-region when performing sliding segmentation on the substrate image; and merging the defect pixel maps corresponding to the first sub-region and the second sub-region respectively into the defective substrate image according to the numbers of the first sub-region and the second sub-region.

[0086] In one example, merging multiple defect pixel maps corresponding to multiple sub-regions into a defective substrate image specifically includes: obtaining the coordinates of a first sub-region and a second sub-region, the coordinates including the coordinates of the upper left corner and the lower right corner of the sub-region, the coordinates being determined when performing sliding segmentation on the substrate image; and merging the defect pixel maps corresponding to the first sub-region and the second sub-region respectively into the defective substrate image based on the coordinates of the first sub-region and the second sub-region.

[0087] In one example, the substrate defect detection method further includes: inputting the defect region into multiple binary classification models, including a point Mura defect binary classification model, a line Mura defect binary classification model, and a particle defect binary classification model; if the confidence values ​​output by the multiple binary classification models are all less than their respective binary classification model confidence thresholds, then the defect region is confirmed as a false alarm defect, so as not to report the defect region as a defect.

[0088] In one example, the substrate defect detection method further includes: if multiple binary classification models have output confidence values ​​greater than or equal to the corresponding binary classification model confidence threshold, then the defect type of the defect area is confirmed so as to report the defect type of the defect area.

[0089] In one example, extracting the region of interest (ROI) from a defective substrate image specifically includes: performing a binarization operation on the defective substrate image according to a preset ROI grayscale threshold; and obtaining the ROI from the defective substrate image through morphological processing and connected component analysis.

[0090] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0091] The specification also discloses a computer-readable storage medium storing instructions that, when executed, perform the method described above.

[0092] This embodiment also discloses an electronic device, which can be a substrate defect detection device, to perform the above-described method. (Refer to...) Figure 6 The electronic device may include: at least one processor 601, at least one communication bus 602, display 603, network interface 604, and at least one memory 605.

[0093] The communication bus 602 is used to enable communication between these components.

[0094] The display 603 may include a display screen and a camera.

[0095] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0096] The processor 601 may include one or more processing cores. The processor 601 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 605, and by calling data stored in the memory 605. Optionally, the processor 601 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 601 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 601 and may be implemented as a separate chip.

[0097] The memory 605 may include random access memory (RAM) or read-only memory. Optionally, the memory 605 may include a non-transitory computer-readable storage medium. The memory 605 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 605 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 605 may also be at least one storage device located remotely from the aforementioned processor 601. As shown in the figure, the memory 605, as a computer storage medium, may include an operating system, a network communication module, and application programs for a display module.

[0098] exist Figure 6In the electronic device shown, the display 603 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 601 can be used to call the application program stored in the memory 605. When executed by one or more processors 601, the electronic device performs one or more methods as described in the above embodiments.

[0099] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0100] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 605 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory 605 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0105] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A substrate defect detection method involving unsupervised learning, characterized in that, The substrate defect detection method includes: The substrate image is divided into multiple sub-regions by a preset size; adjacent sub-regions do not overlap. A pixel map of the first sub-region is constructed. The pixel map is obtained by performing defect detection on the first sub-region using a pre-defined unsupervised learning model. The first sub-region is any one of the multiple sub-regions. The pixel map includes the confidence value of any pixel as a substrate defect and the position coordinates of the pixel in the first region. A defect pixel map corresponding to the first sub-region is constructed to complete the substrate defect detection; the defect pixel map is constructed from pixels in the pixel map whose confidence value is greater than or equal to a preset confidence value.

2. The substrate defect detection method according to claim 1, characterized in that, After constructing the defect pixel map corresponding to the first sub-region, the substrate defect detection method further includes: Multiple defect pixel maps corresponding to multiple sub-regions are merged into a defect substrate image; Extract the region of interest from the defective substrate image. The region of interest is the defective region to be detected in the defective substrate image. The defective region to be detected includes the region after merging two adjacent sub-regions and the region after merging four adjacent sub-regions. If the average gray value of the defect area to be detected is less than the gray value threshold and the area of ​​the defect area is less than the area threshold, then the defect area to be detected is determined to be a defect area.

3. The substrate defect detection method according to claim 2, characterized in that, The substrate defect detection method further includes: If the average gray value of the defect area to be detected is greater than or equal to the gray value threshold, or the area of ​​the defect area to be detected is greater than or equal to the area threshold, then the defect area to be detected is determined to be a defect area.

4. The substrate defect detection method according to claim 2, characterized in that, Before extracting the region of interest (ROI) from the defective substrate image, where the ROI is the defective region to be detected in the defective substrate image, the substrate defect detection method further includes: The defect confidence level of pixels in the defective substrate image is converted into the grayscale value of the pixels using a preset method; the preset method is: Gray = S((x-μ) / σ)*255; Gray is the gray value of the pixel, x is the confidence value of the pixel as a defect, μ is the average confidence value of the pixels in the defect area to be detected, σ is the standard deviation of the confidence values ​​of the pixels in the defect area to be detected, and S is the sigmoid function.

5. The substrate defect detection method according to any one of claims 2-4, characterized in that, The defect pixel images corresponding to multiple sub-regions are merged into a defect substrate image, specifically including: Obtain the numbers of the first sub-region and the second sub-region; the numbers include row numbers and column numbers, the second sub-region is any one of the multiple sub-regions except the first sub-region, and the number is pre-configured for the sub-region when sliding segmenting the substrate image; Based on the numbering of the first and second sub-regions, the defect pixel maps corresponding to the first and second sub-regions are merged into the defect substrate image.

6. The substrate defect detection method according to any one of claims 2-4, characterized in that, The defect pixel images corresponding to multiple sub-regions are merged into a defect substrate image, specifically including: Obtain the coordinates of the first sub-region and the second sub-region. The coordinates include the coordinates of the upper left corner and the lower right corner of the sub-region. The coordinates are determined when performing sliding segmentation on the substrate image. Based on the coordinates of the first and second sub-regions, the defect pixel maps corresponding to the first and second sub-regions are merged into the defect substrate image.

7. The substrate defect detection method according to claim 3, characterized in that, The substrate defect detection method further includes: The defect region is input into multiple binary classification models, including point Mura defect binary classification model, line Mura defect binary classification model and particle defect binary classification model. If the confidence values ​​output by multiple binary classification models are all less than their respective binary classification model confidence thresholds, then the defective region is confirmed as a false defect, so that the defective region is not reported as a defect.

8. The substrate defect detection method according to claim 7, characterized in that, The substrate defect detection method further includes: If multiple binary classification models have output confidence values ​​greater than or equal to the corresponding binary classification model confidence threshold, then the defect type of the defect area is confirmed so that the defect type of the defect area can be reported.

9. The substrate defect detection method according to any one of claims 2-4, characterized in that, Extracting the region of interest from the defective substrate image, specifically including: The defective substrate image is binarized according to the preset grayscale threshold of interest. The region of interest in the defective substrate image was obtained through morphological processing and connected component analysis.

10. A substrate defect detection device, characterized in that, It includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and the user interface and network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the substrate defect detection device performs a substrate defect detection method involving unsupervised learning as described in any one of claims 1-9.

Images