Defect detection method and device for large-size substrate
By combining sliding segmentation and multi-threaded processing with multiple binary classification models, the difficult problem of micron-level defect detection on large-size substrates is solved, and efficient and accurate defect identification and detection are achieved.
Patent Information
- Application Number
- CN202510768197.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-10
- Publication Date
- 2025-09-23
AI Technical Summary
In existing defect detection technologies for large-scale substrates, micron-level defects are difficult to identify, especially during the scaling process, which leads to detection failure.
The sliding segmentation method is used to segment the substrate image, set the image overlap area, and perform defect identification through multi-threaded processing and multiple binary classification models. The defects are confirmed by combining the overlap rate and confidence value judgment, and adjacent defect frames are fused to improve the detection accuracy.
It improves the detection rate and accuracy of large-size substrate defect detection, reduces the omission of minor defects, and improves detection efficiency and user experience.
Smart Images
Figure CN120689292A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of defect detection for inkjet printing of display screens, and specifically to a defect detection method and device for large-size substrates. Background Art
[0002] Currently, inkjet printing technology is widely used in the manufacturing of displays, flexible sensors, and other fields, offering advantages such as high print resolution and minimal material waste. During the inkjet printing process, equipment anomalies can lead to defects such as ink displacement, ink overflow, and scattered dots. These defects can directly impact the user experience, making defect detection crucial.
[0003] However, existing defect detection methods often scale the camera image to a size that the target detection model can handle before performing defect identification. However, this approach only works for larger defects; for large substrates, defects are typically micron-sized; scaling can render these micron-level defects unrecognizable.
[0004] Therefore, how to perform defect detection on large-size substrates has become an urgent problem to be solved. Summary of the Invention
[0005] The present application provides a defect detection method and device for large-size substrates, which can detect defects at the micron level.
[0006] In a first aspect, the present application discloses a defect detection method for large-size substrates, the defect detection method comprising: performing sliding segmentation on a substrate image with a preset size and a preset step size to obtain a plurality of images to be detected; the preset size is the input size of a preset target detection model; there is an image overlap area between any two adjacent images to be detected, and the size of the image overlap area is determined by the defect size of the substrate image; with a preset first confidence value, the plurality of images to be inspected are input into the preset target detection model to obtain a first recognition result; the first recognition result includes the defect frame size and the defect type.
[0007] In the above scheme, the substrate image is segmented by sliding segmentation, and defect detection is performed on the segmented image to avoid the problem of small defects not being detected after directly compressing the substrate image; and overlapping areas are set by adjacent images to be detected to avoid missing small defects between two adjacent images to be detected during the sliding segmentation process. The size of the overlapping area should be able to cover the size of the defect, especially the size of the defect at the position that needs attention (such as the edge of the substrate), so as to facilitate defect identification. In addition, in order to identify all defects in the substrate image and improve the defect detection rate, a lower confidence value will be set; and through the sliding segmentation method, multiple images to be detected can be detected separately in a multi-threaded manner, which can improve the efficiency of defect detection.
[0008] In one possible implementation, a substrate image is subjected to sliding segmentation with a preset size and a preset step size to obtain a plurality of images to be detected; specifically, the method includes: constructing a coordinate system on the substrate image with the upper left corner of the substrate image as the coordinate origin; performing sliding segmentation on the substrate image along the X-axis and the Y-axis with a preset size and a preset step size to obtain a plurality of images to be detected and the positioning coordinates of the images to be detected; the positioning coordinates include the coordinates of the upper left corner and the lower right corner of the image to be detected.
[0009] In the above scheme, it is intended to illustrate that any image to be inspected has positioning coordinates; the coordinates of the defects in the image to be inspected are based on the positioning coordinates of the image to be inspected; this facilitates the subsequent reporting of the defect positioning coordinates to the defect detection platform.
[0010] In a possible embodiment, after obtaining the recognition result, the defect detection method further includes: obtaining a first defect area and a second defect area in any one of the image overlapping areas; wherein the image overlapping area is the overlapping area between the first image to be detected and the second image to be detected, the first defect area is the pixel area of the first defect in the first image to be detected located in the image overlapping area, and the second defect area is the pixel area of the second defect in the second image to be detected located in the image overlapping area; the first defect area and the second defect area are defects of the same type; obtaining the overlap rate of the first defect area and the second defect area; if the overlap rate is greater than or equal to a preset overlap rate threshold, confirming that the first defect and the second defect are the same defect, and removing the first defect or the second defect.
[0011] In the above scheme, it is disclosed how to deal with defects in the overlapping area of the image; the overlap rate is used to determine whether it is the same defect, and the overlap rate can be obtained by calculating the intersection ratio of the two defect areas (the ratio between the intersection of the two defect areas and the union of the two defect areas); if it is the same defect, it only needs to be reported to the defect detection platform once; if it is two defects, both are reported to the defect detection platform.
[0012] In a possible implementation, the defect detection method further includes: if the overlap rate is less than a preset overlap rate threshold, confirming that the first defect and the second defect are two defects.
[0013] In one possible embodiment, after obtaining the first recognition result, the defect detection method further includes: inputting the recognition result into multiple binary classification models in sequence with a preset second confidence value; the preset second confidence value is greater than the preset first confidence value; removing the third defect in the first recognition result to obtain a second recognition result; the confidence value of the third defect is less than the preset second confidence value.
[0014] The above solution aims to improve defect detection accuracy. Multiple binary classification models are trained on commonly used defects. These models can identify defect types. Defects are then fed into these models sequentially. If a defect does not fall into any of the defect types, it may be misidentified by the target detection model. Therefore, the second confidence value is set high, significantly greater than the first confidence value of the target detection model.
[0015] In a possible implementation, the plurality of binary classification models include binary classification models corresponding to point defects, stripe mura, circular mura, stains, scratches, and linear mura.
[0016] In the above scheme, several common defect types in substrate images are listed, but there is no limitation to them.
[0017] In a possible embodiment, after obtaining the second recognition result, the defect detection method further includes: obtaining a first defect frame and a second defect frame, the first defect frame and the second defect frame being two adjacent defect frames of the same defect type; extracting a first sub-defect frame and a second sub-defect frame to form a third defect frame; wherein the first sub-defect frame is located in the first defect frame, the second sub-defect frame is located in the second defect frame, and the first sub-defect frame is adjacent to the second sub-defect frame; inputting the third defect frame into the preset target detection model to obtain a third recognition result; if there is a defect in the third defect frame in the third recognition result, merging the first defect frame and the second defect frame into one defect frame, and sending it to the defect detection platform as a defect result.
[0018] In the above solution, adjacent defect frames of the same defect type are fused. After fusion, they can be reported to the defect detection platform as a single defect, improving both the accuracy of defect detection and the user experience. By extracting and constructing new defect frames and performing defect detection again, the accuracy of fusion can be improved.
[0019] In a possible implementation, the defect detection method further includes: if there are two defects in the third defect frame in the third identification mechanism, treating the first defect frame and the second defect frame as two defect frames and sending them as defect results to a defect detection platform.
[0020] At this time, the reported coordinates of any defect frame are the coordinates of the upper left corner and the lower right corner of the defect frame. In this specification, the target detection model for the defect detection structure is presented as a rectangular or square defect frame.
[0021] In a possible embodiment, the two adjacent defect frames specifically include any one of the following: the coordinates of the lower right corner of the first defect frame and the coordinates of the upper left corner of the second defect frame are within a preset coordinate range; the coordinates of the upper right corner of the first defect frame and the coordinates of the lower left corner of the second defect frame are within a preset coordinate range.
[0022] In the above scheme, two adjacent defect frames are described as having adjacent coordinates.
[0023] In a second aspect, the present application discloses a defect detection device for large-size substrates. The defect detection device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory, so that the defect detection device performs the following instructions:
[0024] Sliding segmentation is performed on the substrate image with a preset size and a preset step size to obtain multiple images to be detected; the preset size is the input size of the preset target detection model; there is an image overlap area between any two adjacent images to be detected, and the size of the image overlap area is determined by the defect size of the substrate image;
[0025] With a preset first confidence value, multiple images to be inspected are input into the preset target detection model to obtain a first recognition result; the first recognition result includes the defect frame size and the defect type.
[0026] The third aspect of the present application further discloses a computer-readable storage medium, which stores instructions. When the instructions are executed, the above method is executed.
[0027] The beneficial effects of this application include:
[0028] The substrate image is segmented by sliding segmentation, and defect detection is performed on the segmented image to avoid the problem that tiny defects cannot be detected after the substrate image is directly compressed; and overlapping areas are set by adjacent images to be detected to avoid missing tiny defects between two adjacent images to be detected during the sliding segmentation process. The size of the overlapping area must be able to cover the size of the defect, especially the size of the defect at the position that needs attention (such as the edge of the substrate), so as to facilitate defect identification. In addition, in order to identify all defects in the substrate image and improve the defect detection rate, a lower confidence value will be set; and, through the sliding segmentation method, multiple images to be detected can be detected separately in a multi-threaded manner, which can improve the efficiency of defect detection;
[0029] Any image to be inspected has positioning coordinates; the coordinates of defects in the image to be inspected are based on the positioning coordinates of the image to be inspected; this facilitates subsequent reporting of the defect positioning coordinates to the defect detection platform;
[0030] The overlap ratio is used to determine whether the defect is the same. The overlap ratio can be obtained by calculating the intersection-over-union ratio of the two defect areas (the ratio between the intersection of the two defect areas and the union of the two defect areas). If it is the same defect, it only needs to be reported to the defect detection platform once; if it is two defects, both defects need to be reported to the defect detection platform.
[0031] Improve the accuracy of defect detection. Train multiple binary classification models for commonly used defects. The binary classification models can identify the defect type. Defects are sequentially input into multiple binary classification models. If a defect does not belong to any defect type, it may be misidentified by the target detection model. Therefore, the second confidence value is set high, much higher than the first confidence value of the target detection model.
[0032] Adjacent defect frames of the same defect type are fused; after fusion, they can be reported to the defect detection platform as a single defect, improving both the accuracy of defect detection and the user's experience. By extracting and constructing new defect frames and performing defect detection again, the accuracy of fusion can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A schematic flow chart of a defect detection method for a large-size substrate disclosed in this application specification;
[0034] Figure 2 This is a schematic diagram of the structure of image overlapping areas in adjacent images to be detected disclosed in this application specification;
[0035] Figure 3This is a schematic diagram of the architecture of a binary classification model group disclosed in this application specification;
[0036] Figure 4 A schematic diagram of the fusion principle of adjacent defect frames disclosed in this application specification;
[0037] Figure 5 This is a schematic diagram of another principle of fusing adjacent defect frames disclosed in this application specification;
[0038] Figure 6 This is a schematic diagram of the structure of a defect detection device for large-size substrates disclosed in this application specification. DETAILED DESCRIPTION
[0039] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.
[0040] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.
[0041] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.
[0042] After thin-film encapsulation is printed on a large substrate, a light source illuminates the substrate and a high-resolution camera captures an image of the substrate. For example, if the substrate is 1500mm*1850mm, a single camera cannot directly capture the entire image. Multiple captures are typically performed to create a complete image. Each captured image has a resolution of 16384*20480, while the image input size for the object detection model is 512*512 pixels. For micron-level defects (even a few pixels), minimizing the substrate image makes it virtually impossible to identify them.
[0043] Furthermore, the term "substrate image" described below in this specification can refer to an image of the entire substrate, or it can be a single image captured during multiple captures of the substrate. This is not a limitation; the specific meaning of "substrate image" varies depending on the actual substrate size and the camera's capture range. This specification primarily discusses large-scale substrates, so the following description uses the example of multiple captures of a substrate, each captured as an example, to define the meaning of "substrate image."
[0044] like Figure 1 As shown, this specification discloses a defect detection method for large-size substrates. The method includes steps S101-S102. The method is applied to a defect detection platform.
[0045] S101. Sliding segmentation is performed on the substrate image with a preset size and a preset step size to obtain multiple images to be detected; the preset size is the input size of the preset target detection model; there is an image overlap area between any two adjacent images to be detected, and the size of the image overlap area is determined by the defect size of the substrate image.
[0046] In this case, the substrate image is segmented using a sliding segmentation method, and defect detection is performed on the segmented images, avoiding the problem of minor defects not being detected after directly compressing the substrate image. Furthermore, by setting an overlapping area between adjacent images to be inspected, the problem of missing minor defects due to the segmentation gap between two adjacent images to be inspected during the sliding segmentation process is avoided. The size of the overlapping area should be able to cover the size of the defect, especially the defect size of the location of concern (such as the edge of the substrate), to facilitate defect identification.
[0047] The preset size in this manual is the maximum size that the preset target detection model can input. The preset step size is set to set the size of the image overlap area. The size of the image overlap area can be set to an empirical value. It can be set according to the general defect size of the defect type that needs to be focused on, or it can be set according to the defect size of the focus area. There is no limit on the specific value. For example: If you are concerned about a 2*2 pixel defect on the edge of the substrate, you can set a 4*4 pixel range to include the image around the defect. In this case, the size of the image overlap area can be set based on the 4*4 pixel range.
[0048] In one example, a substrate image is subjected to sliding segmentation with a preset size and a preset step size to obtain a plurality of images to be detected; specifically, the method includes: constructing a coordinate system on the substrate image with the upper left corner of the substrate image as the coordinate origin; performing sliding segmentation on the substrate image along the X-axis and the Y-axis with a preset size and a preset step size to obtain a plurality of images to be detected and the positioning coordinates of the images to be detected; the positioning coordinates include the coordinates of the upper left corner and the lower right corner of the image to be detected.
[0049] At this point, any image to be inspected has a location coordinate. The coordinates of defects within the image to be inspected are based on the location coordinates of the image to be inspected, making it easier to report the location coordinates of the defects to the defect detection platform.
[0050] like Figure 2 As shown, Figure 2 Two adjacent images to be inspected on the X-axis are shown, along with the step size and image overlap area. The edge size of the image to be inspected on the X-axis equals the step size + the edge size of the image overlap area. When segmenting a substrate image, if the segmented image is missing at the end of the X-axis or Y-axis and is not the standard size for the image to be inspected, image padding (e.g., background image filling) can be used to complete the gap.
[0051] S102 , inputting a plurality of images to be inspected into a preset target detection model with a preset first confidence value to obtain a first recognition result; the first recognition result includes a defect frame size and a defect type.
[0052] Furthermore, to identify all defects in the substrate image and improve the detection rate, a lower confidence value is set. Furthermore, through sliding segmentation, multiple images to be inspected can be inspected separately using multiple threads, improving defect detection efficiency. For example, the first confidence value can be 0.1 or 0.2. This first confidence value can be an empirical value and can be set as needed; there are no restrictions on this.
[0053] This specification does not specifically describe the preset object detection model. Those skilled in the art will appreciate that common object detection models can be used, such as YOLO, DETR, and Faster R-CNN. After conventional training, these object detection models can detect the defects disclosed in this specification.
[0054] Since an overlapping sliding segmentation method is used, the defects in the overlapping areas of the above images need to be paid special attention to.
[0055] In one example, after obtaining the recognition result, the defect detection method also includes: obtaining a first defect area and a second defect area in any image overlapping area; wherein the image overlapping area is the overlapping area between the first image to be detected and the second image to be detected, the first defect area is the pixel area of the first defect in the first image to be detected located in the image overlapping area, and the second defect area is the pixel area of the second defect in the second image to be detected located in the image overlapping area; the first defect area and the second defect area are defects of the same type; obtaining the overlap rate of the first defect area and the second defect area; if the overlap rate is greater than or equal to a preset overlap rate threshold, confirming that the first defect and the second defect are the same defect, and removing the first defect or the second defect.
[0056] At this point, the method for handling defects in overlapping image areas is disclosed; the overlap ratio is used to determine whether they are the same defect, which can be obtained by calculating the intersection-over-union ratio of the two defect areas (the ratio between the intersection of the two defect areas and the union of the two defect areas); if it is the same defect, it only needs to be reported to the defect detection platform once; if it is two defects, both are reported to the defect detection platform. This manual does not provide a detailed explanation of how to obtain the pixel area of the defect. The pixel area can be calculated directly using the defect frame size corresponding to the defect; or the area of the defect can be calculated by calculating the number of pixels occupied by the defect after pixel-level segmentation of the defect. There is no limitation on this.
[0057] Furthermore, in any overlapping image region, the above processing is performed only if both images to be inspected have defects of the same type. If, in one overlapping image region, one image to be inspected has a defect and the other does not, the above processing is not required. Similarly, if, in another overlapping image region, one image to be inspected has a defect of defect type A and the other has a defect of defect type B, the above processing is also not required. In both cases, the defects can be directly reported to the defect detection platform. The information reported to the defect detection platform includes the defect type, defect frame size, and the defect frame's location coordinates.
[0058] In the above description, "first defect" refers to any defect in the first image to be inspected, especially when there are multiple defects in the first image to be inspected. Similarly, "second defect" refers to any defect in the second image to be inspected, especially when there are multiple defects in the second image to be inspected. A single substrate image may correspond to thousands of defects, so reducing duplicate defect reports is of practical significance and also improves the accuracy of substrate reporting.
[0059] like Figure 2 As shown, the defect a1 in the left image to be detected and the defect a2 in the right image to be detected have similar or identical positioning coordinates; the above method can be used to confirm whether they belong to the same defect.
[0060] In one example, the defect detection method further includes: if the overlap rate is less than a preset overlap rate threshold, confirming that the first defect and the second defect are two defects.
[0061] In this example, if there are two defects, both defects will be reported to the defect detection platform. For example, the preset overlap rate can be 0.8, 0.7, etc., and the preset overlap rate threshold can also be an empirical value. It can be set according to actual needs and there is no limit on the specific value.
[0062] The above defect detection is performed with a low first confidence value, which means that defect noise may be introduced. The following discusses how to remove defect noise.
[0063] In one example, after obtaining the first recognition result, the defect detection method also includes: inputting the recognition result into multiple binary classification models in sequence with a preset second confidence value; the preset second confidence value is greater than the preset first confidence value; removing the third defect in the first recognition result to obtain a second recognition result; the confidence value of the third defect is less than the preset second confidence value.
[0064] The goal is to improve defect detection accuracy. Multiple binary classification models are trained on commonly used defects. These models can identify defect types. Defects are sequentially fed into these models. If a defect doesn't fall into any of the defect types, it may be misidentified by the target detection model. Therefore, the second confidence value is set high, significantly greater than the first confidence value of the target detection model.
[0065] For example, the second confidence value can be set to 0.8 or 0.9; it can be an empirical value or set according to actual needs, and there is no restriction on this. Figure 3 As shown in the figure, defects in the image to be inspected that have been identified by the object detection model are sequentially input into multiple binary classification models for screening. If a defect does not belong to any defect type, it is removed. Each binary classification model screens for one type of defect.
[0066] It should be noted that this specification does not specifically describe binary classification models. Conventional binary classification models can be used for conventional training to identify the defect types disclosed in this specification. For example, a binary classification model can be trained using linear mura as positive samples and non-linear mura as negative samples to obtain a binary classification model for linear mura.
[0067] In one example, the multiple binary classification models include binary classification models corresponding to point defects, stripe mura, circular mura, stains, scratches, and linear mura.
[0068] This example lists several common defect types found in substrate images, but this is not a limitation. Band-shaped mura is often shorter in length along the X or Y axis than linear mura; the pixel area of circular mura is often larger than that of dot-shaped defects. Generally speaking, these three types of mura defects are printing defects that occur after thin-film encapsulation printing on large-scale substrates. Dot-shaped defects, stains, and scratches are defects in the substrate itself and can also be detected after inkjet printing.
[0069] In one example, after obtaining the second recognition result, the defect detection method also includes: obtaining a first defect frame and a second defect frame, the first defect frame and the second defect frame are two adjacent defect frames of the same defect type; extracting a first sub-defect frame and a second sub-defect frame to form a third defect frame; wherein the first sub-defect frame is located in the first defect frame, the second sub-defect frame is located in the second defect frame, and the first sub-defect frame is adjacent to the second sub-defect frame; inputting the third defect frame into a preset target detection model to obtain a third recognition result; if there is a defect in the third defect frame in the third recognition result, the first defect frame and the second defect frame are merged into one defect frame, and sent to the defect detection platform as a defect result.
[0070] At this point, adjacent defect frames of the same defect type are fused. After fusion, they can be reported to the defect detection platform as a single defect, improving both the accuracy of defect detection and the user's experience. By extracting and constructing new defect frames and performing defect detection again, the accuracy of fusion can be improved.
[0071] In this example, adjacent defect frames of the same defect type must meet these requirements simultaneously. It's possible for two adjacent defect frames to be different types of defects, or for two defects of the same type to be non-adjacent. In both cases, fusion is not required. Furthermore, for any defect type, fusion can be performed if the above requirements are met.
[0072] In addition, since the target detection model can confirm the defect type and defect location coordinates after recognizing the substrate image, the first defect and the second defect mentioned above can be directly obtained.
[0073] like Figure 4 As shown in Figure 1, a shows two adjacent defect frames of the same type. The positioning coordinates of one defect frame are: the upper left corner coordinates are (x1, y1) and the lower right corner coordinates are (x2, y2); the positioning coordinates of the other defect frame are: the upper left corner coordinates are (x3, y3) and the lower right corner coordinates are (x4, y4). Figure 1 shows the extraction of adjacent parts of these two defect frames, for example, extracting half of each of the two defect frames in a, and detecting them using the above-mentioned object detection model. Figure 1 shows the detection result, a defect frame with positioning coordinates: (x5, y5). The two defect frames with positioning coordinates of (x1, y1)(x2, y2) and (x3, y3)(x4, y4) are then merged into a single defect frame; the coordinates of this new defect frame are: (x1, y1)(x4, y4).
[0074] It should be noted that the upper left corner and lower right corner in this description are relative, and are the coordinates of the two vertices of the diagonal line of the defect frame; the upper left corner in the substrate image is the upper left corner after the substrate image is placed on the conventional horizontal plane, and can also be understood as any corner of the substrate image.
[0075] In one example, the defect detection method further includes: if there are two defects in the third defect frame in the third identification mechanism, the first defect frame and the second defect frame are treated as two defect frames and sent to the defect detection platform as defect results.
[0076] At this time, the reported coordinates of any defect frame are the coordinates of the upper left corner and the lower right corner of the defect frame. In this specification, the target detection model for the defect detection structure is presented as a rectangular or square defect frame.
[0077] like Figure 5As shown in figure a, two adjacent defect frames of the same type are shown. The coordinates of one defect frame are (x1, y1) for the upper left corner and (x2, y2) for the lower right corner; the coordinates of the other defect frame are (x3, y3) for the upper left corner and (x4, y4) for the lower right corner. Figure b shows the extraction of adjacent portions of the two defect frames, for example, half of each of the two defect frames in figure a, and detection using the aforementioned object detection model. Figure c shows the detection result, which still contains two defect frames: one with the coordinates (x7, y7) and (x8, y8) and the other with the coordinates (x9, y9) and (x10, y10). The two defect frames with confirmed positioning coordinates of (x1, y1)(x2, y2) and (x3, y3)(x4, y4) are not merged and are still reported as two defect frames. d shows that the positioning coordinates of the two defect frames are (x1, y1)(x2, y2) and (x3, y3)(x4, y4).
[0078] In an example, two adjacent defect frames specifically include any one of the following: the coordinates of the lower right corner of the first defect frame and the coordinates of the upper left corner of the second defect frame are within a preset coordinate range; the coordinates of the upper right corner of the first defect frame and the coordinates of the lower left corner of the second defect frame are within a preset coordinate range.
[0079] This example illustrates that two adjacent defect frames refer to adjacent coordinates. The preset coordinate range can be either the X-axis or the Y-axis. When two adjacent defect frames are located on the X-axis, the preset coordinate range is the X-axis coordinate range; when two adjacent defect frames are located on the Y-axis, the preset coordinate range is the Y-axis coordinate range. This specification does not limit the specific coordinate range data; for example, it can be a coordinate range corresponding to 50 pixels.
[0080] Figure 4 and Figure 5 The defect frame shown in the example is the defect frame in the X-axis direction. Figure 4 For example, consider two adjacent defect frames. For one defect frame, the upper left corner is (x1, y1) and the lower right corner is (x2, y2); for the other defect frame, the upper left corner is (x3, y3) and the lower right corner is (x4, y4). If the coordinate range between x2 and x3 (either x2-x3 or x3-x2) is within the preset coordinate range, the left and right defect frames are considered adjacent.
[0081] In addition, the above example uses two defect frames as an example to illustrate how to fuse defect frames. However, this is not limited to two defect frames; multiple defect frames can also be used. When there are multiple defect frames, the above method is still used to fuse multiple adjacent defect frames of the same type.
[0082] This specification also discloses a defect detection device for large-size substrates. The defect detection device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory, so that the defect detection device executes the following instructions:
[0083] The substrate image is segmented using a sliding process with a preset size and step size to obtain multiple images to be inspected. The preset size is the input size of the preset target detection model. There is an image overlap between any two adjacent images to be inspected, and the size of the image overlap is determined by the defect size of the substrate image.
[0084] With a preset first confidence value, multiple images to be inspected are input into a preset target detection model to obtain a first recognition result; the first recognition result includes the defect frame size and the defect type.
[0085] In one example, a substrate image is subjected to sliding segmentation with a preset size and a preset step size to obtain a plurality of images to be detected; specifically, the method includes: constructing a coordinate system on the substrate image with the upper left corner of the substrate image as the coordinate origin; performing sliding segmentation on the substrate image along the X-axis and the Y-axis with a preset size and a preset step size to obtain a plurality of images to be detected and the positioning coordinates of the images to be detected; the positioning coordinates include the coordinates of the upper left corner and the lower right corner of the image to be detected.
[0086] In one example, after obtaining the recognition result, the defect detection method also includes: obtaining a first defect area and a second defect area in any image overlapping area; wherein the image overlapping area is the overlapping area between the first image to be detected and the second image to be detected, the first defect area is the pixel area of the first defect in the first image to be detected located in the image overlapping area, and the second defect area is the pixel area of the second defect in the second image to be detected located in the image overlapping area; the first defect area and the second defect area are defects of the same type; obtaining the overlap rate of the first defect area and the second defect area; if the overlap rate is greater than or equal to a preset overlap rate threshold, confirming that the first defect and the second defect are the same defect, and removing the first defect or the second defect.
[0087] In one example, the defect detection method further includes: if the overlap rate is less than a preset overlap rate threshold, confirming that the first defect and the second defect are two defects.
[0088] In one example, after obtaining the first recognition result, the defect detection method also includes: inputting the recognition result into multiple binary classification models in sequence with a preset second confidence value; the preset second confidence value is greater than the preset first confidence value; removing the third defect in the first recognition result to obtain a second recognition result; the confidence value of the third defect is less than the preset second confidence value.
[0089] In one example, the multiple binary classification models include binary classification models corresponding to point defects, stripe mura, circular mura, stains, scratches, and linear mura.
[0090] In one example, after obtaining the second recognition result, the defect detection method also includes: obtaining a first defect frame and a second defect frame, the first defect frame and the second defect frame are two adjacent defect frames of the same defect type; extracting a first sub-defect frame and a second sub-defect frame to form a third defect frame; wherein the first sub-defect frame is located in the first defect frame, the second sub-defect frame is located in the second defect frame, and the first sub-defect frame is adjacent to the second sub-defect frame; inputting the third defect frame into a preset target detection model to obtain a third recognition result; if there is a defect in the third defect frame in the third recognition result, the first defect frame and the second defect frame are merged into one defect frame, and sent to the defect detection platform as a defect result.
[0091] In one example, the defect detection method further includes: if there are two defects in the third defect frame in the third identification mechanism, the first defect frame and the second defect frame are treated as two defect frames and sent to the defect detection platform as defect results.
[0092] In an example, two adjacent defect frames specifically include any one of the following: the coordinates of the lower right corner of the first defect frame and the coordinates of the upper left corner of the second defect frame are within a preset coordinate range; the coordinates of the upper right corner of the first defect frame and the coordinates of the lower left corner of the second defect frame are within a preset coordinate range.
[0093] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0094] The specification also discloses a computer-readable storage medium, which stores instructions. When the instructions are executed, the above method is executed.
[0095] This embodiment also discloses an electronic device, which may be the above-mentioned defect detection device, to perform the above-mentioned method. Figure 6 The electronic device may include: at least one processor 601 , at least one communication bus 602 , a display 603 , a network interface 604 , and at least one memory 605 .
[0096] The communication bus 602 is used to implement the connection and communication between these components.
[0097] The display 603 may include a display screen (Display) and a camera (Camera).
[0098] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0099] The processor 601 may include one or more processing cores. The processor 601 utilizes various interfaces and circuits to connect various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 605, as well as accesses data stored in the memory 605, to perform various server functions and process data. Optionally, the processor 601 may be implemented using at least one hardware form factor selected from the group consisting of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 601 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 601 and may be implemented as a separate chip.
[0100] Among them, the memory 605 may include a random access memory 605 (Random Access Memory, RAM), and may also include a read-only memory 605 (Read-Only Memory). Optionally, the memory 605 includes a non-transitory computer-readable storage medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also be at least one storage device located away from the aforementioned processor 601. As shown in the figure, the memory 605 as a computer storage medium may include an operating system, a network communication module, and application programs of a display module.
[0101] exist Figure 6 In the electronic device shown, the display 603 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 601 can be used to call the application stored in the memory 605. When executed by one or more processors 601, the electronic device executes one or more methods in the above embodiments.
[0102] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.
[0103] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0104] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0105] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0106] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory 605. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory 605 and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory 605 includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.
[0108] The above are merely exemplary embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.
Claims
1. A defect detection method for large-size substrates, characterized in that: The defect detection method comprises: Sliding segmentation is performed on the substrate image with a preset size and a preset step size to obtain multiple images to be detected; the preset size is the input size of the preset target detection model; there is an image overlap area between any two adjacent images to be detected, and the size of the image overlap area is determined by the defect size of the substrate image; With a preset first confidence value, multiple images to be inspected are input into the preset target detection model to obtain a first recognition result; the first recognition result includes the defect frame size and the defect type.
2. The defect detection method according to claim 1, characterized in that: The substrate image is subjected to sliding segmentation with a preset size and a preset step length to obtain multiple images to be inspected; specifically, the following steps are performed: Taking the upper left corner of the substrate image as the coordinate origin, constructing a coordinate system on the substrate image; The substrate image is segmented along the X-axis and the Y-axis with a preset size and a preset step size to obtain a plurality of images to be detected and the positioning coordinates of the images to be detected; the positioning coordinates include the coordinates of the upper left corner and the lower right corner of the images to be detected.
3. The defect detection method according to claim 2, characterized in that: After obtaining the recognition result, the defect detection method further includes: In any one of the image overlapping regions, a first defect area and a second defect area are obtained; wherein the image overlapping region is an overlapping area between a first image to be detected and a second image to be detected, the first defect area is a pixel area of a first defect in the first image to be detected located in the image overlapping region, and the second defect area is a pixel area of a second defect in the second image to be detected located in the image overlapping region; the first defect area and the second defect area are defects of the same type; Obtaining an overlap ratio between the first defect area and the second defect area; If the overlap rate is greater than or equal to a preset overlap rate threshold, it is determined that the first defect and the second defect are the same defect, and the first defect or the second defect is removed.
4. The defect detection method according to claim 3, characterized in that: The defect detection method further includes: If the overlap ratio is less than a preset overlap ratio threshold, it is determined that the first defect and the second defect are two defects.
5. The defect detection method according to claim 3, characterized in that: After obtaining the first recognition result, the defect detection method further includes: Inputting the recognition result into a plurality of binary classification models in sequence with a preset second confidence value; the preset second confidence value is greater than the preset first confidence value; A third defect in the first recognition result is removed to obtain a second recognition result; the confidence value of the third defect is less than the preset second confidence value.
6. The defect detection method according to claim 5, characterized in that: The multiple binary classification models include binary classification models corresponding to point defects, stripe mura, circular mura, stains, scratches, and linear mura.
7. The defect detection method according to claim 5, characterized in that: After obtaining the second recognition result, the defect detection method further includes: Acquire a first defect frame and a second defect frame, where the first defect frame and the second defect frame are two adjacent defect frames of the same defect type; Extracting the first sub-defect frame and the second sub-defect frame to form a third defect frame; wherein the first sub-defect frame is located in the first defect frame, the second sub-defect frame is located in the second defect frame, and the first sub-defect frame is adjacent to the second sub-defect frame; Inputting the third defect frame into the preset target detection model to obtain a third recognition result; If there is a defect in the third defect frame in the third recognition result, the first defect frame and the second defect frame are merged into one defect frame, and the result is sent to the defect detection platform as a defect result.
8. The defect detection method according to claim 7, characterized in that: The defect detection method further includes: If there are two defects in the third defect frame in the third identification mechanism, the first defect frame and the second defect frame are regarded as two defect frames and sent to the defect detection platform as defect results.
9. The defect detection method according to claim 7, characterized in that: The two adjacent defect frames specifically include any of the following: The coordinates of the lower right corner of the first defect frame and the coordinates of the upper left corner of the second defect frame are within a preset coordinate range; The coordinates of the upper right corner of the first defect frame and the coordinates of the lower left corner of the second defect frame are within a preset coordinate range.
10. A defect detection device for large-size substrates, characterized in that: The defect detection device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory, so that the defect detection device performs the following instructions: Sliding segmentation is performed on the substrate image with a preset size and a preset step size to obtain multiple images to be detected; the preset size is the input size of the preset target detection model; there is an image overlap area between any two adjacent images to be detected, and the size of the image overlap area is determined by the defect size of the substrate image; With a preset first confidence value, multiple images to be inspected are input into the preset target detection model to obtain a first recognition result; the first recognition result includes the defect frame size and the defect type.