Display screen defect detection method, terminal equipment and computer readable storage medium
By fusing feature maps of multi-scale texture features and texture regularity, the problem of high false detection and false negative rates in display defect detection is solved, and higher precision defect detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-24
- Publication Date
- 2026-03-27
AI Technical Summary
Existing display defect detection methods are easily affected by fluctuations in the brightness of the display panel itself or changes in ambient light, resulting in a high false detection rate. Furthermore, MURA defects have low contrast with the background, leading to a high missed detection rate.
By acquiring the image of the display screen, and combining it with feature maps of multi-scale texture features and texture regularity, defect detection is performed, including background suppression processing, multi-scale feature fusion and gradient analysis, and thresholds are automatically determined for binarization and defect recognition.
It effectively reduces the probability of false detection and missed detection of display screen defects, improves detection accuracy, enhances the contrast between defects and background, and improves the accuracy of detection results.
Smart Images
Figure CN121746302A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of display screen technology, and in particular relates to a display screen defect detection method, terminal equipment, and computer-readable storage medium. Background Technology
[0002] During the manufacturing process of display panels, various types of defects can occur on the surface or inside the panel due to factors such as production processes, material purity, and environmental cleanliness. MURA defects are a common visual defect. Areas with MURA defects exhibit characteristics such as uneven brightness and color, low contrast with the surrounding background, lack of clear edges, and irregular shapes (such as dots, lines, spots, bands, clumps, etc.).
[0003] Current defect detection methods for displays are often susceptible to fluctuations in the display panel's brightness or changes in ambient lighting, leading to a high false positive rate. Furthermore, due to the low contrast between MURA defects and the background, the false negative rate is also high. Summary of the Invention
[0004] This application provides a method for detecting display screen defects, a terminal device, and a computer-readable storage medium, which can improve the detection accuracy of display screen defects.
[0005] In a first aspect, embodiments of this application provide a method for detecting defects in a display screen, including: Acquire an image to be processed, wherein the image to be processed is an image of the display screen captured by the display screen; A first feature map of the image to be processed is obtained; wherein the first feature map is a feature map obtained by fusing the texture features of the image to be processed at multiple scales. Obtain a second feature map of the image to be processed; wherein the second feature map is used to characterize the regularity of the texture in the image to be processed; A third feature map is obtained by performing feature fusion processing based on the first feature map and the second feature map; The target defects in the image to be processed are detected based on the third feature map, and the detection result is obtained.
[0006] In this embodiment, an image to be processed, i.e., an image of the display screen, is acquired. Defect detection is performed based on the fusion features of a first feature map and a second feature map of the image to be processed. The first feature map characterizes the texture features of the image to be processed at different scales. By extracting texture features at different scales, information loss at a single scale can be reduced, and the complementary nature of multi-scale features helps to strengthen defect features while suppressing interference from background noise and normal pixel textures under different lighting conditions. The second feature map characterizes the regularity of the texture in the image to be processed. By extracting the regularity of the texture, the difference between defect textures and regular textures can be improved, thereby reducing missed detections due to low contrast between defects and the background. In the method of this embodiment, the fusion of the first and second feature maps effectively reduces the probability of false detections and missed detections of display screen defects, thereby improving the detection accuracy of display screen defects.
[0007] In one possible implementation of the first aspect, obtaining the first feature map of the image to be processed includes: The image to be processed is preprocessed to obtain a first image; The first image is subjected to background suppression processing to obtain the second image; The first feature map is calculated based on the second image.
[0008] In the above implementation method, background suppression processing can suppress the interference caused by the brightness gradient of the background image, which is beneficial to enhance the difference between the foreground and the background in the residual image, thus achieving the suppression of the background and highlighting of effective information.
[0009] In one possible implementation of the first aspect, the background suppression processing of the first image to obtain the second image includes: The first image is subjected to local adaptive smoothing to obtain the background image; The second image is obtained by calculating the residual image between the first image and the background image.
[0010] In the above method, the background image obtained by local adaptive smoothing can smooth the details while preserving the edges. Then, the residual image between the first image and the background image is calculated, which effectively suppresses the interference caused by the brightness gradient of the background image and helps to enhance the difference between the foreground and the background in the residual image, thus achieving the suppression of the background and highlighting of effective information.
[0011] In one possible implementation of the first aspect, calculating the first feature map based on the second image includes: Construct a Gaussian pyramid for the second image; wherein the Gaussian pyramid includes sampled images of the second image at multiple scales; Calculate the first variance map corresponding to each sampled image in the Gaussian pyramid; wherein the first variance map is used to characterize the strength of texture in the image; The first feature map is calculated based on the first variance map corresponding to each of the multiple scales.
[0012] In the above method, the first feature map is equivalent to including the strength features of the image texture at multiple scales. By extracting texture features at different scales, the information loss at a single scale can be reduced, and the features at multiple scales are complementary, which is conducive to strengthening defect features, while suppressing the interference of background noise and normal pixel texture under different lighting conditions.
[0013] In one possible implementation of the first aspect, calculating the first feature map based on the first variance maps corresponding to multiple scales includes: Each of the first variance maps is converted into a second variance map; wherein the second variance map has the same scale as the second image; The transformed second variance maps are weighted and fused to obtain the first feature map. The weight coefficients corresponding to the second variance plot are negatively correlated with the scale corresponding to the second variance plot.
[0014] In the above method, by fusing second variance maps at different scales, it is equivalent to fusing image texture features at different scales. This reduces information loss at a single scale, and the complementary nature of features across multiple scales helps to enhance defect features while suppressing interference from background noise and normal pixel textures under different lighting conditions. Furthermore, setting the weight coefficients corresponding to the second variance map to be negatively correlated with the scale corresponding to the second variance map effectively emphasizes defect features at large scales and weakens normal texture features at small scales, thus helping to highlight defects.
[0015] In one possible implementation of the first aspect, obtaining the second feature map of the image to be processed includes: Calculate the gradient magnitude and gradient direction of each pixel in the first image; For each pixel in the first image, the sum of the gradient magnitudes of the target pixels in the preset neighborhood of the pixel is calculated to obtain the first statistical value of the pixel; wherein, the target pixel is a pixel that has a symmetrical relationship with the pixel as the center and has a gradient direction. The second feature map is obtained based on the first statistical value of each pixel in the first image.
[0016] The second feature map obtained by the above method can characterize the regularity of the texture in the image to be processed. By extracting the regularity of the texture, the difference between defect texture and regular texture can be improved, thereby reducing the missed detection caused by the low contrast between defects and background.
[0017] In one possible implementation of the first aspect, the step of detecting the target defect in the image to be processed based on the third feature map to obtain a detection result includes: Calculate the first threshold based on the third feature map; The third feature map is binarized according to the first threshold to obtain a binary image; The target defect is detected based on the binary image, and the detection result is obtained.
[0018] This method automatically determines the threshold based on the grayscale distribution characteristics of the third feature map. Compared to a fixed threshold, the determined first threshold is more adaptable to the characteristics of the third feature map, which helps improve the effect of subsequent binary images. Furthermore, binarization processing further enhances the contrast between defects and non-defects, facilitating subsequent detection.
[0019] In one possible implementation of the first aspect, detecting the target defect based on the binary image to obtain the detection result includes: Identify the morphological features of connected regions in the binary image; Identify the grayscale difference features of the target region in the image to be processed; wherein, the target region is the region in the image to be processed corresponding to the connected region in the binary image; The defect type corresponding to the connected region is identified based on the morphological features and the grayscale difference features, and the detection result is obtained.
[0020] By taking into account both morphological features and grayscale difference features in the above-mentioned method, defect detection is equivalent to identifying defect features from multiple perspectives, which helps to improve the accuracy of defect detection results.
[0021] Secondly, embodiments of this application provide a display screen defect detection device, comprising: An image acquisition unit is used to acquire an image to be processed, wherein the image to be processed is an image of the display screen captured by the display screen; The first feature extraction unit is used to obtain a first feature map of the image to be processed; wherein, the first feature map is a feature map obtained by fusing the texture features corresponding to the image to be processed at multiple scales; The second feature extraction unit is used to obtain a second feature map of the image to be processed; wherein the second feature map is used to characterize the regularity of the texture in the image to be processed; The feature fusion unit is used to perform feature fusion processing based on the first feature map and the second feature map to obtain a third feature map; The defect detection unit is used to detect target defects in the image to be processed based on the third feature map, and obtain the detection result.
[0022] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the display screen defect detection method as described in any one of the first aspects above.
[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the display screen defect detection method as described in any one of the first aspects above.
[0024] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the display screen defect detection method described in any one of the first aspects.
[0025] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic flowchart of the display screen defect detection method provided in the embodiments of this application; Figure 2 This is a schematic diagram illustrating the effect of background suppression processing provided in the embodiments of this application; Figure 3 This is a schematic diagram of the multi-scale texture enhancement effect provided in the embodiments of this application; Figure 4 This is a schematic diagram illustrating the gradient symmetry processing effect provided in the embodiments of this application; Figure 5 This is a schematic diagram illustrating the defect detection effect provided in the embodiments of this application; Figure 6 This is a schematic diagram of the overall defect detection process provided in the embodiments of this application; Figure 7 This is a structural block diagram of the display screen defect detection device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation
[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0031] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0032] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0033] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.
[0034] During the manufacturing process of display panels, various types of defects can occur on the surface or inside the panel due to factors such as production processes, material purity, and environmental cleanliness. MURA defects are a common visual defect. Areas with MURA defects exhibit characteristics such as uneven brightness and color, low contrast with the surrounding background, lack of clear edges, and irregular shapes (such as dots, lines, spots, bands, clumps, etc.).
[0035] Current defect detection methods for displays are often susceptible to fluctuations in the display panel's brightness or changes in ambient lighting, leading to a high false positive rate. Furthermore, due to the low contrast between MURA defects and the background, the false negative rate is also high.
[0036] Based on this, this application provides a method for detecting display screen defects. In this application embodiment, an image to be processed, i.e., an image of the display screen's display, is acquired. Defect detection is performed based on the fusion features of a first feature map and a second feature map of the image to be processed. The first feature map characterizes the texture features of the image to be processed at different scales. By extracting texture features at different scales, information loss at a single scale can be reduced, and the complementary nature of multi-scale features helps to strengthen defect features while suppressing interference from background noise and normal pixel textures under different lighting conditions. The second feature map characterizes the regularity of the texture in the image to be processed. By extracting the regularity of the texture, the difference between defective textures and regular textures can be improved, thereby reducing missed detections due to low contrast between defects and the background. In the method of this application embodiment, the fusion of the first and second feature maps effectively reduces the probability of false detections and missed detections of display screen defects, thereby improving the detection accuracy of display screen defects.
[0037] See Figure 1 This is a flowchart illustrating a display screen defect detection method provided in an embodiment of this application. It is intended as an example and not a limitation. The method may include the following steps: S101, Obtain the image to be processed.
[0038] In this embodiment of the application, the image to be processed is an image of the display screen captured by the application.
[0039] In some application scenarios, a defect detection system may include a terminal device and a data acquisition device. For example, the terminal device may be a desktop computer, laptop computer, or other device with data processing capabilities, while the data acquisition device may be a CCD camera or a CMOS camera. The terminal device and the data acquisition device can be connected via wired or wireless means. For instance, the data acquisition device can be installed on the terminal device, and the two can be connected via a wired connection; alternatively, the data acquisition device can communicate with the terminal device over a long distance. The data acquisition device is used to acquire the display screen image to be inspected, obtain the acquired image, and transmit the acquired image to the terminal device. Correspondingly, the terminal device receives the acquired image, uses it as an image to be processed, and then executes the display screen defect detection method of this application embodiment.
[0040] In other application scenarios, multiple acquisition devices can acquire images of different areas of the display screen and transmit them to the terminal device. The terminal device then stitches the multiple acquired images into a single complete image, which is used as the image to be processed.
[0041] Optionally, the image to be processed in this embodiment is a grayscale image.
[0042] S102, Obtain the first feature map of the image to be processed.
[0043] The first feature map is a feature map obtained by fusing the texture features of the image to be processed at multiple scales. It is understood that the "scale" in this embodiment refers to the window size or receptive field range for observing image texture, determining the level of texture details that can be captured. Smaller scales correspond to smaller windows / receptive fields, enabling focus on local, subtle textures in the image; while larger scales correspond to larger windows / receptive fields, enabling capture of the overall texture distribution and better resistance to interference.
[0044] In one embodiment, S102 may include: The image to be processed is preprocessed to obtain the first image; The first image is subjected to background suppression processing to obtain the second image; The first feature map is calculated based on the second image.
[0045] Optionally, the preprocessing procedure may include: convolving the image to be processed using a Gaussian kernel to obtain a first image. For example, according to the formula... ,in, For Gaussian kernel, The image to be processed. This is the first image. High-frequency noise can be effectively suppressed through Gaussian kernel convolution processing.
[0046] Optionally, image normalization processing can also be performed on the first image. Image normalization processing may include brightness adjustment and contrast adjustment, etc. For example, according to the formula... Perform a linear transformation on the first image, where, α Contrast coefficient, β This represents the brightness offset coefficient. Linear transformation processing can adjust the brightness and contrast of an image to reduce differences in brightness and contrast between different batches of images. For example, it can be calculated using the formula... The first image is normalized, where, The average brightness of the first image. This represents the standard deviation of the brightness of the first image. Normalization can be used to adjust the brightness and contrast of a single image. Alternatively, a linear transformation can be applied to the first image to reduce the differences in brightness and contrast between it and other batches of images; then, normalization can be applied to the linearly transformed first image to adjust the differences in brightness and contrast between its own pixels.
[0047] It should be noted that the above is only an example of image standardization processing. In actual applications, different methods of image standardization processing can be performed according to different needs. This application does not impose any specific limitations on this.
[0048] In the above implementation method, background suppression processing can suppress the interference caused by the brightness gradient of the background image, which is beneficial to enhance the difference between the foreground and the background in the residual image, thus achieving the suppression of the background and highlighting of effective information.
[0049] In one implementation, the background suppression process may include the following steps: The first image is subjected to local adaptive smoothing to obtain the background image; The second image is obtained by calculating the residual image between the first image and the background image.
[0050] Optionally, the first image can be used as a guide image for filtering. For example, the filtering process may include: traversing the first image using preset local windows, calculating statistical data (such as mean and variance) for the local image within each preset local window; calculating filtering parameters (such as local gain a_k and local offset b_k) corresponding to each preset local window based on the statistical data; and filtering the local image within each preset local window according to the filtering parameters (e.g., using the formula...). Perform filtering processing. For the k-th local image, (For the k-th local image after filtering), the filtered local image corresponding to the preset local window is obtained.
[0051] Optional, it can be based on the formula Calculate the second image, where, For the second image; Background image; is the first image; C is a preset constant used to adjust the residual image to a suitable grayscale range.
[0052] In the above method, the background image obtained by local adaptive smoothing can smooth the details while preserving the edges. Then, the residual image between the first image and the background image is calculated, which effectively suppresses the interference caused by the brightness gradient of the background image and helps to enhance the difference between the foreground and the background in the residual image, thus achieving the suppression of the background and highlighting of effective information.
[0053] For example, see Figure 2 This is a schematic diagram illustrating the effect of background suppression processing provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 2 Image (a) shows the first image, as shown in Figure (a). Figure 2 Image (b) shows the background image, as shown below. Figure 2 Image (c) in the diagram is the second image. From... Figure 2 As can be seen, the second image after background suppression processing shows that the background is effectively suppressed compared to the first image, while effective information (such as...) is retained. Figure 2 The foreground texture 21 shown is highlighted.
[0054] In one implementation, the step of calculating the first feature map based on the second image may include: Construct a Gaussian pyramid for the second image; wherein the Gaussian pyramid includes sampled images of the second image at multiple scales; Calculate the first variance map corresponding to each sampled image in the Gaussian pyramid; where the first variance map is used to characterize the strength of texture in the image; The first feature map is calculated based on the first variance map corresponding to each of the multiple scales.
[0055] Optionally, the steps for obtaining the Gaussian pyramid may include: using the second image as a reference, downsampling layer by layer to obtain the sampled image at each scale (i.e., the image of each pyramid layer). For example, taking a three-layer pyramid as an example, the second image is used as the image of the first pyramid layer; the second image is downsampled to obtain the image of the second pyramid layer; the image of the second pyramid layer is downsampled to obtain the image of the third pyramid layer.
[0056] Understandably, the more layers the pyramid has, the more image texture features at different scales can be acquired, but the data processing volume also increases. The number of pyramid layers can be set according to actual needs; this application does not specifically limit the number of Gaussian pyramid layers.
[0057] Optional, it can be based on the formula Calculate the first variance plot. Wherein, The position in the i-th layer of the pyramid image The variance at that location, Position in the second image Pixel value at that location, For A local window centered on the center. For local windows The average pixel value within the range, This represents the total number of local windows.
[0058] In the above method, the first feature map is equivalent to including the strength features of the image texture at multiple scales. By extracting texture features at different scales, the information loss at a single scale can be reduced, and the features at multiple scales are complementary, which is conducive to strengthening defect features, while suppressing the interference of background noise and normal pixel texture under different lighting conditions.
[0059] Optionally, the step of calculating the first feature map based on the first variance maps corresponding to multiple scales may include: Each first variance map is converted into a second variance map; wherein the second variance map has the same scale as the second image; All the transformed second variance maps are weighted and fused to obtain the first feature map; Among them, the weight coefficients corresponding to the second variance plot are negatively correlated with the scale corresponding to the second variance plot.
[0060] For example, taking the three-tiered Gaussian pyramid as an example, it can be calculated according to the formula... The second variance plot is then weighted and merged. This is the first feature map. This is the second variance plot of the first level of the Gaussian pyramid. This is the second variance plot of the second level of the Gaussian pyramid. This is the second variance plot of the third level of the Gaussian pyramid. represents the weighting coefficient. Wherein, .
[0061] In the above method, by fusing second variance maps at different scales, it is equivalent to fusing image texture features at different scales. This reduces information loss at a single scale, and the complementary nature of features across multiple scales helps to enhance defect features while suppressing interference from background noise and normal pixel textures under different lighting conditions. Furthermore, setting the weight coefficients corresponding to the second variance map to be negatively correlated with the scale corresponding to the second variance map effectively emphasizes defect features at large scales and weakens normal texture features at small scales, thus helping to highlight defects.
[0062] For example, see Figure 3 This is a schematic diagram illustrating the multi-scale texture enhancement effect provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 3 Figure (a) shows the second variance plots for scales 1, 2, and 3, respectively, where scale 1 > scale 2 > scale 3. Figure 3 Image (b) shows the first feature map after fusion. From Figure 3 As can be seen from this, the effective information in the first feature map (such as...) Figure 3 The foreground texture 31 shown is highlighted.
[0063] S103, Obtain the second feature map of the image to be processed.
[0064] The second feature map is used to characterize the regularity of the texture in the image to be processed. It is understood that the image texture of defective regions is usually asymmetric, while the image texture of normal regions is usually symmetric. Based on this, defective regions and normal regions can be distinguished by the regularity of the image texture (i.e., the degree of symmetry).
[0065] In one embodiment, S103 may include: Calculate the gradient magnitude and gradient direction of each pixel in the first image; For each pixel in the first image, the sum of the gradient magnitudes of the target pixels in the preset neighborhood of the pixel is calculated to obtain the first statistical value of the pixel; wherein, the target pixel is a pixel that has a symmetrical relationship with the pixel as the center and has a gradient direction. The second feature map is obtained based on the first statistical value of each pixel in the first image.
[0066] Optionally, the calculation process of gradient magnitude and gradient direction may include: calculating the horizontal gradient and vertical gradient of each pixel in the first image; and calculating the gradient magnitude and gradient direction of each pixel in the first image based on the horizontal gradient and vertical gradient of each pixel in the first image.
[0067] Among them, the gradient magnitude can characterize the degree of change in pixel grayscale value, and the gradient direction can characterize the direction of the drastic change in grayscale value.
[0068] For example, the Sobel operator can be used to calculate the horizontal and vertical gradients of pixels. The Sobel operator can be a convolution kernel. During the calculation, the convolution kernel moves along the horizontal direction of the image with a preset stride. At each position, the kernel performs convolution calculations on the pixels within the current image region to obtain the corresponding horizontal gradient. Similarly, the kernel moves along the vertical direction of the image with a preset stride. At each position, the kernel performs convolution calculations on the pixels within the current image region to obtain the corresponding vertical gradient.
[0069] For example, it can be based on the formula Calculate the gradient magnitude using the formula. Calculate the gradient direction. Wherein, For horizontal gradient, For vertical gradient, For gradient magnitude, The gradient direction.
[0070] In this embodiment of the application, the target pixel can be a pixel with the same center and opposite gradient directions. For example, for pixel p, if there are pixels q and q' with opposite gradient directions in its preset neighborhood, and pixel p is the midpoint between pixels q and q', then pixels q and q' are recorded as target pixels.
[0071] Optionally, the range of the preset neighborhood can be matched with the kernel size of the Sobel operator.
[0072] Optionally, the step of obtaining the second feature map based on the first statistical value of each pixel in the first image may include: for each pixel in the first image, according to the formula .in, is the gray value of pixel P in the second feature map, and is the first statistical value of pixel P.
[0073] The second feature map obtained by the above method can characterize the regularity of the texture in the image to be processed. By extracting the regularity of the texture, the difference between defect texture and regular texture can be improved, thereby reducing the missed detection caused by the low contrast between defects and background.
[0074] For example, see Figure 4 This is a schematic diagram illustrating the gradient symmetry processing effect provided in the embodiments of this application. It is intended as an example and not a limitation. Figure 4 Image (a) shows the first image, as shown in Figure (a). Figure 4 Image (b) shows the horizontal gradient map of the first image, as shown below. Figure 4 In the diagram, (c) represents the vertical gradient map of the first image, as shown below. Figure 4In the image, (d) represents the gradient magnitude map of the first image, such as... Figure 4 In the diagram, (e) represents the second feature map. From... Figure 4 As can be seen from this, the defect texture in the second feature map ( Figure 4 Texture 41 in the image is highlighted, while other normal textures are toned down.
[0075] S104, perform feature fusion processing based on the first feature map and the second feature map to obtain the third feature map.
[0076] Optionally, the first feature map and the second feature map can be multiplied by pixels to obtain the third feature map.
[0077] Feature fusion processing can highlight areas that are both significant and textured and irregular, thus facilitating subsequent defect detection.
[0078] S105, detect the target defect in the image to be processed based on the third feature map, and obtain the detection result.
[0079] In one embodiment, S105 may include: Calculate the first threshold based on the third feature map; The third feature map is binarized according to the first threshold to obtain a binary image; The detection results are obtained by detecting target defects based on binary images.
[0080] Optionally, the process of calculating the first threshold may include: counting the number of pixels appearing at each gray level in the third feature map to obtain a gray-level histogram; dividing the third feature map into multiple gray levels and a threshold corresponding to each gray level based on the gray-level histogram; calculating statistical data (such as inter-class variance or gray-level mean) corresponding to each gray level; and using the threshold of the gray level corresponding to the optimal statistical data as the first threshold.
[0081] This method automatically determines the threshold based on the grayscale distribution characteristics of the third feature map itself. Compared with a fixed threshold, the determined first threshold is more adaptable to the characteristics of the third feature map, which helps to improve the effect of subsequent binary images.
[0082] Optionally, before detection based on the binary image, the process may further include: performing morphological opening operations (such as erosion followed by dilation) on the binary image to remove small noise points; then performing morphological closing operations (such as dilation followed by erosion) to connect broken regions; finally, filtering out smaller connected regions based on an area threshold to obtain a processed binary image, and then detecting the target defect based on the processed binary image. This processing further highlights the defect region, which is beneficial for improving subsequent detection accuracy.
[0083] In one implementation, the step of detecting target defects based on a binary image may include: Identify the morphological features of connected regions in a binary image; Identify the gray-level difference features of the target region in the image to be processed; wherein, the target region is the region in the image to be processed corresponding to the connected region in the binary image; Based on morphological features and grayscale difference features, the defect type corresponding to the connected region is identified, and the detection result is obtained.
[0084] Optionally, morphological features may include at least one of the following: area, perimeter, compactness, and minimum bounding rectangle. For example, the number of pixels within each connected region of a binary image can be counted, and the area of the connected region can be calculated based on the number of pixels. The number of boundary pixels of each connected region in a binary image can be detected to calculate the perimeter of the connected region. The ratio of the area to the perimeter of each connected region can be calculated as the compactness of the connected region. The minimum bounding rectangle of the connected region can be calculated based on the boundary pixels of the connected region.
[0085] Optionally, the step of obtaining grayscale difference features may include: obtaining the target region corresponding to the connected region in the image to be processed based on the position of each connected region in the binary image, and calculating the grayscale difference between the target region and the surrounding region.
[0086] Optionally, the defect type identification step may include: obtaining preset rules; identifying the defect type corresponding to the connected region based on morphological features, grayscale difference features, and the preset rules, and obtaining the detection result. The preset rules may include multiple rules, and each rule may include a threshold corresponding to the morphological features, a threshold corresponding to the grayscale difference features, and the corresponding defect type.
[0087] For example, a rule states that if the area is less than a first value and the grayscale difference is greater than a second value, then the defect type is a point defect. A rule states that if the perimeter is greater than a third value and the aspect ratio of the smallest bounding rectangle is greater than a fourth value, then the defect type is a line defect. It should be noted that the above are merely examples of rules, and the judgment conditions of the rules and the correspondence between the conditions and the defect types are not specifically limited in the embodiments of this application.
[0088] By taking into account both morphological features and grayscale difference features in the above-mentioned method, defect detection is equivalent to identifying defect features from multiple perspectives, which helps to improve the accuracy of defect detection results.
[0089] For example, see Figure 5 This is a schematic diagram illustrating the defect detection effect provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 5 Image (a) shows the first feature map, as shown in Figure (a). Figure 5 Image (b) shows the second feature map, as shown below. Figure 5(c) in the figure shows the third feature map, as shown in the figure. Figure 5 (d) in the image shows a binary image, as shown below. Figure 5 (e) in the diagram shows the detection results. Figure 5 As shown, the fusion of the first feature map and the second feature map can enhance the contrast between the defect area and the background, and improve the difference between the defect texture and the regular texture, thereby improving the accuracy of defect detection.
[0090] See Figure 6 This is a schematic diagram of the overall defect detection process provided in the embodiments of this application. It is intended as an example and not a limitation. Figure 6 As shown, the overall defect detection process can include the following steps: S601, Obtain the image to be processed.
[0091] S602, preprocess the image to be processed to obtain the first image.
[0092] S603, perform background suppression processing on the first image to obtain the second image.
[0093] S604, Calculate the first feature map based on the second image.
[0094] Steps S602-S604 are the same as step S102 above. For details, please refer to the description in the embodiment of S102, which will not be repeated here.
[0095] S605, Calculate the second feature map based on the first image.
[0096] Step S605 is the same as step S103 above. For details, please refer to the description in the embodiment of S103, which will not be repeated here.
[0097] S606, the first feature map and the second feature map are fused to obtain the third feature map.
[0098] Step S606 is the same as step S104 above. For details, please refer to the description in the embodiment of S104, which will not be repeated here.
[0099] S607, post-process the third feature map to obtain a binary image.
[0100] S608, detects target defects based on binary images and obtains detection results.
[0101] Steps S607-S608 are the same as step S105 above, and can be found in the description in the embodiment of S105, which will not be repeated here. Post-processing may include binarization and processing steps prior to detection based on the binary image, such as morphological opening operations, morphological closing operations, and area filtering.
[0102] In this embodiment, an image to be processed, i.e., an image of the display screen, is acquired. Defect detection is performed based on the fusion features of a first feature map and a second feature map of the image to be processed. The first feature map characterizes the texture features of the image to be processed at different scales. By extracting texture features at different scales, information loss at a single scale can be reduced, and the complementary nature of multi-scale features helps to strengthen defect features while suppressing interference from background noise and normal pixel textures under different lighting conditions. The second feature map characterizes the regularity of the texture in the image to be processed. By extracting the regularity of the texture, the difference between defect textures and regular textures can be improved, thereby reducing missed detections due to low contrast between defects and the background. In the method of this embodiment, the fusion of the first and second feature maps effectively reduces the probability of false detections and missed detections of display screen defects, thereby improving the detection accuracy of display screen defects.
[0103] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0104] Corresponding to the display screen defect detection method described in the above embodiments, Figure 7 This is a structural block diagram of the display screen defect detection device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0105] Reference Figure 7 The device 7 includes: Image acquisition unit 71 is used to acquire an image to be processed, wherein the image to be processed is an image of the display screen captured by the screen.
[0106] The first feature extraction unit 72 is used to obtain a first feature map of the image to be processed; wherein the first feature map is a feature map obtained by fusing the texture features corresponding to the image to be processed at multiple scales.
[0107] The second feature extraction unit 73 is used to obtain a second feature map of the image to be processed; wherein the second feature map is used to characterize the regularity of the texture in the image to be processed.
[0108] The feature fusion unit 74 is used to perform feature fusion processing based on the first feature map and the second feature map to obtain a third feature map.
[0109] The defect detection unit 75 is used to detect target defects in the image to be processed based on the third feature map, and obtain the detection result.
[0110] Optionally, the first feature extraction unit 72 is also used for: The image to be processed is preprocessed to obtain a first image; The first image is subjected to background suppression processing to obtain the second image; The first feature map is calculated based on the second image.
[0111] Optionally, the first feature extraction unit 72 is also used for: The first image is subjected to local adaptive smoothing to obtain the background image; The second image is obtained by calculating the residual image between the first image and the background image.
[0112] Optionally, the first feature extraction unit 72 is also used for: Construct a Gaussian pyramid for the second image; wherein the Gaussian pyramid includes sampled images of the second image at multiple scales; Calculate the first variance map corresponding to each sampled image in the Gaussian pyramid; wherein the first variance map is used to characterize the strength of texture in the image; The first feature map is calculated based on the first variance map corresponding to each of the multiple scales.
[0113] Optionally, the first feature extraction unit 72 is also used for: Each of the first variance maps is converted into a second variance map; wherein the second variance map has the same scale as the second image; The transformed second variance maps are weighted and fused to obtain the first feature map. The weight coefficients corresponding to the second variance plot are negatively correlated with the scale corresponding to the second variance plot.
[0114] Optionally, the second feature extraction unit 73 is also used for: Calculate the gradient magnitude and gradient direction of each pixel in the first image; For each pixel in the first image, the sum of the gradient magnitudes of the target pixels in the preset neighborhood of the pixel is calculated to obtain the first statistical value of the pixel; wherein, the target pixel is a pixel that has a symmetrical relationship with the pixel as the center and has a gradient direction. The second feature map is obtained based on the first statistical value of each pixel in the first image.
[0115] Optionally, the defect detection unit 75 is also used for: Calculate the first threshold based on the third feature map; The third feature map is binarized according to the first threshold to obtain a binary image; The target defect is detected based on the binary image, and the detection result is obtained.
[0116] Optionally, the defect detection unit 75 is also used for: Identify the morphological features of connected regions in the binary image; Identify the grayscale difference features of the target region in the image to be processed; wherein, the target region is the region in the image to be processed corresponding to the connected region in the binary image; The defect type corresponding to the connected region is identified based on the morphological features and the grayscale difference features, and the detection result is obtained.
[0117] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0118] in addition, Figure 7 The display screen defect detection device shown can be a software unit, hardware unit, or a combination of software and hardware built into existing terminal equipment, or it can be integrated into the terminal equipment as an independent component, or it can exist as an independent terminal equipment.
[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0120] Figure 8 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 8 As shown, the terminal device 8 of this embodiment includes: at least one processor 80 ( Figure 8(Only one is shown in the diagram) a processor, a memory 81, and a computer program 82 stored in the memory 81 and executable on the at least one processor 80, wherein the processor 80 executes the computer program 82 to implement the steps in any of the above embodiments of the display defect detection method.
[0121] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 8 This is merely an example of terminal device 8 and does not constitute a limitation on terminal device 8. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.
[0122] The processor 80 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0123] In some embodiments, the memory 81 may be an internal storage unit of the terminal device 8, such as a hard disk or memory of the terminal device 8. In other embodiments, the memory 81 may be an external storage device of the terminal device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 8. Furthermore, the memory 81 may include both internal and external storage units of the terminal device 8. The memory 81 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 81 can also be used to temporarily store data that has been output or will be output.
[0124] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.
[0125] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments.
[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.
[0127] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0128] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0129] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0131] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for detecting defects in a display screen, characterized in that, include: Acquire an image to be processed, wherein the image to be processed is an image of the display screen captured by the display screen; A first feature map of the image to be processed is obtained; wherein the first feature map is a feature map obtained by fusing the texture features of the image to be processed at multiple scales. Obtain a second feature map of the image to be processed; wherein the second feature map is used to characterize the regularity of the texture in the image to be processed; A third feature map is obtained by performing feature fusion processing based on the first feature map and the second feature map; The target defects in the image to be processed are detected based on the third feature map, and the detection result is obtained.
2. The display screen defect detection method as described in claim 1, characterized in that, The step of obtaining the first feature map of the image to be processed includes: The image to be processed is preprocessed to obtain a first image; The first image is subjected to background suppression processing to obtain the second image; The first feature map is calculated based on the second image.
3. The display screen defect detection method as described in claim 2, characterized in that, The step of performing background suppression processing on the first image to obtain the second image includes: The first image is subjected to local adaptive smoothing to obtain the background image; The second image is obtained by calculating the residual image between the first image and the background image.
4. The display screen defect detection method as described in claim 2, characterized in that, The step of calculating the first feature map based on the second image includes: Construct a Gaussian pyramid for the second image; wherein the Gaussian pyramid includes sampled images of the second image at multiple scales; Calculate the first variance map corresponding to each sampled image in the Gaussian pyramid; wherein the first variance map is used to characterize the strength of texture in the image; The first feature map is calculated based on the first variance map corresponding to each of the multiple scales.
5. The display screen defect detection method as described in claim 4, characterized in that, The step of calculating the first feature map based on the first variance maps corresponding to multiple scales includes: Each of the first variance maps is converted into a second variance map; wherein the second variance map has the same scale as the second image; The transformed second variance maps are weighted and fused to obtain the first feature map. The weight coefficients corresponding to the second variance plot are negatively correlated with the scale corresponding to the second variance plot.
6. The display screen defect detection method as described in claim 2, characterized in that, The process of obtaining the second feature map of the image to be processed includes: Calculate the gradient magnitude and gradient direction of each pixel in the first image; For each pixel in the first image, the sum of the gradient magnitudes of the target pixels in the preset neighborhood of the pixel is calculated to obtain the first statistical value of the pixel; wherein, the target pixel is a pixel that has a symmetrical relationship with the pixel as the center and has a gradient direction. The second feature map is obtained based on the first statistical value of each pixel in the first image.
7. The display screen defect detection method according to any one of claims 1 to 6, characterized in that, The step of detecting target defects in the image to be processed based on the third feature map to obtain detection results includes: Calculate the first threshold based on the third feature map; The third feature map is binarized according to the first threshold to obtain a binary image; The target defect is detected based on the binary image, and the detection result is obtained.
8. The display screen defect detection method as described in claim 7, characterized in that, The step of detecting the target defect based on the binary image and obtaining the detection result includes: Identify the morphological features of connected regions in the binary image; Identify the grayscale difference features of the target region in the image to be processed; wherein, the target region is the region in the image to be processed corresponding to the connected region in the binary image; The defect type corresponding to the connected region is identified based on the morphological features and the grayscale difference features, and the detection result is obtained.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.