Image defect detection method, device and storage medium

Abstract

The embodiment of the present application discloses an image defect detection method, device and storage medium for detecting screen defects and improving the defect detection rate. The method of the present application includes: obtaining a first image and a second image and calculating the mean of the first image according to different light sources and light source positions of the camera; calculating a correction coefficient matrix according to the first image, the second image and the mean of the first image; determining the coordinate position of the dust and the coordinate position of the noise in the first image; adjusting the correction coefficient matrix according to the coordinate position of the dust and the coordinate position of the noise, and correcting the first image with the adjusted correction coefficient matrix; obtaining the fitting edge of the display screen in the first image, and obtaining a third image by calculating the homography matrix; performing a convolution operation with the third image according to the adaptive spatial filter kernel matrix; performing threshold control on the third image after the convolution operation to filter out the fourth image; performing eigenvalue analysis on the fourth image to obtain defect features.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to an image defect detection method, device, and storage medium. Background Art

[0002] With the advancement of science and technology, display technology is constantly iterating. New display technologies have higher resolution, color performance and response speed, but at the same time they also bring challenges to existing automated detection methods.

[0003] During the production process, displays are susceptible to various defects, such as bright spots, dark spots, line defects, and color spots, due to factors such as process equipment, labor, and the environment. Furthermore, the variety of display defect types presents significant challenges for detection. Due to this wide range of defect types, common contrast enhancement algorithms are easily affected by the characteristics of different defect types, making it difficult to effectively distinguish defects.

[0004] Furthermore, the defects that appear on the surface of display screens are usually extremely small. Existing networks frequently use strided convolutional layers or pooling layers to perform downsampling operations during forward propagation to reduce computational complexity and expand the receptive field. However, this process inevitably results in a significant loss of fine-grained feature information about the defective target. This is especially true when processing low-resolution images, where the information carried by the image itself is relatively limited, and downsampling further exacerbates this information loss. When the size of a defective target is extremely small, its fine-grained features are almost completely lost after multiple downsampling cycles, ultimately making it difficult to achieve the ideal level of detection accuracy and unable to meet the stringent requirements of industrial production for high-precision detection. Summary of the Invention

[0005] The embodiments of the present application provide an image defect detection method, device, and storage medium for detecting screen defects and improving the defect detection rate.

[0006] A first aspect of an embodiment of the present application provides a method for detecting defects in an image, comprising:

[0007] According to different light sources and light source positions of the camera, a first image and a second image are acquired and a first image mean is calculated, where the first image mean is the mean of all pixel values ​​in the first image;

[0008] Calculating a correction coefficient matrix according to the first image, the second image, and the mean of the first image;

[0009] Determining the coordinate positions of dust and noise points in the first image;

[0010] adjusting the correction coefficient matrix according to the coordinate positions of the dust and the coordinate positions of the noise points, and correcting the first image using the adjusted correction coefficient matrix;

[0011] Obtaining a fitted edge of the display screen in the first image, and obtaining a third image by calculating a homography matrix, where the third image is a display area of ​​the display screen in the first image;

[0012] performing a convolution operation on the third image according to an adaptive spatial filter kernel matrix;

[0013] Performing a threshold control on the third image after the convolution operation to filter out a fourth image, where the fourth image is an image portion of the third image that meets the threshold control requirement;

[0014] Performing feature value analysis on the fourth image to obtain defect features.

[0015] Optionally, acquiring the first image and the second image and calculating the mean of the first image according to different light sources and light source positions of the camera includes:

[0016] Setting the light source at different positions to take photos of the display screen, collecting a first set of photos and superimposing the first set of photos to obtain a first image;

[0017] After shielding the camera lens, take photos of the display screen, collect a second set of photos, and superimpose the second set of photos to obtain a second image;

[0018] Statistical calculation is performed on the pixel values ​​of the first image to obtain a first image mean.

[0019] Optionally, determining the coordinate positions of dust and noise in the first image includes:

[0020] determining the coordinate position of dust in the first image according to the amplitude and direction of the first image;

[0021] The coordinate positions of the noise points in the first image are determined by comparing the vector diagram mapping of the first image.

[0022] Optionally, determining the coordinate position of dust in the first image by using the amplitude and direction of the first image includes:

[0023] performing smoothing filtering on the first image;

[0024] Passing a gradient filter kernel through the first image to obtain a gradient magnitude and a gradient direction of the first image;

[0025] For each pixel point in the first image, along the gradient direction of the pixel point, the gradient amplitude of the pixel point is compared with the gradient amplitudes of the pixels on both sides of the gradient direction to determine the coordinate position of the dust in the first image.

[0026] Optionally, determining the coordinate position of the noise point in the first image by comparing the vector diagram mapping of the first image includes:

[0027] Generate two completely black images with the same size as the first image;

[0028] The row coordinate values ​​and column coordinate values ​​of the completely black image are respectively set to the grayscale values ​​of the two completely black images;

[0029] Taking each pixel in the completely black image as the center, calculate the neighborhood row and column coordinate array of each pixel;

[0030] Perform arithmetic operations on the grayscale values ​​according to the order of the neighborhood row and column coordinate arrays, and perform grayscale threshold verification;

[0031] Taking the intersection of image regions that meet the grayscale threshold condition in the completely black image, retaining the corresponding intersection region from the first image, and converting the intersection region into a vector image;

[0032] Mapping the first image with the vector image to obtain a plurality of mapped images centered on pixel points;

[0033] Performing arithmetic operations on the plurality of mapped images and the first image respectively, and determining the coordinate position of the noise point according to the output results, wherein the output results include the grayscale values ​​of the same position in the plurality of mapped images.

[0034] Optionally, obtaining the fitted edge of the display screen in the first image and obtaining the third image through homography matrix calculation includes:

[0035] constructing a plurality of rectangular frames at the edge of the display screen in the first image;

[0036] Traverse the grayscale values ​​of the pixels in each rectangular frame and calculate the grayscale difference between each pixel and its adjacent pixels;

[0037] Filtering the edge pixel coordinates of the display screen in the first image according to the grayscale difference;

[0038] Fitting the edge pixel coordinates to the fitted edges of the display screen in the first image, and calculating the intersection of the fitted edges, where the intersection of the fitted edges is the edge corner point of the display screen in the first image;

[0039] The fitted edges and intersections are corrected using a homography matrix to obtain a third image.

[0040] Optionally, after obtaining the fitted edge of the display screen in the first image and obtaining the third image through homography matrix calculation, the method further includes:

[0041] The third image is regionally repaired using a harmonic difference algorithm.

[0042] A second aspect of an embodiment of the present application provides an image defect detection device, comprising:

[0043] an acquisition unit, configured to acquire a first image and a second image according to different light sources and light source positions of the camera and calculate a first image mean, where the first image mean is a mean of all pixel values ​​in the first image;

[0044] A first calculation unit, configured to calculate a correction coefficient matrix based on the first image, the second image, and the mean of the first image;

[0045] a determining unit, configured to determine the coordinate position and coordinate coefficients of dust and the coordinate position and coordinate coefficients of noise in the first image;

[0046] a correction unit, configured to adjust the correction coefficient matrix according to the coordinate positions and coordinate coefficients of the dust and the coordinate positions and coordinate coefficients of the noise points, and correct the first image using the adjusted correction coefficient matrix;

[0047] a second calculation unit, configured to obtain a fitted edge of the display screen in the first image, and obtain a third image by calculating a homography matrix, where the third image is a display area of ​​the display screen in the first image;

[0048] a third computing unit, configured to perform a convolution operation on the third image according to an adaptive spatial filter kernel matrix;

[0049] a processing unit, configured to perform threshold control on the third image after the convolution operation, and filter out a fourth image, where the fourth image is an image portion of the third image that satisfies the threshold control;

[0050] An analysis unit is used to perform feature value analysis on the fourth image to obtain defect features.

[0051] Optionally, the defect detection device further includes:

[0052] A repair unit is configured to perform regional repair on the third image using a harmonic difference algorithm.

[0053] A third aspect of the present application provides an image defect detection device, including:

[0054] processor, memory, input and output units, and buses;

[0055] The processor is connected to the memory, the input and output unit, and the bus;

[0056] A program is stored in the memory, and the processor calls the program to execute the defect detection method in the first aspect and any possible implementation manner of the first aspect.

[0057] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the computer executes the defect detection method in the first aspect and any possible implementation method of the first aspect.

[0058] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0059] By calculating a correction coefficient matrix, the effects of varying light sources and positions on image brightness and contrast are eliminated, ensuring consistent image quality. Dust and noise are removed during the correction process, reducing interference from non-target areas in the first image on subsequent analysis and improving defect detection accuracy. By fitting the display edge and combining homography operations, the original image is converted into a third image containing only the display area, significantly reducing background noise interference.

[0060] Convolution operations are performed on the third image using an adaptive filter kernel, effectively smoothing noise and enhancing defect edges or texture features. Furthermore, the filter kernel can highlight minor defects within the displayed area (such as scratches, bright spots, and dark spots), making them easier to identify during subsequent threshold control. Threshold control is used to filter out the fourth image that meets the requirements, eliminating irrelevant background or non-defect signals and retaining only suspected defect areas, effectively reducing false detections caused by noise or image inhomogeneity. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic flow chart of an embodiment of the image defect detection method in the present application;

[0062] Figure 2 This is a flowchart of an embodiment of a method for obtaining an image and an image mean in the present application;

[0063] Figure 3 A flowchart of an embodiment of a method for determining dust and noise locations in the present application;

[0064] Figure 4 A schematic flow chart of an embodiment of a method for determining dust position coordinates in the present application;

[0065] Figure 5This is a flow chart of an embodiment of a method for determining the position coordinates of noise points in the present application;

[0066] Figure 6 A schematic flow chart of an embodiment of a method for acquiring a third image in the present application;

[0067] Figure 7 A schematic structural diagram of an embodiment of an apparatus for detecting defects in an image in the present application;

[0068] Figure 8 This is a schematic structural diagram of another embodiment of the device for detecting image defects in the present application. DETAILED DESCRIPTION

[0069] The embodiments of the present application provide an image defect detection method, device, and storage medium for detecting screen defects and improving the defect detection rate.

[0070] The method of the present application can be applied to a server, a terminal or other device with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is based on an example in which the execution subject is a server.

[0071] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0072] See also Figure 1 In one embodiment of the present application, a method for detecting defects in an image includes:

[0073] 101. Acquire a first image and a second image according to different light sources and light source positions of the camera and calculate a mean value of the first image, where the first image mean value is the mean value of all pixel values ​​in the first image;

[0074] When capturing images through a camera, the image often appears bright in the center and dark around the edges. Dust and stains on the camera lens also cause the grayscale values ​​of corresponding parts of the image to be inconsistent with the normal display grayscale values. There is also a nonlinear response between the image grayscale value and the energy collected by the device pixels. Therefore, grayscale non-uniformity correction is required for the image.

[0075] Before capturing an image, configure the camera's light source and its position based on the characteristics of the image capture device and the target object. Specifically, you can choose between backlight, frontlight, or shading light sources, and adjust the light source's position on the screen (e.g., center, left, upper left, bottom, etc.).

[0076] After completing the settings of the light source and position, use the same camera to sequentially capture the first image and the second image under different light sources and different positions (the two images correspond to two different light sources, and the different positions are multiple sets of photos taken at different positions under the same light source, that is, multiple shooting positions are the same under two different light sources).

[0077] Due to differences in position, multiple images under the same light source can capture information about the target object from different perspectives. For example, at a certain position under the same light source, the grayscale value of some areas may be low due to shadows, while at another position the grayscale value of the area will be improved. The two complement each other to present a more comprehensive view of the display screen.

[0078] After acquiring the first image, perform a mean calculation on it. Assume the size of the first image is m*n, where m is the number of rows and n is the number of columns. The mean of the first image is obtained by traversing every pixel in the first image, adding up the pixel values, and dividing by the total number of pixels in the image (m*n). This mean reflects the overall grayscale level of the first image and serves as an important reference for subsequent grayscale nonuniformity correction.

[0079] 102. Calculate a correction coefficient matrix based on the first image, the second image, and the mean of the first image;

[0080] The correction coefficient matrix is ​​calculated using the following formula:

[0081]

[0082] Wherein, N is the first image taken under the first light source, H is the second image taken under the second light source, n is the mean of the first image, and K is the correction coefficient matrix.

[0083] The correction coefficient matrix is ​​used to describe the correction relationship of the image under different lighting conditions. By calculating the correction coefficient matrix K, the first image is corrected to eliminate or reduce the impact of lighting changes on image quality.

[0084] 103. Determine the coordinate positions of dust and noise points in the first image;

[0085] In the frequency domain, dust typically appears in an image as isolated or clustered dark or bright spots with sharp edges but random distribution, unrelated to the image content. The grayscale values ​​of dust regions are typically significantly lower or higher than the background and differ significantly from neighboring pixels. Based on the grayscale distribution characteristics of the first image, an appropriate thresholding method is selected to segment the first image. The first image is divided into dust regions and background regions, such that the grayscale values ​​of the pixels in the dust regions are below the threshold, while the grayscale values ​​of the pixels in the background regions are above the threshold. Morphological operations are performed on the dust regions, and contour detection algorithms are used to locate the contours of the dust regions. For each dust region contour, the corresponding centroid coordinates are calculated, which serve as the coordinate position of the dust.

[0086] Noise typically manifests as random variations in grayscale values ​​within an image. The grayscale values ​​of a noise point differ significantly from those of its neighbors, but may exhibit a continuous or discrete distribution depending on the noise type. Noise can be identified by analyzing the local grayscale statistics of an image.

[0087] The gradient filter kernel is used to calculate the grayscale change rate of the first image in different directions. The grayscale mean and variance of each pixel in the first image within a local neighborhood (e.g., a 3x3 or 5x5 window) are also calculated. If the grayscale value of a pixel differs significantly from the grayscale mean of the neighborhood, and the variance of the neighborhood exceeds a certain threshold, the pixel is considered a noise point, and the coordinates of the noise point are obtained.

[0088] 104. Adjust the correction coefficient matrix according to the coordinate positions of the dust and the coordinate positions of the noise points, and correct the first image using the adjusted correction coefficient matrix;

[0089] Correction coefficient matrices are commonly used to perform various image transformations, such as geometric correction and color correction. The correction coefficient matrix is ​​adjusted based on dust and noise information. For example, each pixel in the first image is traversed to determine whether it is dust or noise. If dust or noise is detected, the correction coefficient corresponding to the dust or noise is adjusted accordingly to enhance the correction effect for that area. For pixels that are not dust or noise, their pixel values ​​are directly adjusted based on the correction coefficient corresponding to the pixel in the correction coefficient matrix. The adjustment method is to set the correction coefficient corresponding to dust or noise to 1, so that the correction coefficient matrix can better adapt to the distribution of dust and noise in the image.

[0090] Traverse each pixel in the first image. Scan the first image row by row, from left to right and top to bottom. Represent the grayscale value of each pixel as a vector, and multiply the vector by the correction coefficient matrix to obtain the transformed grayscale value. Ensure that the transformed grayscale value is within the legal range. If it is out of range, truncate or normalize it. Assign the grayscale value obtained after transformation and processing to the pixel at the corresponding position in the first image, ultimately obtaining the corrected first image.

[0091] 105. Obtain a fitted edge of the display screen in the first image, and obtain a third image through homography matrix calculation, where the third image is a display area of ​​the display screen in the first image;

[0092] The first image is captured and positioned to obtain edge points of the display area of ​​the display screen. After the edge points of the display screen are determined, these edge points are fitted using a suitable fitting method.

[0093] The homography matrix describes the mapping relationship between two images. When acquiring the display area, it is necessary to determine the homography matrix from the first image to the display area. First, mark the four vertices of the display's fitted edge in the first image (assuming the display is rectangular). Then, based on the actual display's geometric information (such as its actual size and shape), determine the corresponding four points on the ideal display plane. Using these two sets of corresponding points (the fitted edge corresponding points and the ideal corresponding points), a specific algorithm (such as a direct linear transformation algorithm) is used to calculate the homography matrices M and B. The algorithm solves each element of the homography matrices M and B by establishing a system of linear equations.

[0094] With the homography matrices M and B, for each pixel in the first image, the corresponding point in the third image (the display area) is calculated using the following formula. By traversing all pixels in the first image, performing the above transformation, and rearranging the transformed pixels, we can obtain the third image, which is the display area of ​​the display in the first image.

[0095] The algorithm for calculating the third image is as follows:

[0096]

[0097] Among them, M and B are rotation matrices and translation matrices respectively. is the mapping point pair before image correction, is the mapping point pair after image rectification.

[0098] 106. Perform a convolution operation on the third image according to the adaptive spatial filter kernel matrix;

[0099] First, the size of the filter kernel matrix is ​​set. The filter kernel matrix consists of three parts: the inner layer, the middle layer, and the outer layer. The number of layers and the size of the values ​​are automatically changed when detecting different defect sizes and defect strengths. The purpose is to highlight the locations of areas with large differences in pixel grayscale values ​​within a certain neighborhood, while trying to eliminate the locations of areas with similar grayscale values ​​to the neighborhood.

[0100] The filter kernel matrix constructed in this embodiment is as follows:

[0101]

[0102]

[0103] Among them, i is the number of rows in the matrix whose outer values ​​are all 1, j is the number of rows in the matrix whose middle values ​​are 1 and 0, and k is the number of rows in the matrix whose inner values ​​are 1, 0, and the number of rows, is the center weight of the matrix, i, j and k are all variable values;

[0104] Starting from the top-left pixel of the third image, traverse row by row, pixel by pixel, until reaching the bottom-right pixel. For each pixel, select a local region centered on it with the same size as the filter kernel matrix. Multiply the filter kernel matrix by the corresponding local region of the image element-wise and sum the results to obtain the convolution result. Assign the convolution result for each pixel to the corresponding position in the output image. After traversing all pixels, the image after the adaptive spatial filter convolution operation is obtained.

[0105] 107. Perform threshold control on the third image after the convolution operation to select a fourth image, where the fourth image is an image portion of the third image that meets the threshold control requirement.

[0106] After the convolution operation, the third image is traversed and the number of pixels at each grayscale level is counted to generate a grayscale histogram. This histogram provides an intuitive understanding of the image's grayscale distribution. The threshold is manually set based on the application scenario of the third image and human experience.

[0107] Traverse each pixel of the third image and compare the grayscale value of each pixel with the determined threshold. Perform threshold control according to the following formula:

[0108]

[0109] in is the lower limit of the gray value. is the upper limit of the grayscale value set, is the pixel grayscale value of the pixel in the third image;

[0110] If the pixel grayscale value meets the threshold condition (within the threshold range), the pixel is marked as a valid pixel and its position and grayscale value in the image are retained. If the pixel grayscale value does not meet the threshold condition (greater than the threshold or less than the threshold), the pixel is marked as an invalid pixel. Based on the pixel classification results, a temporary image is generated, in which only the pixels that meet the threshold condition are retained, and the pixels that do not meet the condition are processed.

[0111] Connected domain analysis is performed on the temporary image to identify interconnected pixel regions within the image. By marking different connected domains, different objects or regions within the image can be distinguished. Based on actual needs, an area threshold is set and connected domains with an area smaller than the threshold are removed from the temporary image. These small areas are often noise or irrelevant minor details. After connected domain analysis and removal of small areas, the final fourth image is obtained. The fourth image is the portion of the third image that meets the threshold.

[0112] 108. Perform eigenvalue analysis on the fourth image to obtain defect features.

[0113] Performing eigenvalue analysis on the obtained fourth image, using optional feature quantities such as grayscale, contrast, and grayscale co-occurrence matrix correlation, to ultimately obtain defects on the display screen;

[0114] Taking the grayscale feature as an example, calculate the eigenvalue:

[0115]

[0116] Where: i, j are the grayscale values ​​of the image, and p(i, j) is the probability of the grayscale (i, j) appearing in the specified direction;

[0117] According to the formula, the Con value is calculated, which reflects the degree of grayscale variation in the image. During the calculation process, each element in the grayscale co-occurrence matrix is ​​traversed, the product of the square of the grayscale difference between pixel pairs and the probability of occurrence is calculated, and these products are summed to obtain the Con value. By analyzing the Con value, defective areas in the image can be identified. Defective areas will show abnormally high Con values. By setting an appropriate threshold and marking areas exceeding the threshold as defective areas, the defect characteristics in the image can be obtained.

[0118] In this embodiment, the effects of different light sources and positions on image brightness and contrast are eliminated by calculating a correction coefficient matrix. During the correction process, edge points and noise are removed, reducing the interference of non-target areas in the first image on subsequent analysis. By fitting the display edge and combining homography matrix operations, the original image is converted into a third image containing only the display area, significantly reducing the interference of background noise.

[0119] Convolution operations are performed on the third image using an adaptive filter kernel, effectively smoothing noise and enhancing defect edges or texture features. Furthermore, the filter kernel can highlight minor defects within the displayed area (such as scratches, bright spots, and dark spots), making them easier to identify during subsequent threshold control. Threshold control is used to filter out the fourth image that meets the requirements, eliminating irrelevant background or non-defect signals and retaining only suspected defect areas, effectively reducing false detections caused by noise or image inhomogeneity.

[0120] See also Figure 2 In the embodiment of the present application, an embodiment of the method for obtaining an image and an image mean value includes:

[0121] 201. Setting light sources at different positions to take photos of the display screen, collecting a first set of photos, and superimposing the first set of photos to obtain a first image;

[0122] Based on the size and shape of the display, as well as the testing requirements, choose a variety of different locations for light sources. These locations should cover the typical angles and distances at which the display might be illuminated. For example, light sources can be placed in front of the display, to the left, to the right, above, or below it. Each location can also have different distances, such as close (0.5 meters), medium (1 meter), and long (2 meters).

[0123] At each preset light source position, a camera is used to take a picture of the display screen. All pictures taken at different light source positions are combined into a first set of pictures. This first set of pictures is then overlaid using pixel value averaging. That is, for pixels at the same position in each picture, their pixel values ​​are added together and then divided by the total number of pictures to obtain the pixel value at the corresponding position in the overlaid first image.

[0124] 202. After performing light shielding processing on the camera lens, take a picture of the display screen, collect a second set of pictures, and superimpose the second set of pictures to obtain a second image;

[0125] Use an opaque material, such as a black lens hood or blackout cloth, to tightly shield the camera lens from light, ensuring that no external light can enter the camera lens. After the lens shading process is complete, maintain the camera parameters consistent with step 201 and take a photo of the display screen. All the captured photos are combined into a second set of photos. The second set of photos is then overlaid using the same overlay method as the first set of photos to produce a second image.

[0126] 203. Perform statistical calculation on the pixel values ​​of the first image to obtain a first image mean.

[0127] The first image is read, and each pixel of the first image is traversed row by row and column by column to obtain its pixel value. Statistical calculations are performed on all the extracted pixel values ​​to calculate the average of all pixel values, i.e., the first image mean.

[0128] In this embodiment, by taking photos from different light source positions and superimposing them, it is possible to integrate lighting information from various angles and directions, effectively eliminating the uneven lighting that may occur from a single light source position, making the lighting in the first image more uniform, and thus improving the accuracy of subsequent defect detection. Superimposing multiple photos can reduce the impact of random noise. Because random noise is randomly distributed across different photos, the superposition operation can offset the noise, improving the image's signal-to-noise ratio, making the image clearer and facilitating defect identification and analysis. The first image mean reflects the overall brightness level of the first image, providing data support for subsequent image processing and defect detection.

[0129] See also Figure 3 In one embodiment of the present application, a method for determining the location of dust and noise includes:

[0130] 301. Determine the coordinate position of dust in the first image based on the amplitude and direction of the first image;

[0131] The first image is filtered to obtain a gradient magnitude and direction of the first image, and the coordinate position of dust in the first image is determined according to the gradient magnitude and gradient direction of the first image;

[0132] The horizontal filter kernel is:

[0133] The vertical filter kernel is:

[0134] For each pixel in the first image, a filter kernel is used to perform a weighted summation of the pixel and its neighborhood to obtain the horizontal and vertical gradient values ​​for that pixel. Based on the horizontal and vertical gradient values, the gradient magnitude and gradient direction of each pixel are calculated.

[0135] 302. Determine the coordinate position of the noise point in the first image by comparing the vector diagram mapping of the first image.

[0136] A vector diagram is a graphical representation that describes image content based on mathematical formulas. Converting a first image into a vector diagram establishes a mapping between the first image and its vector diagram. This means that for every pixel in the first image, a corresponding location or element can be found in the vector diagram.

[0137] Based on the mapping, the first image and the vector map are compared point by point or region by region. During the comparison, pixels with significantly different positions from the corresponding positions in the vector map are likely to be noise points. By comparing and analyzing the entire image, the coordinates of the pixels identified as noise points are recorded, thereby obtaining the coordinate position information of the noise points in the first image.

[0138] In this embodiment, dust is located by analyzing amplitude and direction, avoiding the difficulty of identifying dust due to its similar color and shape to the background, thereby improving dust location accuracy. A vector map is used to highlight and locate noise points by comparing the map with the original image, thus avoiding misidentification of normal image details as noise points and improving noise detection accuracy. Determining the coordinates of dust and noise points provides precise location information for subsequent processing operations.

[0139] See also Figure 4 In one embodiment of the present application, a method for determining dust position coordinates includes:

[0140] 401. Perform smoothing filtering on the first image;

[0141] Smoothing filtering is an image processing technology used to reduce image noise and make the image smoother. It reduces the sudden change of pixel values ​​by performing a weighted average operation on each pixel point and its neighboring pixels in the first image, thereby achieving the effect of suppressing noise and smoothing the image.

[0142] Select the appropriate filter kernel size based on the image noise and actual needs. Starting from the upper left corner of the image, traverse each pixel in the image row by row and column by column.

[0143] The first image is filtered according to the following formula:

[0144]

[0145] in, is a pixel in the first image, is the pixel point after mean filtering, It is a filter kernel of size M*N.

[0146] The calculated grayscale value G(x,y) is assigned to the pixel with coordinates (x,y) in the image to complete the smoothing filter processing of the pixel. Repeat the above steps until all pixels in the image are traversed to obtain the complete image after smoothing filter processing.

[0147] 402. Obtain a gradient magnitude and a gradient direction of the first image through a gradient filter kernel.

[0148] The gradient filter kernel contains the horizontal direction and vertical direction The first image after smoothing filtering is processed with two filter kernels. Filter kernel and The filter kernel performs a convolution operation to obtain the gradient components of the image in the horizontal and vertical directions. For each pixel in the image, the result of the convolution operation is the sum of the products of the pixel values ​​in the neighborhood of that pixel and the corresponding element of the convolution kernel.

[0149] 403 . For each pixel in the first image, along the gradient direction of the pixel, compare the gradient amplitude of the pixel with the gradient amplitudes of the pixels on both sides of the gradient direction to determine dust in the first image.

[0150] For each pixel in the first image, its corresponding discrete direction is determined based on its gradient direction. Then, along the discrete direction, the adjacent pixels on both sides of the gradient direction of the pixel are found, and the gradient magnitude of the pixel is compared with the gradient magnitudes of the adjacent pixels on both sides.

[0151] Set two thresholds: a low threshold and a high threshold. For the image obtained after the above comparison, pixels with gradient magnitudes greater than the high threshold are identified as dust spots. Pixels with gradient magnitudes less than the low threshold are discarded. Pixels with gradient magnitudes between the low and high thresholds are discarded. Finally, the coordinates of the dust spots (i.e., the dust spots in the first image) are recorded.

[0152] In this embodiment, smoothing filtering can suppress noise in the image, making the first image smoother, reducing the generation of false edges, and making edge detection more accurate. Furthermore, smoothing filtering can make the edges in the image more continuous, facilitating subsequent edge analysis and processing.

[0153] By calculating the gradient magnitude and direction, dust particles can be extracted from an image. By comparing the gradient magnitude of a pixel with the gradient magnitudes of pixels on either side of the gradient, dust particles are more clearly identified. By setting high and low thresholds, true dust particles can be distinguished from noise and other interference, improving the reliability of edge detection. Only pixels with large gradient magnitudes are retained as dust particles, reducing the possibility of false positives.

[0154] See also Figure 5 In one embodiment of the present application, a method for determining the position coordinates of a noise point includes:

[0155] 501. Generate two completely black images with the same size as the first image;

[0156] Create two new images based on the width and height of the first image. Initialize the grayscale values ​​of all pixels in the images to 0, setting them all to black. This ensures that the two images are exactly the same size as the first image, providing a foundation for subsequent operations.

[0157] 502. Set the row coordinate value and the column coordinate value of the all-black image to the grayscale values ​​of the two all-black images respectively;

[0158] Traverse each pixel of the two completely black images. For the first completely black image, the row coordinate value is used as the grayscale value of the pixel; for the second completely black image, the column coordinate value is used as the grayscale value of the pixel. For example, the pixel with coordinates (i, j) has the grayscale value i in the first completely black image and the grayscale value j in the second completely black image.

[0159] 503. Calculate the row and column coordinate array of the neighborhood of each pixel in the completely black image, taking each pixel as the center;

[0160] For each pixel in the completely black image, use it as the center and determine the neighborhood of the central pixel. The neighborhood can be a square, rectangle, or other shaped area centered on the pixel. Typically, a 3×3, 5×5, or other sized square neighborhood is chosen.

[0161] After defining a neighborhood of appropriate size, for each pixel, loop through the calculations to obtain the row and column coordinates of all pixels within its neighborhood. Taking a 3×3 neighborhood as an example, for a pixel with coordinates (0, 0), its neighborhood row and column coordinate array contains (−1,−1), (−1,0), (−1,1), (0,−1), (0,0), (0,1), (1,−1), (1,0), (1,1). Convert the neighborhood row and column coordinate array into the following format:

[0162] [-1,-1,-1,0,0,0,1,1,1],[-1,-0,1,-1,0,1,-1,0,1]

[0163] or, ,

[0164] 504. Perform arithmetic operations on the grayscale values ​​according to the order of the neighborhood row and column coordinate arrays, and perform grayscale threshold verification;

[0165] For each pixel and its neighborhood, arithmetic operations are performed on the grayscale values ​​of the pixels in the neighborhood according to the order of the neighborhood row and column coordinate arrays, and operations such as summation and averaging can be performed.

[0166] Set an appropriate grayscale threshold, limiting the image's grayscale values ​​to within the width and height ranges of 0 to (Height - 1) and 0 to (Width - 1). Compare the result of the arithmetic operation with the threshold to determine whether the conditions are met. If the result is within the threshold, record the relevant information for that pixel.

[0167] 505. Take the intersection of image regions that meet the grayscale threshold condition in the completely black image, retain the corresponding intersection region from the first image, and convert the intersection region into a vector image;

[0168] Analyze the distribution of pixels that meet the grayscale threshold condition in two completely black images and find the area of ​​overlap, also known as the intersection. For example, if the set of pixels that meet the condition in the first image is A, and the set of pixels that meet the condition in the second image is B, find the intersection of A and B: A∩B.

[0169] Based on the coordinates of the intersection area, the corresponding area in the first image is found and retained. Then, using an image vectorization tool, the retained intersection area is converted into a vector image. Vector images have the advantages of being resolution-independent, making them easier to process and analyze.

[0170] 506. Map the first image and the vector image to obtain multiple mapped images centered on the pixel points;

[0171] Taking each pixel in the vector image as the center, the corresponding area in the first image is determined to generate multiple mapping images. Multiple such mapping images are obtained by traversing all the pixels in the vector image.

[0172] 507. Perform arithmetic operations on the first image and the multiple mapped images respectively, and determine the coordinate position of the noise point according to the output results, where the output results include the grayscale values ​​of the same position in the multiple mapped images.

[0173] Perform arithmetic operations, such as summing, averaging, and calculating variance, on the grayscale values ​​of pixels at the same location in the multiple mapped images and the first image. Based on the results of these operations and the set judgment criteria, noise points are identified. If the variance at a location is greater than a preset threshold, it indicates significant grayscale fluctuations at that location, suggesting the presence of noise, and is marked as such. Otherwise, the pixel is considered normal. Finally, the coordinates of the noise point are recorded.

[0174] In this embodiment, through a series of grayscale value calculations, threshold checks, and region intersection operations, the relevant regions of noise points can be accurately identified from the first image. Converting the relevant regions into vector images allows for accurate local image processing, improving both precision and effectiveness. By performing arithmetic operations on multiple mapped images, noise points can be effectively distinguished from normal pixels based on their grayscale distribution characteristics, and the coordinates of the noise points can be determined, reducing the possibility of misidentification and improving image quality.

[0175] See also Figure 6 In one embodiment of the present application, a method for obtaining a third image includes:

[0176] 601. Construct a plurality of rectangular frames at the edge of the display screen in the first image;

[0177] Determine the size and number of rectangular frames based on the display's edge. If the display's edge is regular, use rectangular frames of the same size. If the edge is irregular, use rectangular frames of different sizes in areas with varying curvatures. The number of rectangular frames should be sufficient to cover the display's edge, but not too densely packed to create excessive computational overhead.

[0178] 602. Traverse the grayscale values ​​of the pixels within each rectangular frame and calculate the grayscale difference between each pixel and its adjacent pixels;

[0179] For each constructed rectangular box, the image traversal algorithm is used to visit each pixel in the box row by row and column by column, starting from the upper left corner pixel of the rectangular box.

[0180] Adjacent pixels are those that are directly adjacent horizontally and vertically (i.e., adjacent above, below, left, or right). They may also include diagonally adjacent pixels. For each traversed pixel, its own grayscale value and the grayscale values ​​of its adjacent pixels are obtained, and the grayscale difference between them is calculated through subtraction. For example, for pixel P(i,j) and its horizontally adjacent pixel P(i+1,j), the grayscale difference is |P(i,j)-P(i+1,j)|.

[0181] 603. Filtering edge pixel coordinates of the display screen in the first image based on the grayscale difference;

[0182] Based on the characteristics of the first image and user experience, set an appropriate grayscale difference threshold to determine whether a pixel is at the edge of the display. Calculate the grayscale difference within each rectangular box and consider any pixels with a grayscale difference greater than the threshold to be at the edge of the display. Record the coordinates of these pixels.

[0183] 604. Fitting the edge pixel coordinates to the fitted edge of the display screen in the first image, and calculating the intersection of the fitted edge, where the intersection of the fitted edge is the edge corner point of the display screen in the first image;

[0184] For rectangular displays, a line fitting algorithm can be used to fit the selected edge pixel coordinates into a continuous curve or straight line, obtaining the fitted edge of the display. The principle of least squares line fitting is to determine the line parameters by minimizing the sum of the squares of the perpendicular distances from the edge points to the fitted line. By fitting the four edges, a more accurate and smooth representation of the display edge can be obtained.

[0185] For the multiple edges obtained by fitting, the intersection points between the edges are calculated by solving the equation group according to the equation of the straight line or curve. These intersection points are the edge corner points of the display screen.

[0186] 605. Correct the fitted edges and intersections using a homography matrix to obtain a third image.

[0187] Based on the calculated homography, the coordinates of the fitted edges and intersections are transformed, correcting the edges and corners of the display to the correct position and shape, resulting in a third image. Specifically, the transformation formula of the homography is applied to the coordinates of each edge pixel and intersection point, updating their coordinate values ​​to obtain the corrected image.

[0188] 606. Perform regional repair on the third image using a harmonic difference algorithm.

[0189] In the actual process, after the second image is corrected by the homography matrix to obtain the third image in step 605, it is still necessary to deal with the non-integer problem that may appear in the pixels of the third image. An interpolation algorithm is usually used to determine the grayscale value of the pixel to ensure the quality of the image.

[0190] First, we need to identify the image portion that needs to be repaired, specifically the background area around the corners and the dust filter, which may cause interference. We then identify appropriate known pixels around the area to be repaired. These known pixels are evenly distributed around the area to be repaired.

[0191] Perform regional repair on the part of the third image that needs to be repaired using the harmonic interpolation algorithm:

[0192]

[0193] in, is the grayscale of the pixel to be repaired in the third image, is the grayscale of the known pixels around the point to be repaired, is the distance between two pixels;

[0194] Let a known pixel be a and the pixel to be repaired be b. Calculate the distance (ba) between each known pixel a and the pixel to be repaired b. Determine the weight of each known pixel based on the distance; generally, closer distances give higher weights.

[0195] For each pixel b in the area to be repaired, a weighted average is performed based on the weights of the multiple known pixels a to obtain the final grayscale value of the pixel b to be repaired. Repeat this step for all pixels in the entire area to be repaired, performing grayscale estimation point by point until the entire area is repaired, resulting in the repaired third image.

[0196] In this embodiment, by constructing a rectangular frame around the edge of the display and calculating pixel grayscale differences, edge pixels can be screened, reducing the influence of interference factors such as background, and improving the accuracy of edge pixel positioning. Using a fitting algorithm and intersection calculation method, the fitted edges and corners of the display can be accurately obtained. Correcting the fitted edges and intersections using a homography matrix corrects for any geometric distortion in the display image, resulting in a more realistic third image and improved image quality.

[0197] Performing regional repair on the third image through the harmonic difference algorithm can effectively fill in the defects, damage or incomplete parts in the image, making the image more visually complete and continuous, and avoiding the impact of missing or abnormal local areas on the analysis and understanding of the entire display image.

[0198] See also Figure 7 The present application provides an embodiment of an image defect detection device, comprising:

[0199] An acquisition unit 701 is configured to acquire a first image and a second image according to different light sources and light source positions of the camera and calculate a first image mean, where the first image mean is a mean of all pixel values ​​in the first image;

[0200] A first calculation unit 702 is configured to calculate a correction coefficient matrix based on the first image, the second image, and the mean of the first image;

[0201] The determining unit 703 is configured to determine the coordinate position and coordinate coefficients of dust and the coordinate position and coordinate coefficients of noise in the first image;

[0202] The correction unit 704 is configured to adjust a correction coefficient matrix according to the coordinate positions and coordinate coefficients of the dust and the coordinate positions and coordinate coefficients of the noise points, and correct the first image using the adjusted correction coefficient matrix;

[0203] The second calculation unit 705 is configured to obtain a fitted edge of the display screen in the first image, and obtain a third image by calculating a homography matrix, where the third image is a display area of ​​the display screen in the first image;

[0204] A third computing unit 706 is configured to perform a convolution operation on the third image according to the adaptive spatial filter kernel matrix;

[0205] The processing unit 707 is configured to perform threshold control on the third image after the convolution operation, and select a fourth image, where the fourth image is an image portion of the third image that meets the threshold control requirement.

[0206] The analyzing unit 708 is configured to perform feature value analysis on the fourth image to obtain defect features.

[0207] Optionally, the defect detection device further includes:

[0208] The repair unit 709 performs regional repair on the third image by using a harmonic difference algorithm.

[0209] In this implementation, the functions of each unit and module are the same as those mentioned above. Figures 1 to 6 The steps in the illustrated embodiment correspond to each other and will not be repeated here.

[0210] See also Figure 8 Another embodiment of an image defect detection device in the embodiment of the present application includes:

[0211] Processor 801, memory 802, input and output unit 803 and bus 804;

[0212] The processor 801 is connected to the memory 802, the input and output unit 803 and the bus 804;

[0213] The memory 802 stores a program, and the processor 801 calls the program to execute Figures 1 to 6 Steps in the illustrated embodiment.

[0214] In this embodiment, the function of the processor 801 is the same as that of the aforementioned Figures 1 to 6 The steps in the illustrated embodiment correspond to each other and will not be repeated here.

[0215] The embodiment of the present application further provides a computer-readable storage medium having a program stored thereon, which, when executed on a computer, causes the computer to execute the aforementioned Figures 1 to 6 A defect detection method in any possible embodiment.

[0216] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0217] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0218] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0219] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0220] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A method for detecting defects in an image, characterized in that: include: According to different light sources and light source positions of the camera, a first image and a second image are acquired and a first image mean is calculated, where the first image mean is the mean of all pixel values ​​in the first image; Calculating a correction coefficient matrix according to the first image, the second image, and the mean of the first image; Determining the coordinate positions of dust and noise points in the first image; adjusting the correction coefficient matrix according to the coordinate positions of the dust and the coordinate positions of the noise points, and correcting the first image using the adjusted correction coefficient matrix; Obtaining a fitted edge of the display screen in the first image, and obtaining a third image by calculating a homography matrix, where the third image is a display area of ​​the display screen in the first image; performing a convolution operation on the third image according to an adaptive spatial filter kernel matrix; Performing a threshold control on the third image after the convolution operation to filter out a fourth image, where the fourth image is an image portion of the third image that meets the threshold control requirement; Performing feature value analysis on the fourth image to obtain defect features.

2. The defect detection method according to claim 1, characterized in that: The method of acquiring the first image and the second image and calculating the mean of the first image according to different light sources and light source positions of the camera includes: Setting the light source at different positions to take photos of the display screen, collecting a first set of photos and superimposing the first set of photos to obtain a first image; After shielding the camera lens, take photos of the display screen, collect a second set of photos, and superimpose the second set of photos to obtain a second image; Statistical calculation is performed on the pixel values ​​of the first image to obtain a first image mean.

3. The defect detection method according to claim 2, characterized in that: The determining the coordinate positions of dust and noise points in the first image includes: determining the coordinate position of dust in the first image according to the amplitude and direction of the first image; The coordinate positions of the noise points in the first image are determined by comparing the vector diagram mapping of the first image.

4. The defect detection method according to claim 3, characterized in that: Determining the coordinate position of dust in the first image according to the amplitude and direction of the first image includes: performing smoothing filtering on the first image; Passing a gradient filter kernel through the first image to obtain a gradient magnitude and a gradient direction of the first image; For each pixel point in the first image, along the gradient direction of the pixel point, the gradient amplitude of the pixel point is compared with the gradient amplitudes of the pixels on both sides of the gradient direction to determine the coordinate position of the dust in the first image.

5. The defect detection method according to claim 4, characterized in that: The determining the coordinate position of the noise point in the first image by comparing the vector diagram mapping of the first image includes: Generate two completely black images with the same size as the first image; The row coordinate values ​​and column coordinate values ​​of the completely black image are respectively set to the grayscale values ​​of the two completely black images; Taking each pixel in the completely black image as the center, calculate the neighborhood row and column coordinate array of each pixel; Perform arithmetic operations on the grayscale values ​​according to the order of the neighborhood row and column coordinate arrays, and perform grayscale threshold verification; Taking the intersection of image regions that meet the grayscale threshold condition in the completely black image, retaining the corresponding intersection region from the first image, and converting the intersection region into a vector image; Mapping the first image with the vector image to obtain a plurality of mapped images centered on pixel points; Performing arithmetic operations on the plurality of mapped images and the first image respectively, and determining the coordinate position of the noise point according to the output results, wherein the output results include the grayscale values ​​of the same position in the plurality of mapped images.

6. The defect detection method according to any one of claims 1 to 5, characterized in that: The obtaining of the fitted edge of the display screen in the first image and obtaining the third image by calculating the homography matrix includes: constructing a plurality of rectangular frames at the edge of the display screen in the first image; Traverse the grayscale values ​​of the pixels in each rectangular frame and calculate the grayscale difference between each pixel and its adjacent pixels; Filtering the edge pixel coordinates of the display screen in the first image according to the grayscale difference; Fitting the edge pixel coordinates to the fitted edges of the display screen in the first image, and calculating the intersection of the fitted edges, where the intersection of the fitted edges is the edge corner point of the display screen in the first image; The fitted edges and intersections are corrected using a homography matrix to obtain a third image.

7. The defect detection method according to any one of claims 1 to 5, characterized in that: After obtaining the fitted edge of the display screen in the first image and obtaining the third image through homography matrix calculation, the method further includes: The third image is regionally repaired using a harmonic difference algorithm.

8. An image defect detection device, characterized in that: include: an acquisition unit, configured to acquire a first image and a second image according to different light sources and light source positions of the camera and calculate a first image mean, where the first image mean is a mean of all pixel values ​​in the first image; A first calculation unit, configured to calculate a correction coefficient matrix based on the first image, the second image, and the mean of the first image; a determining unit, configured to determine the coordinate positions of dust and noise points in the first image; a correction unit, configured to adjust the correction coefficient matrix according to the coordinate positions of the dust and the coordinate positions of the noise points, and correct the first image using the adjusted correction coefficient matrix; a second calculation unit, configured to obtain a fitted edge of the display screen in the first image, and obtain a third image by calculating a homography matrix, where the third image is a display area of ​​the display screen in the first image; a third computing unit, configured to perform a convolution operation on the third image according to an adaptive spatial filter kernel matrix; a processing unit, configured to perform threshold control on the third image after the convolution operation, and filter out a fourth image, where the fourth image is an image portion of the third image that satisfies the threshold control; An analysis unit is used to perform feature value analysis on the fourth image to obtain defect features.

9. An image defect detection device, characterized in that: include: processor, memory, input and output units, and buses; The processor is connected to the memory, the input and output unit, and the bus; A program is stored in the memory, and the processor calls the program to execute the defect detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the defect detection method according to any one of claims 1 to 7.

Images