Image defect detection method and device and storage medium

Abstract

The embodiment of the invention discloses an image defect detection method and device and a storage medium, which are used for detecting screen defects and improving the defect detection rate. The method comprises the following steps: acquiring a first image and a second image according to different light sources and light source positions of a camera, and calculating a first image mean value; calculating a correction coefficient matrix according to the first image, the second image and the first image mean value; determining a coordinate position of dust and a coordinate position of a noisy point in the first image; adjusting a correction coefficient matrix according to the coordinate position of the dust and the coordinate position of the noise point, and correcting the first image through the adjusted correction coefficient matrix; obtaining a fitting edge of the display screen in the first image, and calculating through a homography matrix to obtain a third image; performing convolution operation according to the adaptive spatial filtering kernel matrix and the third image; performing threshold value control on the third image after the convolution operation is completed, and screening out a fourth image; and performing characteristic value analysis on the fourth image to obtain defect characteristics.

Description

Technical Field

[0001] The present application relates to the technical field of defect detection, and particularly relates to a method, device and storage medium for defect detection of images. Background Art

[0002] With the progress of technology, display technologies are constantly iterating. New display technologies have higher resolution, color performance and response speed, but at the same time, they also pose challenges to existing automated detection methods.

[0003] During the production process of a display screen, various types of defects are likely to occur due to factors such as process equipment, manual labor and environment, such as bright spots, dark spots, line defects, color spots, etc. Moreover, the types of defects on the display screen are rich and diverse, bringing great challenges to the detection work. Due to the complexity of defect types, common contrast enhancement algorithms are extremely vulnerable to the interference of different types of defect features in practical applications and are difficult to effectively distinguish defects.

[0004] In addition, the defects that appear on the surface of the display screen usually have extremely small sizes. When existing networks perform forward propagation, in order to reduce the computational amount and expand the receptive field, strided convolutional layers or pooling layers are frequently used for downsampling operations. However, this process will inevitably lead to a large loss of fine-grained feature information of defect targets. Especially when processing low-resolution images, the information carried by the image itself is relatively limited, and the downsampling operation further exacerbates the information loss. When the size of the defect target is extremely small, its own fine-grained features almost disappear after multiple downsamplings, ultimately resulting in the detection accuracy being difficult to reach an ideal level and unable to meet the strict requirements of industrial production for high-precision detection. Summary of the Invention

[0005] Embodiments of the present application provide a method, device and storage medium for defect detection of images, which are used to detect screen defects and improve the defect detection rate.

[0006] The first aspect of the embodiments of the present application provides a method for defect detection of images, including: Obtain a first image and a second image according to different light sources and light source positions of a camera, and calculate the mean value of the first image, where the mean value of the first image is the mean value of all pixel values in the first image; Calculate a correction coefficient matrix based on the first image, the second image and the mean value of the first image; Determine the coordinate positions of dust and noise points in the first image; Adjust the correction coefficient matrix according to the coordinate positions of the dust and the noise points, and correct the first image through the adjusted correction coefficient matrix; Obtain the fitted edge of the display screen in the first image, and calculate the third image through the homography matrix, where the third image is the display area of the display screen in the first image; Perform convolution operation on the third image according to the adaptive spatial filtering kernel matrix; Perform threshold control on the third image after the convolution operation is completed, and filter out the fourth image, where the fourth image is the image part in the third image that meets the threshold control; Perform eigenvalue analysis on the fourth image to obtain defect features.

[0007] Optionally, the obtaining the first image and the second image according to different light sources and light source positions of the camera and calculating the average value of the first image includes: Set the light source at different positions to take pictures of the display screen, summarize the first group of pictures and superimpose the first group of pictures to obtain the first image; After shading the lens of the camera, take pictures of the display screen, summarize the second group of pictures and superimpose the second group of pictures to obtain the second image; Perform statistical calculation on the pixel values of the first image to obtain the average value of the first image.

[0008] Optionally, the determining the coordinate positions of dust and the coordinate positions of noise points in the first image includes: Determine the coordinate positions of dust in the first image through the amplitude and direction of the first image; Determine the coordinate positions of noise points in the first image through vector map mapping comparison of the first image.

[0009] Optionally, the determining the coordinate positions of dust in the first image through the amplitude and direction of the first image includes: Perform smoothing filtering on the first image; The first image obtains the gradient amplitude and gradient direction of the first image through the gradient filtering kernel; For each pixel point in the first image, along the gradient direction of the pixel point, compare the gradient amplitude of the pixel point with the gradient amplitudes of the pixel points on both sides of the gradient direction to determine the coordinate positions of dust in the first image.

[0010] Optionally, the determining the coordinate positions of noise points in the first image through vector map mapping comparison of the first image includes: Generate two all-black images with the same size as the first image; Set the row coordinate values and column coordinate values of the all-black images as the gray values of the two all-black images respectively; Taking each pixel point in the all - black image as the center, calculate the neighborhood row - column coordinate array of each pixel point; Perform arithmetic operations on the gray - scale values in the order of the neighborhood row - column coordinate array, and perform gray - scale threshold verification; Take the intersection of the image regions in the all - black image that meet the gray - scale threshold condition, retain the corresponding intersection region from the first image, and convert the intersection region into a vector image; Map the first image and the vector image to obtain multiple mapped images centered on pixel points; Perform arithmetic operations on each of the multiple mapped images and the first image respectively, and determine the coordinate positions of noise points according to the output results, where the output results include the gray - scale values at the same positions in the multiple mapped images.

[0011] Optionally, the obtaining of the fitted edge of the display screen in the first image and calculating the third image through a homography matrix includes: Construct multiple rectangular frames on the edge of the display screen in the first image; Traverse the gray - scale values of pixel points in each rectangular frame, and calculate the gray - scale difference between each pixel point and its adjacent pixel points; According to the gray - scale difference, screen out the coordinate positions of the edge pixel points of the display screen in the first image; Fit the coordinate positions of the edge pixel points to the fitted edge of the display screen in the first image, and calculate the intersection points of the fitted edge, where the intersection points of the fitted edge are the edge corner points of the display screen in the first image; Correct the fitted edge and intersection points through a homography matrix to obtain the third image.

[0012] Optionally, after obtaining the fitted edge of the display screen in the first image and calculating the third image through a homography matrix, it further includes: Perform regional patching on the third image through a harmonic difference algorithm.

[0013] The second aspect of the embodiments of the present application provides an image defect detection device, including: An acquisition unit, configured to obtain a first image and a second image according to different light sources and light source positions of a camera, and calculate the first - image mean value, where the first - image mean value is the mean value of all pixel values in the first image; A first calculation unit, configured to calculate a correction coefficient matrix according to the first image, the second image, and the first - image mean value; A determination unit, configured to determine the coordinate positions and coordinate coefficients of dust in the first image and the coordinate positions and coordinate coefficients of noise points; A correction unit, configured to adjust the correction coefficient matrix according to the coordinate positions and coordinate coefficients of the dust and the coordinate positions and coordinate coefficients of the noise points, and correct the first image by using the adjusted correction coefficient matrix; A second calculation unit, configured to obtain a fitted edge of a display screen in the first image, and calculate a third image through a homography matrix, where the third image is a display area of the display screen in the first image; A third calculation unit, configured to perform a convolution operation on the third image according to an adaptive spatial filtering kernel matrix; A processing unit, configured to perform threshold control on the third image after the convolution operation is completed, and screen out a fourth image, where the fourth image is an image part in the third image that meets the threshold control; An analysis unit, configured to perform eigenvalue analysis on the fourth image to obtain defect features.

[0014] Optionally, the defect detection device further includes: A patching unit, configured to perform regional patching on the third image through a harmonic difference algorithm.

[0015] A third aspect of an embodiment of the present application provides a defect detection device for an image, including: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the defect detection method in the first aspect and any possible implementation manner of the first aspect.

[0016] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, where a program is stored on the computer-readable storage medium, and when the program is executed on a computer, the computer is caused to execute the defect detection method in the first aspect and any possible implementation manner of the first aspect.

[0017] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages: By calculating the correction coefficient matrix, the influence of different light sources and positions on the image brightness and contrast is eliminated, ensuring the consistency of the image quality. During the correction process, dust and noise points are removed, reducing the interference of non-target areas in the first image on subsequent analysis and improving the accuracy of defect detection. By fitting the edge of the display screen and combining the homography matrix operation, the original image is converted into a third image that only contains the display area of the display screen, significantly reducing the interference of background noise.

[0018] Construct an adaptive filter kernel to perform convolution operation on the third image, which can effectively smooth the noise, enhance the defect edges or texture features. Moreover, the filter kernel can highlight the tiny defects (such as scratches, bright spots, dark spots) within the region, making them more easily recognizable in the subsequent threshold control. By screening out the fourth image that meets the conditions through threshold control, the irrelevant background or non-defect signals can be eliminated, and only the suspected defect regions are retained, effectively reducing the false detection caused by noise or image non-uniformity. Description of the Drawings

[0019] Figure 1 It is a schematic flowchart of an embodiment of the method for defect detection of images in this application; Figure 2 It is a schematic flowchart of an embodiment of the method for obtaining an image and the image mean in this application; Figure 3 It is a schematic flowchart of an embodiment of the method for determining the positions of dust and noise in this application; Figure 4 It is a schematic flowchart of an embodiment of the method for determining the dust position coordinates in this application; Figure 5 It is a schematic flowchart of an embodiment of the method for determining the noise position coordinates in this application; Figure 6 It is a schematic flowchart of an embodiment of the method for obtaining the third image in this application; Figure 7 It is a schematic structural diagram of an embodiment of the device for defect detection of images in this application; Figure 8 It is a schematic structural diagram of another embodiment of the device for defect detection of images in this application. Detailed Description of the Embodiment

[0020] The embodiments of this application provide a method, device and storage medium for defect detection of images, which are used to detect screen defects and improve the defect detection rate.

[0021] The method of this application can be applied to servers, terminals or other devices with logical processing capabilities. In this regard, this application makes no limitation. For the sake of convenience of description, the following will describe it by taking the execution entity as a server as an example.

[0022] The following will describe the embodiments in this application with reference to the drawings.

[0023] Please refer to Figure 1 , an embodiment of the method for defect detection of images in the embodiments of this application includes: 101. Obtain a first image and a second image according to different light sources and light source positions of the camera, and calculate the first image mean, where the first image mean is the mean of all pixel values in the first image; When collecting images through a camera, the images often appear brighter in the middle and darker around the edges. At the same time, dust and stains on the camera lens also cause the gray values of the corresponding parts of the image to be inconsistent with the normal display gray values, and there is also a non-linear response between the gray values of the image and the energy collected by the device pixels. Therefore, it is necessary to correct the uneven grayness of the image; Before image acquisition, configure the light source and its position of the camera according to the characteristics of the image acquisition device and the target object. Specifically, the light source type of the camera can be selected from different types such as backlight, front light, and shading. At the same time, adjust the different positions of the light source on the screen (such as the center, left, upper left, lower, etc.).

[0024] After completing the settings of the light source and position, use the same camera to sequentially acquire the first image and the second image under different light sources and different positions (the two images correspond to two different light sources, and different positions are to take multiple groups of photos at different positions under the same light source, that is, the shooting positions are the same under two different light sources).

[0025] Due to the difference in position, multiple images under the same light source can capture information about the target object from different perspectives. For example, at a certain position of the same light source, the gray value of some areas may be low due to shadows, while at another position, the gray value of this area will be improved. The two complement each other and present the display screen more comprehensively.

[0026] After obtaining the first image, calculate the mean value of the first image. Assume that the size of the first image is m*n, where m is the number of rows of the image and n is the number of columns of the image. By traversing each pixel point in the first image, adding up the pixel values of all pixel points, and then dividing by the total number of pixels m*n of the image, the mean value of the first image can be obtained. This mean value reflects the overall gray level of the first image and is an important reference index for subsequent image gray unevenness correction.

[0027] 102. Calculate the correction coefficient matrix based on the first image, the second image, and the mean value of the first image; Use the following formula to calculate the correction coefficient matrix:

[0028] where N is the first image taken under the first light source, H is the second image taken under the second light source, n is the mean value of the first image, and K is the correction coefficient matrix.

[0029] The correction coefficient matrix is used to describe the correction relationship of the image under different illumination conditions. By calculating the correction coefficient matrix K to correct the first image, the influence of illumination changes on the image quality can be eliminated or reduced.

[0030] 103. Determine the coordinate positions of dust and noise points in the first image; In the frequency domain, dust typically appears as isolated or clustered dark / bright spots in the image, with clear edges but random distribution and no relation to the image content. The gray values in the dust area are usually significantly lower or higher than the background, and there are large differences from neighboring pixels. According to the gray distribution characteristics of the first image, an appropriate threshold method is selected to segment the first image. The first image is divided into a dust area and a background area, such that the pixel gray values in the dust area are lower than the threshold, while the pixel gray values in the background area are higher than the threshold. Morphological operations are performed on the dust area, and a contour detection algorithm is used to find the contour of the dust area. For the contour of each dust area, the corresponding centroid coordinates are calculated as the coordinate position of the dust.

[0031] Noise usually appears as random variations in the gray values of the image. The gray values of the noise are significantly different from the neighborhood, but may show continuous or discrete distributions depending on the type of noise. The noise can be determined by analyzing the local gray statistical characteristics of the image.

[0032] Using a gradient filtering kernel, the gray change rate of the first image in different directions can be calculated. At the same time, the gray mean and variance within the local neighborhood (such as a 3*3 or 5*5 window) of each pixel point in the first image are calculated. If the gray value of a certain pixel point differs greatly from the gray mean of the neighborhood and the variance of the neighborhood exceeds a certain threshold, then this pixel point is considered noise, and the coordinate position of the noise is obtained.

[0033] 104. Adjust the correction coefficient matrix according to the coordinate positions of the dust and the noise, and correct the first image through the adjusted correction coefficient matrix; The correction coefficient matrix is usually used to perform various transformations on the image, such as geometric correction, color correction, etc. According to the information of the dust and the noise, the correction coefficient matrix is adjusted. For example, traverse each pixel point of the first image and determine whether it is dust or noise; if dust or noise is detected, the corresponding correction coefficient for the dust or noise is adjusted accordingly to enhance the correction effect on this area. For pixels that are neither dust nor noise, the pixel values are directly adjusted according to the correction coefficient corresponding to the pixel point in the correction coefficient matrix; the adjustment method is to set the correction coefficient corresponding to the dust or noise to 1, so that the correction coefficient matrix can better adapt to the distribution of dust and noise in the image.

[0034] Traverse each pixel point in the first image. Scan the first image row by row in the order from left to right and from top to bottom; represent the gray value of the pixel point as a vector, multiply the vector by the correction coefficient matrix to obtain the transformed gray value. For the transformed gray value, it is necessary to ensure that it is within the legal range. If it exceeds the range, truncation or normalization processing is required. Assign the gray value obtained after transformation and processing to the pixel point at the corresponding position in the first image, and finally obtain the corrected first image.

[0035] 105. Obtain the fitted edge of the display screen in the first image, and calculate the third image through the homography matrix, where the third image is the display area of the display screen in the first image. Perform grasping and positioning on the first image to obtain the edge points of the display area of the display screen. After determining the edge points of the display screen, use a suitable fitting method to fit these edge points.

[0036] The homography matrix describes the mapping relationship between two images. In the process of obtaining the display area of the display screen, it is necessary to determine the homography matrix from the first image to the display area of the display screen. First, mark the four vertices of the fitted edge of the display screen in the first image (assuming the display screen is rectangular). Then, according to the geometric information of the actual display screen (such as actual size, shape, etc.), determine the corresponding four points on the ideal display screen plane. Use these two sets of corresponding points (the corresponding points of the fitted edge and the ideal corresponding points) to calculate the homography matrices M and B through a specific algorithm (such as the direct linear transformation algorithm). The algorithm establishes a system of linear equations and solves for the elements of the homography matrices M and B.

[0037] After obtaining the homography matrices M and B, for each pixel point in the first image, calculate its corresponding point in the third image (the display area of the display screen) through the following formula. By traversing all pixel points in the first image and performing the above transformation, and rearranging and combining the transformed pixel points, the third image, that is, the display area of the display screen in the first image, can be obtained.

[0038] The algorithm for calculating the third image is as follows:

[0039] where M and B are the rotation matrix and the translation matrix respectively, is the mapping point pair before image correction, is the mapping point pair after image correction.

[0040] 106. Perform convolution operation on the third image according to the adaptive spatial filter kernel matrix; First, set the size of the filter kernel matrix. The filter kernel matrix includes three parts: the inner layer, the middle layer, and the outer layer. It automatically changes its number of layers and numerical size when detecting different defect sizes and different degrees of defect strength. The purpose is to highlight the positions of areas with large differences in pixel gray values within a certain neighborhood, and at the same time, try to eliminate the positions of areas with similar gray values to the neighborhood.

[0041] The filter kernel matrix constructed in this embodiment is as follows:

[0042]

[0043] Among them, i is the number of rows with all 1s in the outer layer of the matrix, j is the number of rows with 1s and 0s in the middle layer of the matrix, and k is the number of rows with 1s, 0s, and in the inner layer of the matrix; is the central weight of the matrix, and i, j, and k are all variable values; Starting from the top-left pixel of the third image, traverse pixel by pixel row by row until the bottom-right pixel. For each pixel, select a local area with the same size as the filter kernel matrix centered on it. Multiply the filter kernel matrix element-wise with the corresponding local area of the image and sum them up to obtain the convolution result. Assign the convolution result of each pixel to the corresponding position in the output image. After traversing all pixels, an image after the adaptive spatial filtering convolution operation is obtained.

[0044] 107. Perform threshold control on the third image after the convolution operation to screen out the fourth image, which is the part of the third image that meets the threshold control; Traverse the third image after the convolution operation, count the number of pixels with each gray level, generate a gray histogram, and the gray distribution of the image can be intuitively understood through the histogram. The threshold is a fixed threshold manually set according to the application scenario of the third image and human experience.

[0045] Traverse each pixel of the third image and compare the gray value of each pixel with the determined threshold. Perform threshold control according to the following formula:

[0046] where is the set lower limit of the gray value, is the set upper limit of the gray value, is the pixel gray value of the pixel point in the third image; If the pixel gray value meets the set threshold condition (within the threshold range), mark the pixel as a valid pixel and retain its position and gray value in the image. If the pixel gray value does not meet the threshold condition (greater than the threshold or less than the threshold), mark the pixel as an invalid pixel. According to the pixel classification result, generate a temporary image, which only retains the pixels that meet the threshold condition, and the pixels that do not meet the condition have been processed.

[0047] Perform connected component analysis on the temporary image to identify the connected pixel regions in the image. By marking different connected components, different objects or regions in the image can be distinguished. According to actual needs, set an area threshold and remove the connected components with an area smaller than this threshold from the temporary image. These small-area regions are usually likely to be noise or irrelevant minor details. After connected component analysis and removing small-area regions, the final fourth image is obtained, which is the part of the third image that meets the threshold control.

[0048] 108. Perform eigenvalue analysis on the fourth image to obtain defect features.

[0049] Perform eigenvalue analysis on the obtained fourth image. Optional feature quantities such as grayscale, contrast, and correlation of the gray-level co-occurrence matrix can be selected. Finally, defects on the display screen are obtained; Taking the grayscale feature as an example, calculate the eigenvalue:

[0050] where: i and j are image grayscale values, and p(i, j) is the probability that the gray level (i, j) appears in the specified direction; According to the formula, calculate the Con value, which reflects the degree of gray-scale change in the image. During the calculation process, it is necessary to traverse each element in the gray-level co-occurrence matrix, calculate the product of the square of the gray-scale difference between pixel pairs and the occurrence probability, and sum these products to obtain the Con value. By analyzing the Con value, the defective area in the image can be identified, and the defective area will show an abnormally high value in the Con value. Set a suitable threshold, mark the area exceeding the threshold as the defective area, and the defect features in the image can be obtained.

[0051] In this embodiment, by calculating the correction coefficient matrix, the influence of different light sources and positions on the image brightness and contrast is eliminated. During the correction process, edge points and noise points are removed to reduce the interference of non-target areas in the first image on subsequent analysis. By fitting the edge of the display screen and combining the homography matrix operation, the original image is converted into a third image that only contains the display area of the display screen, significantly reducing the interference of background noise.

[0052] Construct an adaptive filter kernel to perform convolution operation on the third image, which can effectively smooth noise, enhance defect edges or texture features. And the filter kernel can highlight small defects (such as scratches, bright spots, dark spots) in the display area, making them more easily recognized in subsequent threshold control. By screening the fourth image that meets the conditions through threshold control, irrelevant backgrounds or non-defect signals can be removed, and only the suspected defect areas are retained, effectively reducing false detections caused by noise or image non-uniformity.

[0053] Please refer to Figure 2 , an embodiment of the method for obtaining an image and the image mean in the embodiment of the present application includes: 201. Set the light source at different positions to take pictures of the display screen, summarize the first group of pictures and superimpose the first group of pictures to obtain the first image; According to the size, shape, and detection requirements of the display screen, select multiple different positions to place the light source. These positions should cover various typical angles and distances at which the display screen may be illuminated. For example, the light source can be set in different orientations such as directly in front of the display screen, on the left side, on the right side, above, and below, and different distances can be set for each orientation, such as a short distance (e.g., 0.5 meters), a medium distance (e.g., 1 meter), and a long distance (e.g., 2 meters), etc.

[0054] At each preset light source position, use the camera to take pictures of the display screen, and aggregate all the pictures taken at different light source positions into a first group of pictures. The first group of pictures is superimposed by means of pixel value averaging, that is, for the pixel points at the same position in each picture, their pixel values are added and then divided by the total number of pictures, so as to obtain the pixel value at the corresponding position in the first image after superposition.

[0055] 202. After shading the lens of the camera, take pictures of the display screen, aggregate the second group of pictures and superimpose the second group of pictures to obtain a second image; Use light-blocking materials, such as a black light-shielding hood or light-shielding cloth, to tightly shade the lens of the camera to ensure that no external light can enter the camera lens. After the lens shading treatment is completed, keep the camera parameters consistent with those in step 201, and take pictures of the display screen. Aggregate all the pictures taken into a second group of pictures, and perform a superimposing operation on the second group of pictures using the same superimposing method as the first group of pictures to obtain a second image.

[0056] 203. Statistically calculate the pixel values of the first image to obtain the first image mean value.

[0057] Read the first image, traverse each pixel point of the first image row by row and column by column to obtain its pixel value. Statistically calculate all the extracted pixel values, and calculate the average value of all pixel values, that is, the first image mean value.

[0058] In the embodiment, by taking pictures and superimposing at different light source positions, the illumination information at various angles and directions can be integrated, effectively eliminating the problem of uneven illumination that may be generated at a single light source position, making the illumination of the first image more uniform, thereby improving the accuracy of subsequent defect detection. Superimposing multiple pictures can reduce the influence of random noise. Since the distribution of random noise in different pictures is random, the superimposing operation can make the noise cancel each other out, improve the signal-to-noise ratio of the image, make the image clearer, and facilitate the identification and analysis of defects. The first image mean value reflects the overall brightness level of the first image, providing data support for the subsequent image processing and defect detection processes.

[0059] Please refer to Figure 3 , an embodiment of the method for determining the positions of dust and noise points in the embodiment of the present application includes: 301. Determine the coordinate position of dust in the first image based on the amplitude and direction of the first image; The first image obtains the gradient amplitude and direction of the first image through a filter kernel, and determines the coordinate position of dust in the first image according to the gradient amplitude and gradient direction of the first image; The horizontal direction filter kernel is:

[0060] The vertical direction filter kernel is:

[0061] For each pixel point in the first image, use the filter kernel to perform weighted summation within the pixel point and its neighborhood to obtain the gradient values of the pixel point in the horizontal and vertical directions. According to the gradient values in the horizontal and vertical directions, calculate the gradient amplitude and gradient direction of each pixel point.

[0062] 302. Determine the coordinate position of noise points in the first image through vector map mapping comparison of the first image.

[0063] A vector map is a graphical representation method that describes the content of an image based on mathematical formulas. Convert the first image into a vector map and establish a mapping relationship between the first image and its vector map. This means that for each pixel point in the first image, a corresponding position or element can be found in the vector map.

[0064] On the basis of the mapping, perform point-by-point or region-by-region comparison of the first image and the vector map. During the comparison process, those pixel points with relatively large differences from the corresponding positions in the vector map may be noise points. By comparing and analyzing the entire image, record the coordinate positions of those pixel points determined to be noise points, so as to obtain the coordinate position information of noise points in the first image.

[0065] In this embodiment, by analyzing the amplitude and direction, the location of the dust is found, avoiding the problem of difficult identification due to the similar color and shape of the dust to the background, and improving the accuracy of dust positioning. The vector map highlights and locates noise points through mapping comparison with the original image, which can avoid misjudging normal details in the image as noise points and improve the accuracy of noise point detection. Determining the coordinate position of the dust and the position coordinates of the noise points provides an accurate position basis for subsequent processing operations.

[0066] Please refer to Figure 4 , an embodiment of the method for determining the dust position coordinates in the embodiment of the present application includes: 401. Perform smoothing filtering processing on the first image; Smoothing filtering is an image processing technique used to reduce image noise and make the image smoother. It reduces the sudden change of pixel values by performing a weighted average operation on each pixel point and its neighboring pixel points in the first image, thereby achieving the effect of suppressing noise and smoothing the image.

[0067] Select an appropriate filter kernel size according to the noise situation of the image and actual requirements. Starting from the upper left corner of the image, traverse each pixel point in the image row by row and column by column.

[0068] Perform filtering on the first image according to the following formula:

[0069] where, is the pixel point in the first image, is the pixel point after mean filtering, is the filter kernel of size M*N.

[0070] Assign the calculated gray value G(x,y) to the pixel point with coordinates (x,y) in the image to complete the smoothing filtering process of this pixel point. Repeat the above steps until all pixel points in the image are traversed to obtain a complete image after smoothing filtering.

[0071] 402. The first image obtains the gradient magnitude and gradient direction of the first image through the gradient filter kernel; The gradient filter kernel includes a horizontal direction and a vertical direction of two filter kernels. Convolve the first image after smoothing filtering with the filter kernel and the filter kernel respectively to obtain the gradient components of the image in the horizontal and vertical directions. For each pixel point in the image, the result of its convolution operation is the sum of the products of the pixel values in the neighborhood of this pixel point and the corresponding elements of the convolution kernel.

[0072] 403. For each pixel point in the first image, along the gradient direction of the pixel point, compare the gradient magnitude of the pixel point with the gradient magnitudes of the pixel points on both sides of the gradient direction to determine the dust in the first image.

[0073] For each pixel point in the first image, determine its corresponding discrete direction according to its gradient direction. Then along this discrete direction, find the adjacent pixel points on both sides of the gradient direction of this pixel point, and compare the gradient magnitude of this pixel point with the gradient magnitudes of the adjacent pixel points on both sides.

[0074] Set two thresholds, a low threshold and a high threshold. For the image obtained after the above comparison, the pixel points with gradient magnitude greater than the high threshold are determined as dust points, the pixel points with gradient magnitude less than the low threshold are discarded, and the pixel points with gradient magnitude between the low threshold and the high threshold. Finally, record the coordinate positions of the dust points (i.e., the dust in the first image).

[0075] In this embodiment, through smoothing filtering, the noise in the image can be suppressed, the first image can be made smoother, the generation of false edges can also be reduced, and edge detection can be made more accurate. At the same time, smoothing filtering can also make the edges in the image more continuous, facilitating subsequent analysis and processing of the edges.

[0076] By calculating the gradient magnitude and direction, the dust in the image can be extracted. By comparing the gradient magnitude of a pixel point with the gradient magnitudes of the pixel points on both sides of the gradient direction, the dust becomes clearer. By setting two thresholds, the true dust can be distinguished from interferences such as noise, improving the reliability of edge detection. Only the pixel points with a relatively large gradient magnitude will be retained as dust points, thus reducing the possibility of misjudgment.

[0077] Please refer to Figure 5 , an embodiment of the method for determining the coordinate position of the noise point in the embodiment of the present application includes: 501. Generate two all - black images with the same size as the first image; According to the width and height information of the first image, create two new images. During the creation process, initialize the gray - scale values of all pixel points in the image to 0, that is, set them all to black, ensuring that the sizes of these two images are exactly the same as the first image, providing a basis for subsequent operations.

[0078] 502. Set the row - coordinate values and column - coordinate values of the all - black images as the gray - scale values of the two all - black images respectively; Traverse each pixel point of the two all - black images. For the first all - black image, use its row - coordinate value as the gray - scale value of this pixel point; for the second all - black image, use its column - coordinate value as the gray - scale value of this pixel point. For example, for a pixel point with coordinates (i,j), its gray - scale value in the first all - black image is set to i, and its gray - scale value in the second all - black image is set to j.

[0079] 503. Taking each pixel point in the all - black image as the center, calculate the neighborhood row - column coordinate array of each pixel point; For each pixel point in the all - black image, take it as the center and determine the neighborhood range of this central pixel point. The neighborhood can be a square, rectangle, or other shaped area centered on this pixel point. Usually, a square neighborhood with a size of 3×3, 5×5, etc. centered on the central pixel point is selected.

[0080] After defining a neighborhood of an appropriate size, for each pixel, loop through the calculations to obtain the row and column coordinates of all pixels in its neighborhood respectively. Taking a 3×3 neighborhood as an example, for the pixel with coordinates (0, 0), its neighborhood row and column coordinate array contains (−1,−1), (−1,0), (−1,1), (0,−1), (0,0), (0,1), (1,−1), (1,0), (1,1). Convert the neighborhood row and column coordinate array into the following form: [-1,-1,-1,0,0,0,1,1,1],[-1,-0,1,-1,0,1,-1,0,1] Or, ,

[0081] 504. Perform arithmetic operations on the grayscale values in the order of the neighborhood row and column coordinate array, and perform grayscale threshold verification; For each pixel and its neighborhood, according to the order of the neighborhood row and column coordinate array, perform arithmetic operations on the grayscale values of the pixels in the neighborhood, and operations such as summation and averaging can be performed Set an appropriate grayscale threshold, that is, limit the grayscale value of the image within the width and height range, namely 0~(Height - 1), 0~(Width - 1). Compare the result obtained from the arithmetic operation with this threshold to determine whether the condition is met. If the operation result is within the threshold, record the relevant information of the pixel.

[0082] 505. Take the intersection of the image regions that meet the grayscale threshold condition in the all - black image, and retain the corresponding intersection region from the first image, and convert the intersection region into a vector image; Analyze the distribution of the pixels that meet the grayscale threshold condition in the two all - black images, and find their overlapping region, that is, the intersection region. For example, the set of pixels that meet the condition in the first image is A, and the set of pixels that meet the condition in the second image is B, then find the intersection A∩B of A and B.

[0083] According to the coordinate information of the intersection region, find the corresponding region in the first image and retain it. Then use an image vectorization tool to convert the retained intersection region into a vector image. Vector images have advantages such as resolution independence and are convenient for subsequent processing and analysis.

[0084] 506. Map the first image with the vector image to obtain multiple mapped images centered on pixels; Taking each pixel in the vector image as the center, determine the corresponding region in the first image to generate multiple mapped images. By traversing all the pixels in the vector image, multiple such mapped images are obtained.

[0085] 507. Perform arithmetic operations on multiple mapped images and the first image respectively, and determine the coordinate positions of noise points according to the output results. The output results include the gray values at the same positions in multiple mapped images.

[0086] Perform arithmetic operations on the gray values of pixel points at the same positions in multiple mapped images and the first image, such as summation, averaging, variance calculation, etc. According to the results of the arithmetic operations and combined with the set judgment criteria, determine the noise points. If the variance at a certain position is greater than a preset threshold, it indicates that the gray value at this position fluctuates greatly and there may be noise points, and mark them as noise points; otherwise, it is considered a normal pixel point. Finally, record the coordinate positions of the noise points.

[0087] In this embodiment, through a series of gray value operations, threshold verification, and region intersection operations, the relevant regions of noise points can be accurately found in the first image. Converting the relevant regions into vector images can accurately process local parts of the image and improve the accuracy and effect of processing. By performing arithmetic operations on multiple mapped images, the noise points and normal pixel points can be effectively distinguished according to the distribution characteristics of gray values, and the position coordinates of the noise points can be determined, reducing the possibility of misjudgment and improving the image quality.

[0088] Please refer to Figure 6 , an embodiment of the method for obtaining the third image in the embodiments of the present application includes: 601. Construct multiple rectangular frames at the edge of the display screen in the first image; Determine the size and quantity of the rectangular frames according to the situation of the display screen edge. If the edge of the display screen is relatively regular, rectangular frames of the same size can be set; if the edge is irregular, rectangular frames of different sizes can be set in different curvature change regions. The quantity of the rectangular frames should be sufficient to cover the edge of the display screen, and at the same time, it should not be too dense to cause excessive calculation.

[0089] 602. Traverse the gray values of pixel points in each rectangular frame and calculate the gray value difference between each pixel point and its adjacent pixel points; For each constructed rectangular frame, use the image traversal algorithm to start from the upper left corner pixel point of the rectangular frame and access each pixel point in the frame row by row and column by column.

[0090] The adjacent pixel points are the pixel points directly adjacent in the horizontal and vertical directions (i.e., adjacent above, below, left, and right), and can also include the pixel points adjacent diagonally. For each traversed pixel point, obtain its own gray value and the gray value of the adjacent pixel point, and calculate the gray value difference between them through subtraction. For example, for the pixel point P(i,j) and its horizontal adjacent pixel point P(i + 1,j), the gray value difference is |P(i,j) - P(i + 1,j)|.

[0091] 603. Select the coordinates of the edge pixels of the display screen in the first image according to the grayscale difference value; According to the characteristics of the first image and human experience, set a suitable grayscale difference threshold, which is used to determine whether a pixel is located at the edge of the display screen. Traverse the grayscale difference values calculated within each rectangular frame. For the pixels with grayscale difference values greater than the set threshold, it is considered that they are located at the edge of the display screen, and record the coordinates of these pixels.

[0092] 604. Fit the coordinates of the edge pixels into the fitted edge of the display screen in the first image, and calculate the intersection points of the fitted edges. The intersection points of the fitted edges are the edge corner points of the display screen in the first image; For a rectangular display screen, a linear fitting algorithm can be used to fit the coordinates of the selected edge pixels into a continuous curve or straight line to obtain the fitted edge of the display screen. The principle of the least squares linear fitting is to determine the parameters of the straight line by minimizing the sum of the squares of the perpendicular distances from the edge points to the fitted straight line. By fitting the four sides, a more accurate and smooth representation of the display screen edge can be obtained.

[0093] For multiple fitted edges, according to the equations of the straight lines or curves, calculate the intersection points between the edges by solving the system of equations. These intersection points are the edge corner points of the display screen.

[0094] 605. Correct the fitted edges and intersection points through the homography matrix to obtain the third image; According to the calculated homography matrix, perform coordinate transformation on the fitted edges and intersection points to correct the edges and corner points of the display screen to the correct positions and shapes, obtaining the third image. Specifically, apply the transformation formula of the homography matrix to the coordinates of each edge pixel and intersection point to update their coordinate values, thereby obtaining the corrected image.

[0095] 606. Perform regional patching on the third image through the harmonic difference algorithm.

[0096] In the actual process, after obtaining the third image by correcting the second image through the homography matrix in step 605, it is also necessary to handle the possible non-integer problems of the pixels in the third image. Usually, an interpolation algorithm is used to determine the grayscale values of the pixels to ensure the quality of the image.

[0097] First, it is necessary to clarify the parts of the image that need to be patched, that is, the background areas of the arc-shaped corners and the dust filtering parts and other areas that may cause interference. Determine suitable known pixel points around the area to be repaired. The known pixel points are pixel points evenly distributed around the area to be repaired.

[0098] Perform regional patching on the parts of the third image that need to be patched through the harmonic interpolation algorithm:

[0099] Among them, is the gray value of the pixel point to be repaired in the third image, is the gray value of the known pixel points around the point to be repaired, is the distance between two pixel points; Let the known pixel point be a and the pixel point to be repaired be b. Calculate the distance (b - a) between each known pixel point a and the pixel point to be repaired b. Determine the weight of each known pixel point according to the distance. Generally, the closer the distance, the greater the weight.

[0100] For each pixel point b in the area to be repaired, perform weighted averaging according to the respective weights of multiple known pixel points a to obtain the gray value of the final pixel point b to be repaired. For all pixel points in the entire area to be repaired, repeat the above steps to estimate the gray value point by point until the repair of the entire area is completed, and obtain the repaired third image.

[0101] In this embodiment, by constructing a rectangular frame at the edge of the display screen and calculating the gray value difference of pixel points, edge pixel points can be screened out, the influence of interference factors such as the background can be reduced, and the accuracy of edge pixel point positioning can be improved. Using the fitting algorithm and the intersection calculation method, the fitting edge and corner points of the display screen can be accurately obtained. By correcting the fitting edge and intersection points through the homography matrix, the geometric deformation that may exist in the display screen image can be corrected, a more realistic third image can be obtained, and the image quality is improved.

[0102] By performing regional repair on the third image through the harmonic difference algorithm, the defects, damages or incomplete parts existing in the image can be effectively filled, making the image visually more complete and continuous, and avoiding the influence of the missing or abnormal local area on the analysis and understanding of the entire display screen image.

[0103] Please refer to Figure 7 , this application provides an embodiment of a defect detection device for images, including: An acquisition unit 701, configured to obtain a first image and a second image according to different light sources and light source positions of the camera and calculate the first image mean value, where the first image mean value is the mean value of all pixel values in the first image; A first calculation unit 702, configured to calculate a correction coefficient matrix according to the first image, the second image, and the first image mean value; A determination unit 703, configured to determine the coordinate position and coordinate coefficient of dust in the first image and the coordinate position and coordinate coefficient of noise points; A correction unit 704, configured to adjust the correction coefficient matrix according to the coordinate position and coordinate coefficient of dust and the coordinate position and coordinate coefficient of noise points, and correct the first image through the adjusted correction coefficient matrix; The second calculation unit 705 is configured to obtain the fitted edge of the display screen in the first image and calculate a third image through a homography matrix, where the third image is the display area of the display screen in the first image; The third calculation unit 706 is configured to perform a convolution operation on the third image according to the adaptive spatial filtering kernel matrix; The processing unit 707 is configured to perform threshold control on the third image after the convolution operation is completed to screen out a fourth image, where the fourth image is the image part in the third image that meets the threshold control; The analysis unit 708 is configured to perform eigenvalue analysis on the fourth image to obtain defect features.

[0104] Optionally, the defect detection device further includes: The repair unit 709 repairs the area of the third image through a harmonic difference algorithm.

[0105] In this embodiment, the functions of each unit and module correspond to the steps in the foregoing Figures 1 to 6 illustrated embodiment, and will not be elaborated here.

[0106] Please refer to Figure 8 , another embodiment of an image defect detection device in an embodiment of the present application includes: A processor 801, a memory 802, an input / output unit 803, and a bus 804; The processor 801 is connected to the memory 802, the input / output unit 803, and the bus 804; A program is stored on the memory 802, and the processor 801 calls the program to execute Figures 1 to 6 the steps in the illustrated embodiment.

[0107] In this embodiment, the function of the processor 801 corresponds to the steps in the foregoing Figures 1 to 6 illustrated embodiment, and will not be elaborated here.

[0108] The embodiment of the present application further provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the computer is caused to execute the defect detection method in any of the foregoing Figures 1 to 6 possible implementation manners.

[0109] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated here.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0113] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs, and other various media that can store program codes.

Claims

1. A method for defect detection of an image, characterized in that, Including: Obtain a first image and a second image according to different light sources and light source positions of the camera, and calculate the mean value of the first image, where the mean value of the first image is the mean value of all pixel values in the first image; Calculate a correction coefficient matrix according to the first image, the second image, and the mean value of the first image; Determine the coordinate positions of dust and noise points in the first image; Adjust the correction coefficient matrix according to the coordinate positions of the dust and the noise points, and correct the first image through the adjusted correction coefficient matrix; Obtain the fitted edge of the display screen in the first image, and calculate a third image through a homography matrix, where the third image is the display area of the display screen in the first image; Perform a convolution operation on the third image according to an adaptive spatial filtering kernel matrix; Perform a threshold control on the third image after the convolution operation is completed, and screen out a fourth image, where the fourth image is the image part in the third image that satisfies the threshold control; Perform eigenvalue analysis on the fourth image to obtain defect features.

2. The defect detection method according to claim 1, wherein The obtaining the first image and the second image according to different light sources and light source positions of the camera and calculating the mean value of the first image includes: Set the light source at different positions to take pictures of the display screen, summarize the first group of pictures, and superimpose the first group of pictures to obtain the first image; After shading the lens of the camera, take pictures of the display screen, summarize the second group of pictures, and superimpose the second group of pictures to obtain the second image; Statistically calculate the pixel values of the first image to obtain the mean value of the first image.

3. The defect detection method according to claim 2, wherein The determining the coordinate positions of dust and noise points in the first image includes: Determine the coordinate position of dust in the first image through the amplitude and direction of the first image; Determine the coordinate position of noise points in the first image through vector map mapping comparison of the first image.

4. The defect detection method according to claim 3, wherein The determining the coordinate position of dust in the first image through the amplitude and direction of the first image includes: Perform a smoothing filtering process on the first image; The first image obtains the gradient amplitude and gradient direction of the first image through a gradient filtering kernel; For each pixel point in the first image, along the gradient direction of the pixel point, compare the gradient amplitude of the pixel point with the gradient amplitudes of the pixel points on both sides of the gradient direction to determine the coordinate position of dust in the first image.

5. The defect detection method according to claim 4, wherein, The determining the coordinate position of noise points in the first image through vector map mapping comparison of the first image includes: Generate two all-black images with the same size as the first image; Set the row coordinate values and column coordinate values of the all-black images as the gray values of the two all-black images respectively; Taking each pixel point in the all-black image as the center, calculate the neighborhood row and column coordinate array of each pixel point; Perform arithmetic operations on the gray values in the order of the neighborhood row and column coordinate array, and perform gray threshold verification; Take the intersection of the image areas in the all-black image that meet the gray threshold condition, retain the corresponding intersection area from the first image, and convert the intersection area into a vector image; Map the first image to the vector image to obtain multiple mapped images centered on pixel points; Perform arithmetic operations on the multiple mapped images and the first image respectively, and determine the coordinate positions of noise points according to the output results, where the output results include the gray values at the same position in the multiple mapped images.

6. The defect detection method according to any one of claims 1-5, characterized in that, The obtaining of the fitted edge of the display screen in the first image and the calculation of the third image through the homography matrix include: Construct multiple rectangular frames at the edge of the display screen in the first image; Traverse the gray values of the pixel points in each rectangular frame, and calculate the gray difference between each pixel point and its adjacent pixel points; According to the gray difference, screen out the coordinate positions of the edge pixel points of the display screen in the first image; Fit the coordinate positions of the edge pixel points to the fitted edge of the display screen in the first image, and calculate the intersection points of the fitted edge, where the intersection points of the fitted edge are the edge corner points of the display screen in the first image; Correct the fitted edge and the intersection points through the homography matrix to obtain the third image.

7. The defect detection method according to any one of claims 1-5, characterized in that, After obtaining the fitted edge of the display screen in the first image and calculating the third image through the homography matrix, it further includes: Perform regional repair on the third image through the harmonic difference algorithm.

8. An image defect detection device, characterized in that, It includes: An acquisition unit, configured to acquire a first image and a second image according to different light sources and light source positions of the camera, and calculate the first image mean value, where the first image mean value is the mean value of all pixel values in the first image; A first calculation unit, configured to calculate a correction coefficient matrix according to the first image, the second image, and the first image mean value; A determination unit, configured to determine the coordinate positions of dust and noise points in the first image; A correction unit, configured to adjust the correction coefficient matrix according to the coordinate positions of the dust and noise points, and correct the first image through the adjusted correction coefficient matrix; A second calculation unit, configured to acquire the fitted edge of the display screen in the first image, and calculate a third image through the homography matrix, where the third image is the display area of the display screen in the first image; A third calculation unit, configured to perform a convolution operation on the third image according to the adaptive spatial filtering kernel matrix; A processing unit, configured to perform threshold control on the third image after the convolution operation is completed, and screen out a fourth image, where the fourth image is the image part in the third image that meets the threshold control; An analysis unit, configured to perform eigenvalue analysis on the fourth image to obtain defect features.

9. An image defect detection device, characterized in that, It includes: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the defect detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored on the computer-readable storage medium, and when the program is executed on a computer, the computer is caused to execute the defect detection method according to any one of claims 1 to 7.

Images