LED Special-shaped Screen Surface Defect Detection Method and Device Based on Image Analysis

Abstract

The present invention provides a method and device for detecting surface defects of LED special-shaped screens based on image analysis, which relates to the technical field of image processing. The device for detecting surface defects of LED special-shaped screens based on image analysis includes a data acquisition module, a geometric correction module, a grayscale image processing module, a feature region extraction module, a feature region fusion module, and a brightness anomaly detection module. The present invention performs geometric correction processing on multiple collected test images to obtain multiple first grayscale images, performs edge detection and line detection on the first grayscale images, and effectively eliminates the interference of perspective distortion and noise based on morphological operations. By combining wavelet transform and multi-scale analysis, it accurately extracts and identifies the brightness anomaly regions on the LED special-shaped screens, realizing high-precision detection of the brightness anomaly regions on the surface of the LED special-shaped screens. Especially for the detection of regions with gradual brightness change, the detection effect is remarkable. While improving the detection accuracy and enhancing the detection robustness, it effectively reduces false detection and missed detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and device for detecting surface defects of LED special-shaped screens based on image analysis. Background Art

[0002] With the increasing demand for personalized displays, LED special-shaped screens have gradually become a hot spot in the market. These LED special-shaped screens not only have the display function of conventional LED screens, but can also be customized according to different scenarios and requirements, showing more unique visual effects.

[0003] Due to the complex structure and irregular module arrangement of LED special-shaped screens, it is easier for them to have surface defects during manufacturing and use. Among them, affected by various factors, such as the aging of LED lamp beads themselves, the failure of drive circuits, or uneven heat dissipation, etc., in an LED special-shaped screen, the brightness of the LED lamp beads in a certain area may gradually decrease from the center to the periphery, forming a visually relatively blurred brightness gradient area. Since this brightness abnormal area is an area where the brightness gradually changes from the center to the edge, there is usually no obvious boundary, and the blurred boundary feature makes it difficult for conventional edge detection and simple threshold segmentation methods to accurately identify the brightness defect area. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a method and device for detecting surface defects of LED special-shaped screens based on image analysis, which can effectively extract the brightness defect areas with gradually decreasing brightness, and significantly improve the accuracy of detecting surface defects of LED special-shaped screens.

[0005] To achieve the above object, in the first aspect of the present invention, a method for detecting surface defects of LED special-shaped screens based on image analysis is provided, including:

[0006] Collecting image data of the LED special-shaped screen to be detected under a brightness test pattern and constructing a dataset to be detected containing multiple test images;

[0007] Performing geometric correction on multiple test images in the dataset to be detected according to the three-dimensional structure parameters of the LED special-shaped screen to be detected, and performing gray-scale conversion on the images obtained after geometric correction to obtain multiple first gray-scale images;

[0008] Performing line detection on multiple first gray-scale images based on the Hough transform to determine multiple target line regions, traversing each first gray-scale image based on a preset feature window to determine multiple target pixel points in the first gray-scale image, and processing the multiple target line regions and multiple target pixel points based on morphological operations to obtain a second gray-scale image corresponding to each first gray-scale image;

[0009] Perform wavelet transform and multi-scale wavelet decomposition on multiple second grayscale images, and extract the low-frequency sub-images and multiple high-frequency sub-images of each second grayscale image at each scale;

[0010] Traverse each high-frequency sub-image based on a preset feature window, determine multiple feature thresholds of each high-frequency sub-image, and extract the first feature region image of each high-frequency sub-image at each scale based on the feature thresholds;

[0011] Fuse multiple first feature region images of each second grayscale image at the same scale to obtain the second feature region image of each second grayscale image at each scale;

[0012] Perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain the target feature region image corresponding to each second grayscale image;

[0013] Perform feature fusion on multiple feature region images to obtain a brightness anomaly image, and determine multiple target regions from the brightness anomaly image according to the grayscale values of each pixel point in the brightness anomaly image;

[0014] Generate the brightness anomaly detection result of the LED special-shaped screen to be detected according to the target pixel points, target line regions and target regions.

[0015] Preferably, for the first grayscale image and the second grayscale image, it further includes:

[0016] Perform edge detection on each first grayscale image based on the Canny operator to obtain the corresponding edge binary image of each first grayscale image, perform line detection on the edge binary image based on the Hough transform, determine multiple target line regions in the edge binary image and map them to the first grayscale image, divide the multiple target line regions in the first grayscale image into the first line region and the second line region, perform erosion processing on the first line region, and perform dilation processing on the second line region;

[0017] Traverse each high-frequency sub-image based on a preset feature window, determine the maximum and minimum values of the grayscale values in each preset feature window, determine multiple target pixel points in each first grayscale image based on the maximum and minimum values of the grayscale values, divide the multiple target pixel points into the first pixel points and the second pixel points, perform erosion processing on the first pixel points, and perform dilation processing on the second pixel points to obtain the second grayscale image corresponding to each first grayscale image.

[0018] Preferably, traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting the first feature region image of each high-frequency sub-image at each scale based on the feature thresholds includes:

[0019] Traverse each high-frequency subgraph based on a preset feature window, calculate the mean and standard deviation of the wavelet coefficients within each preset feature window, and determine the feature threshold corresponding to multiple pixel points within each preset feature window;

[0020] Among them, , in the formula, is the feature threshold corresponding to multiple pixel points within the th preset feature window, is the mean of the wavelet coefficients within the th preset feature window, is the standard deviation of the wavelet coefficients within the th preset feature window, is an adjustment parameter;

[0021] According to the horizontal edge feature subgraph and the vertical edge feature subgraph in multiple high-frequency subgraphs, calculate the correction factor for each preset feature window, and correct the feature threshold corresponding to multiple pixel points within each preset feature window based on the correction factor to obtain the adaptive threshold corresponding to multiple pixel points within each preset feature window. Among them, the following formula is used to determine the adaptive threshold corresponding to multiple pixel points within each preset feature window: In the formula, is the adaptive threshold corresponding to multiple pixel points within the th preset feature window, is the correction factor of the th preset feature window, is the brightness change intensity factor of the th preset feature window, , is the sum of the wavelet coefficients within the th preset feature window in the horizontal edge feature subgraph, is the sum of the wavelet coefficients within the th preset feature window in the vertical edge feature subgraph, is the maximum value of the brightness change intensity factors in multiple preset feature windows;

[0022] For multiple pixel points within any preset feature window in any high-frequency subgraph, set the wavelet coefficients corresponding to the pixel points whose wavelet coefficients are not greater than the adaptive threshold to 0, so as to extract the first feature region image of each high-frequency subgraph at each scale based on the adaptive threshold.

[0023] Preferably, fuse the multiple first feature region images of each second grayscale image at the same scale to obtain the second feature region image of each second grayscale image at each scale, including:

[0024] For multiple first feature region images of each second grayscale image at the same scale, calculate the standard deviation of the wavelet coefficients of each first feature region image among the multiple first feature region images, calculate the mean value of the standard deviations of the wavelet coefficients of the multiple first feature region images, and use the ratio between the standard deviation of the wavelet coefficients of the first feature region image and the mean value of the standard deviations of the wavelet coefficients of the multiple first feature region images as the fusion weight of the first feature region image. Based on the fusion weight corresponding to each first feature region image, perform weighted fusion on the multiple first feature region images of each second grayscale image at the same scale to obtain the second feature region image of each second grayscale image at each scale.

[0025] Preferably, perform feature fusion on multiple feature region images to obtain a brightness anomaly image, and determine multiple target regions from the brightness anomaly image according to the grayscale value of each pixel point in the brightness anomaly image, including:

[0026] Traverse each feature region image based on a preset feature window, calculate the average value of the grayscale values of the pixel points within each preset feature window, calculate the standard deviation of the grayscale values of the pixel points within each preset feature window, calculate the mean value of the standard deviations of the grayscale values of the pixel points within multiple preset feature windows in each feature region image, and use the ratio between the average value of the grayscale values of the pixel points within the preset feature window and the mean value of the standard deviations of the grayscale values of the pixel points within multiple preset feature windows in the feature region image to which the preset feature window belongs as the local weight of the preset feature window. Perform normalization processing on the local weights corresponding to multiple preset feature windows in each feature region image, and perform weighted fusion on multiple preset feature windows at the same position in multiple feature region images based on the normalized local weights to obtain a brightness anomaly image;

[0027] Perform edge detection on the brightness anomaly image according to the grayscale value of each pixel point in the brightness anomaly image to determine multiple brightness anomaly regions, denoted as target regions.

[0028] Preferably, perform wavelet transform processing on multiple second grayscale images using Daubechies wavelet as the wavelet basis function.

[0029] The second aspect of the present invention provides an LED special-shaped screen surface defect detection device based on image analysis, which is used to implement the above-mentioned LED special-shaped screen surface defect detection method based on image analysis, including:

[0030] A data acquisition module, which is used to collect image data of the LED special-shaped screen to be detected under a brightness test pattern and construct a to-be-detected data set including multiple test images;

[0031] A geometric correction module, configured to perform geometric correction on multiple test images in a dataset to be detected according to the three-dimensional structure parameters of the LED special-shaped screen to be detected, and perform grayscale conversion on the images obtained after geometric correction to obtain multiple first grayscale images;

[0032] A grayscale image processing module, configured to perform line detection on multiple first grayscale images based on the Hough transform to determine multiple target line regions, traverse each first grayscale image based on a preset feature window to determine multiple target pixel points in the first grayscale image, and perform processing on the multiple target line regions and multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image;

[0033] A feature region extraction module, configured to perform wavelet transform and multi-scale wavelet decomposition on multiple second grayscale images, extract the low-frequency sub-images and multiple high-frequency sub-images of each second grayscale image at each scale, traverse each high-frequency sub-image based on a preset feature window to determine multiple feature thresholds of each high-frequency sub-image, and extract the first feature region images of each high-frequency sub-image at each scale based on the feature thresholds;

[0034] A feature region fusion module, configured to fuse multiple first feature region images of each second grayscale image at the same scale to obtain a second feature region image of each second grayscale image at each scale, and perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image;

[0035] A brightness anomaly detection module, configured to perform feature fusion on multiple feature region images to obtain a brightness anomaly image, determine multiple target regions from the brightness anomaly image according to the grayscale values of each pixel point in the brightness anomaly image, and generate a brightness anomaly detection result of the LED special-shaped screen to be detected according to the target pixel points, target line regions, and target regions.

[0036] Preferably, for the grayscale image processing module, it further includes:

[0037] Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform to determine multiple target line regions in the edge binary image and map them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing erosion processing on the first line region, and performing dilation processing on the second line region;

[0038] Traverse each high-frequency subgraph based on a preset feature window, determine the maximum and minimum gray values in each preset feature window, determine multiple target pixel points in each first gray image based on the maximum and minimum gray values, divide the multiple target pixel points into first pixel points and second pixel points, perform erosion processing on the first pixel points, and perform dilation processing on the second pixel points to obtain a second gray image corresponding to each first gray image.

[0039] The present invention has the following beneficial effects:

[0040] The present invention performs geometric correction processing on multiple collected test images to obtain multiple first gray images, performs edge detection and line detection on the first gray images, and effectively eliminates the interference of perspective distortion and noise based on morphological operations. Combining wavelet transform and multi-scale analysis, it accurately extracts and identifies the brightness abnormal regions on the LED special-shaped screen, including regions with gradually changing brightness, bright spots, dark spots, etc., realizing high-precision detection of the brightness abnormal regions on the surface of the LED special-shaped screen. Especially for the detection of regions with gradually changing brightness, the detection effect is remarkable, improving the detection accuracy and enhancing the detection robustness while effectively reducing false detections and missed detections. Description of the Drawings

[0041] Figure 1 It is a schematic flowchart of a method for detecting surface defects of an LED special-shaped screen based on image analysis provided by an embodiment of the present invention.

[0042] Figure 2 It is a schematic structural diagram of a device for detecting surface defects of an LED special-shaped screen based on image analysis provided by an embodiment of the present invention. Detailed Embodiments

[0043] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] See Figure 1 , an embodiment of the present invention provides a method for detecting surface defects of an LED special-shaped screen based on image analysis, including the following steps:

[0045] S10. Collect image data of the LED special-shaped screen to be detected under a brightness test pattern and construct a to-be-detected data set including multiple test images, perform geometric correction on the multiple test images in the to-be-detected data set according to the three-dimensional structure parameters of the LED special-shaped screen to be detected, and perform gray conversion on the images obtained after geometric correction to obtain multiple first gray images.

[0046] In this embodiment, a high-resolution camera or image sensor device is used to capture the LED special-shaped screen to be detected under a preset brightness test pattern. The brightness test pattern can be a series of solid-color patterns with different gray values, such as 0% (black), 25%, 50%, 75%, 100% (white), so as to effectively stimulate the brightness characteristics of the LED special-shaped screen to be detected, thereby collecting a series of test images and constructing a dataset to be detected containing multiple test images. Since the LED special-shaped screen may have complex geometric shapes, such as arc-shaped, spherical or other irregular shapes, geometric correction of the test images can convert these images into standard two-dimensional plane projections, thereby eliminating the distortion caused by the viewing angle and perspective effect in the images. Geometric correction usually uses image registration technology or camera calibration method to correct the geometric shape of the image by matching control points or feature points, and specifically realizes the correction process based on the three-dimensional structure parameters of the LED special-shaped screen (such as curvature, bending angle, module arrangement method, etc.). Geometric correction is a well-known technical means for those skilled in the art and will not be elaborated here. The geometrically corrected images are subjected to gray-scale conversion to generate multiple first gray-scale images, so as to simplify the color information of each pixel in the image into a single brightness value, which is convenient for subsequent image processing and analysis.

[0047] S20. Perform line detection on multiple first gray-scale images based on the Hough transform to determine multiple target line regions, traverse each first gray-scale image based on a preset feature window to determine multiple target pixel points in the first gray-scale image, and process the multiple target line regions and multiple target pixel points based on morphological operations to obtain a second gray-scale image corresponding to each first gray-scale image.

[0048] In this embodiment, the Hough transform is a commonly used line detection algorithm. By mapping the straight lines in the image space to the parameter space, it can effectively detect the straight line features in the image. Perform the Hough transform on each first gray-scale image to detect the line structure in the image. The detected target line regions may represent the boundaries between LED modules, the edges of defect regions such as bright lines and dark lines, etc. The multiple target pixel points determined by traversing each first gray-scale image based on the preset feature window may represent potential brightness abnormal regions, such as bright points and dark points, etc. The preset feature window is a rectangular region with a fixed size (for example, 3×3 or 5×5). Those skilled in the art can select an appropriate window size according to actual needs, and this embodiment does not specifically limit it. Using morphological operations (such as erosion, dilation, opening operation, closing operation, etc.) to process the detected multiple target line regions and target pixel points can eliminate noise, smooth the edges, and enhance the connectivity of the target regions, so as to improve the detection accuracy and reduce the interference of pseudo-defects or isolated pixel points in the image on subsequent analysis, and finally generate a second gray-scale image corresponding to each first gray-scale image.

[0049] S30. Perform wavelet transform and multi-scale wavelet decomposition on multiple second grayscale images, extract the low-frequency sub-images and multiple high-frequency sub-images of each second grayscale image at each scale, traverse each high-frequency sub-image based on a preset feature window, determine multiple feature thresholds for each high-frequency sub-image, and extract the first feature region image of each high-frequency sub-image at each scale based on the feature thresholds.

[0050] In this embodiment, wavelet transform detects regions with gradually changing brightness through multi-scale analysis. It decomposes an image into wavelet coefficients of different scales and directions, and these coefficients can reflect the brightness change information of the image at different scales. Perform wavelet transform on each second grayscale image to decompose the image into different frequency components at multiple scales, obtaining a low-frequency sub-image and multiple high-frequency sub-images. Among them, the low-frequency sub-image represents the low-frequency components of the image, contains most of the energy and structural data of the image, and reflects the overall brightness information of the image. The multiple high-frequency detail sub-images include a horizontal edge feature sub-image, a vertical edge feature sub-image, and a diagonal edge feature sub-image to capture the edge features and detail changes in the horizontal, vertical, and diagonal directions of the image.

[0051] Specifically, the choice of wavelet basis function can be Haar wavelet, Daubechies wavelet, Symlet, etc. In this embodiment, Daubechies wavelet is selected as the wavelet basis function. Daubechies wavelet has good spatio-temporal localization ability and smoothness, and can better capture the brightness gradual change characteristics of the image. During the multi-scale wavelet decomposition process, the number of decomposition layers determines the decomposition degree of the details and low-frequency components of the transformed image. Those skilled in the art can select an appropriate number of decomposition layers (also called the decomposition level or scale number) based on the resolution of the image. For example, select the number of decomposition layers to be 3, so as to obtain the corresponding low-frequency sub-image and multiple high-frequency sub-images for each layer. Traverse each high-frequency sub-image through a preset feature window to determine multiple feature thresholds for each sub-image, specifically to determine the feature thresholds corresponding to each local window, and process each high-frequency sub-image through the feature thresholds to extract the first feature region image of each high-frequency sub-image at each scale. The purpose is to remove insignificant (usually representing noise or irrelevant details) wavelet coefficients, and the ultimate goal is to enhance and highlight the region where the brightness gradually darkens from the center to the surrounding in the image.

[0052] S40. Fuse the multiple first feature region images of each second grayscale image at the same scale to obtain the second feature region image of each second grayscale image at each scale, and perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain the target feature region image corresponding to each second grayscale image.

[0053] In this embodiment, multiple first feature region images of each second grayscale image at the same scale are fused. The purpose is to summarize the prominent brightness changes in each first feature region image and reduce the influence of noise, thereby obtaining the second feature region image of each second grayscale image at each scale.

[0054] Optionally, for multiple first feature region images of each second grayscale image at the same scale, calculate the standard deviation of wavelet coefficients of each first feature region image among the multiple first feature region images. The standard deviation of wavelet coefficients reflects the intensity and distribution of brightness changes in the image within this region. Then calculate the mean of the standard deviations of wavelet coefficients of the multiple first feature region images. The mean of the standard deviations of wavelet coefficients reflects the overall brightness change degree of all feature region images at this scale. Determine the weight of each first feature region image according to the standard deviation of wavelet coefficients and the mean of the standard deviations of wavelet coefficients. Specifically, take the ratio between the standard deviation of wavelet coefficients of the first feature region image and the mean of the standard deviations of wavelet coefficients of the multiple first feature region images as the fusion weight of the first feature region image. The feature region image with a larger standard deviation represents a more significant brightness change, so a higher weight is given during the fusion process. Finally, perform weighted fusion on multiple first feature region images of each second grayscale image at the same scale according to the fusion weight corresponding to each first feature region image, so as to obtain the second feature region image of each second grayscale image at each scale. The fused second feature region image can more accurately reflect the regions with significant brightness changes, especially those regions where there may be a gradual attenuation of brightness. These images retain the feature information of each scale in the multi-scale analysis and can better reflect the potential brightness defects on the surface of the LED special-shaped screen.

[0055] It should be noted that this method of fusing multiple first feature region images pays more attention to the overall features of the image. By calculating the standard deviation of wavelet coefficients of the whole image to allocate the fusion weight, it highlights the global brightness change features of different images, and the weight allocation is more global.

[0056] After completing the fusion of the feature information of each scale in the multi-scale analysis, perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to reconstruct the target feature region image corresponding to each second grayscale image. Some interference data is discarded in the reconstructed target feature region image, which can better reflect the regions where there may be a gradual attenuation of brightness on the surface of the LED special-shaped screen.

[0057] S50. Feature fusion is performed on multiple feature region images to obtain a brightness anomaly image. Multiple target regions are determined from the brightness anomaly image according to the gray value of each pixel point in the brightness anomaly image. A brightness anomaly detection result of the LED special-shaped screen to be detected is generated based on the target pixel points, target line regions, and target regions.

[0058] In this embodiment, feature fusion is performed on multiple target feature region images to obtain a final brightness anomaly image. The brightness anomaly image contains enhanced brightness gradient features, which can highlight all possible brightness anomaly regions. Edge detection is performed according to the gray value of each pixel point in the brightness anomaly image to identify and extract multiple target regions. The target regions represent the regions with brightness anomalies in the image, such as regions with gradually decaying brightness. The combination of target pixel points, target line regions, and target regions is used to generate a brightness anomaly detection result of the LED special-shaped screen to be detected. The brightness anomaly detection result may include information such as the position, shape, and area of each brightness anomaly region in the LED special-shaped screen to be detected, as well as the positions of bright points / dark points and bright lines / dark lines in the LED special-shaped screen to be detected, so as to provide data reference for further repair or maintenance of the LED special-shaped screen to be detected.

[0059] In an alternative implementation, for the first grayscale image and the second grayscale image in step S20, it further includes:

[0060] Edge detection is performed on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image. Among them, the edge pixels in the edge binary image are represented as white (gray value is 1), and non-edge pixels are represented as black (gray value is 0). These edges may correspond to bright lines or dark lines. Line detection is performed on the edge binary image based on the Hough transform to determine multiple target line regions in the edge binary image and map them to the first grayscale image to mark the positions and ranges of these line regions in the first grayscale image. Among them, the multiple target line regions may be bright line regions (high gray value) or dark line regions (low gray value).

[0061] The multiple target line regions mapped to the first grayscale image are divided into a first line region and a second line region according to their gray scale characteristics, representing bright line regions and dark line regions respectively. Among them, the bright line region refers to the lines with relatively high gray values in the image, usually corresponding to the high-brightness boundaries or transitions in the brightness anomaly regions. The dark line region refers to the lines with relatively low gray values in the image, usually corresponding to the dim boundaries or shadow transitions in the brightness anomaly regions. Erosion processing is performed on the bright line region, that is, the first line region, which can shrink these high-brightness edge regions, remove noise and fine high-brightness boundaries, so as to avoid interference with other brightness anomaly regions; dilation processing is performed on the dark line region, that is, the second line region, to fill the dark lines by expanding the brighter parts of the image.

[0062] After morphological processing of the bright lines and dark lines, interference from the bright lines and dark lines in detecting possible bright or dark spots in the image can be avoided. Traverse each high-frequency sub-image based on a preset feature window, determine the maximum and minimum gray values in each preset feature window, and determine multiple target pixel points in each first gray image based on the maximum and minimum gray values. Specifically, the multiple target pixel points include bright or dark spots. Divide the multiple target pixel points into first pixel points and second pixel points, which represent bright spot and dark spot defects respectively. Similarly, perform erosion processing on the bright spots, that is, the first pixel points, to shrink the high-brightness edge region, and perform dilation processing on the dark spots, that is, the second pixel points, to fill the dark spots according to the brighter part of the image. This process helps to enhance the features of the brightness anomaly region and reduce the interference of noise and irrelevant details. The finally generated second gray image more prominently shows the possible brightness anomaly region, providing a more accurate basis for subsequent detection and analysis.

[0063] In an alternative implementation, for step S30, traverse each high-frequency sub-image based on a preset feature window, determine multiple feature thresholds for each high-frequency sub-image, and extract the first feature region image of each high-frequency sub-image at each scale based on the feature thresholds. Specifically, it includes:

[0064] Traverse each high-frequency sub-image based on a preset feature window, calculate the mean and standard deviation of the wavelet coefficients within each preset feature window, and determine the feature thresholds corresponding to multiple pixel points within each preset feature window. The mean and standard deviation reflect the average level and variation range of the image brightness change within the window.

[0065] Among them, for the calculation of the feature threshold: , where in the formula, is the feature threshold corresponding to multiple pixel points within the th preset feature window, is the mean of the wavelet coefficients within the th preset feature window, is the standard deviation of the wavelet coefficients within the th preset feature window, is an adjustment parameter used to adjust the sensitivity of the feature threshold to the standard deviation. The larger the adjustment parameter, the more the feature threshold depends on the amplitude of the brightness change, and it can better capture the regions with significant brightness anomalies.

[0066] After calculating the feature thresholds corresponding to multiple pixel points within each preset feature window, according to the horizontal edge feature sub - graph and vertical edge feature sub - graph in multiple high - frequency sub - graphs, calculate the correction factor for each preset feature window, and based on the correction factor, correct the feature thresholds corresponding to multiple pixel points within each preset feature window to obtain the adaptive thresholds corresponding to multiple pixel points within each preset feature window.

[0067] Among them, the following formula is used to determine the adaptive threshold corresponding to multiple pixel points within each preset feature window: In the formula, is the adaptive threshold corresponding to multiple pixel points within the th preset feature window, is the correction factor of the th preset feature window, is the brightness change intensity factor of the th preset feature window, , is the sum of wavelet coefficients within the th preset feature window in the horizontal edge feature sub - graph, representing the change intensity of brightness in the horizontal and vertical directions, is the sum of wavelet coefficients within the th preset feature window in the vertical edge feature sub - graph, is the maximum value of the brightness change intensity factors in multiple preset feature windows, ensuring that the relative scale of the brightness change intensity factor is between [1, 2], and dynamically adjusting the sensitivity of the threshold.

[0068] For multiple pixel points within any preset feature window in any high - frequency sub - graph, set the wavelet coefficients corresponding to the pixel points whose wavelet coefficients are not greater than the adaptive threshold to 0, so as to extract the first feature region image of each high - frequency sub - graph at each scale based on the adaptive threshold. By correcting to obtain the adaptive threshold, the selection of wavelet coefficients can more accurately reflect the local brightness abnormal region, reduce false detection and missed detection. Since the adaptive threshold takes into account the local gradient change, it can enhance the edge details, make the boundary of the brightness abnormal region clearer, help to accurately locate these regions, and the extracted feature region can better reflect the brightness abnormal region in the image. This method combines multi - scale analysis and adaptive threshold calculation, which helps to improve the accuracy and robustness of brightness abnormal detection.

[0069] In an alternative embodiment, for step S50, determining multiple target regions from the brightness abnormal image according to the gray - scale value of each pixel point in the brightness abnormal image includes:

[0070] Traverse each feature region image based on a preset feature window. During the traversal process, calculate the average gray value of the pixel points within each preset feature window, calculate the standard deviation of the gray values of the pixel points within each preset feature window based on the average value, then calculate the average value of the standard deviations of the gray values of the pixel points within multiple preset feature windows in each feature region image. Then, take the ratio between the average gray value of the pixel points within the preset feature window and the average value of the standard deviations of the gray values of the pixel points within multiple preset feature windows in the feature region image to which the preset feature window belongs as the local weight of the preset feature window. This method pays more attention to the local features of the image. By analyzing the brightness features of each window, it can effectively highlight the regions with local brightness anomalies in the image.

[0071] After determining the local weight of each preset feature window, perform normalization processing on the local weights corresponding to multiple preset feature windows in each feature region image, and perform weighted fusion on multiple preset feature windows at the same position in multiple feature region images based on the normalized local weights to obtain a brightness anomaly image. The brightness anomaly regions (such as regions with gradually changing brightness) in the feature region images corresponding to test patterns with different gray values usually have significant brightness change features (such as a higher standard deviation or contrast). That is, for regions with gradually changing brightness, since the brightness changes significantly in these regions (for example, the brightness gradually weakens from the center to the periphery), their brightness contrast or standard deviation is usually large. When calculating the local weight, these regions will be assigned a larger weight. In other words, the fused image will further magnify these brightness anomaly regions, making it possible to better highlight these regions and making these anomaly regions more clearly visible in the image.

[0072] After fusing to obtain the brightness anomaly image, according to the gray value of each pixel point in the brightness anomaly image, an edge detection algorithm can be used to perform edge detection on the brightness anomaly image. For example, perform edge detection based on the Canny operator to detect the boundaries of the brightness anomaly regions to determine multiple brightness anomaly regions, denoted as target regions.

[0073] An LED special-shaped screen surface defect detection method based on image analysis provided by an embodiment of the present invention obtains multiple first gray images through geometric correction processing on multiple collected test images, performs edge detection and line detection on the first gray images, and effectively eliminates the interference of perspective distortion and noise based on morphological operations. By combining wavelet transform and multi-scale analysis, it accurately extracts and identifies the brightness anomaly regions on the LED special-shaped screen, including regions with gradually changing brightness, bright spots, dark spots, etc., realizing high-precision detection of the brightness anomaly regions on the surface of the LED special-shaped screen. Especially for the detection of regions with gradually changing brightness, the detection effect is remarkable, improving the detection accuracy and enhancing the detection robustness while effectively reducing false detection and missed detection.

[0074] SeeFigure 2 , an embodiment of the present invention provides a surface defect detection device for an LED special-shaped screen based on image analysis, which is used to implement the above-mentioned surface defect detection method for an LED special-shaped screen based on image analysis, and includes:

[0075] A data acquisition module, configured to acquire image data of the LED special-shaped screen to be detected under a brightness test pattern and construct a dataset to be detected including multiple test images;

[0076] A geometric correction module, configured to perform geometric correction on multiple test images in the dataset to be detected according to the three-dimensional structure parameters of the LED special-shaped screen to be detected, and perform gray-scale conversion on the images obtained after geometric correction to obtain multiple first gray-scale images;

[0077] A gray-scale image processing module, configured to perform line detection on multiple first gray-scale images based on the Hough transform to determine multiple target line regions, traverse each first gray-scale image based on a preset feature window to determine multiple target pixel points in the first gray-scale image, and perform processing on multiple target line regions and multiple target pixel points based on morphological operations to obtain a second gray-scale image corresponding to each first gray-scale image;

[0078] Specifically, perform edge detection on each first gray-scale image based on the Canny operator to obtain an edge binary image corresponding to each first gray-scale image, perform line detection on the edge binary image based on the Hough transform to determine multiple target line regions in the edge binary image and map them to the first gray-scale image, divide the multiple target line regions in the first gray-scale image into a first line region and a second line region, perform erosion processing on the first line region, and perform dilation processing on the second line region;

[0079] Traverse each high-frequency sub-image based on a preset feature window to determine the maximum and minimum gray-scale values in each preset feature window, determine multiple target pixel points in each first gray-scale image based on the maximum and minimum gray-scale values, divide the multiple target pixel points into first pixel points and second pixel points, perform erosion processing on the first pixel points, and perform dilation processing on the second pixel points to obtain a second gray-scale image corresponding to each first gray-scale image.

[0080] A feature region extraction module, configured to perform wavelet transform and multi-scale wavelet decomposition on multiple second gray-scale images, extract the low-frequency sub-image and multiple high-frequency sub-images of each second gray-scale image at each scale, traverse each high-frequency sub-image based on a preset feature window to determine multiple feature thresholds of each high-frequency sub-image, and extract the first feature region image of each high-frequency sub-image at each scale based on the feature thresholds;

[0081] Specifically, traverse each high-frequency subgraph based on a preset feature window, calculate the mean and standard deviation of wavelet coefficients within each preset feature window, and determine the feature threshold corresponding to multiple pixel points within each preset feature window;

[0082] Among them, , in the formula, is the feature threshold corresponding to multiple pixel points within the th preset feature window, is the mean of wavelet coefficients within the th preset feature window, is the standard deviation of wavelet coefficients within the th preset feature window, is the adjustment parameter;

[0083] According to the horizontal edge feature subgraph and vertical edge feature subgraph in multiple high-frequency subgraphs, calculate the correction factor of each preset feature window, and correct the feature threshold corresponding to multiple pixel points within each preset feature window based on the correction factor to obtain the adaptive threshold corresponding to multiple pixel points within each preset feature window. Among them, the following formula is used to determine the adaptive threshold corresponding to multiple pixel points within each preset feature window: In the formula, is the adaptive threshold corresponding to multiple pixel points within the th preset feature window, is the correction factor of the th preset feature window, is the brightness change intensity factor of the th preset feature window, , is the sum of wavelet coefficients within the th preset feature window in the horizontal edge feature subgraph, is the sum of wavelet coefficients within the th preset feature window in the vertical edge feature subgraph, is the maximum value of the brightness change intensity factor among multiple preset feature windows;

[0084] For multiple pixel points within any preset feature window in any high-frequency subgraph, set the wavelet coefficients corresponding to the pixel points whose wavelet coefficients are not greater than the adaptive threshold to 0, so as to extract the first feature region image of each high-frequency subgraph at each scale based on the adaptive threshold.

[0085] The feature region fusion module is used to fuse multiple first feature region images of each second grayscale image at the same scale to obtain second feature region images of each second grayscale image at each scale, and perform inverse wavelet transform on the low-frequency sub-images and the second feature region images of each second grayscale image at each scale to obtain the target feature region images corresponding to each second grayscale image;

[0086] The brightness anomaly detection module is used to perform feature fusion on multiple feature region images to obtain a brightness anomaly image, determine multiple target regions from the brightness anomaly image according to the grayscale values of each pixel point in the brightness anomaly image, and generate a brightness anomaly detection result of the LED special-shaped screen to be detected based on the target pixel points, target line regions, and target regions.

[0087] The above are only specific implementation manners of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The parts not described in detail in this specification belong to the prior art well-known to those skilled in the art.

Claims

1. A method for detecting surface defects of LED special-shaped screens based on image analysis, characterized in that: include: Collect image data of the special-shaped LED screen to be detected under the brightness test pattern and construct a data set to be detected containing multiple test images; According to the three-dimensional structural parameters of the special-shaped LED screen to be detected, geometric correction is performed on the data set to be detected, and grayscale conversion is performed on the image obtained after the geometric correction to obtain a plurality of first grayscale images; Performing line detection on the multiple first grayscale images based on Hough transform to determine multiple target line regions, traversing each first grayscale image based on a preset feature window to determine multiple target pixel points in the first grayscale image, and processing the multiple target line regions and the multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; Performing wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, and extracting a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image; Traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature threshold, including traversing each high-frequency sub-image based on the preset feature window, calculating the mean and standard deviation of the wavelet coefficients in each preset feature window, and determining the feature thresholds corresponding to multiple pixel points in each preset feature window; According to the horizontal edge feature sub-image and the vertical edge feature sub-image in the multiple high-frequency sub-images, the correction factor of each preset feature window is calculated, and the feature threshold is corrected based on the correction factor to obtain the adaptive threshold corresponding to the multiple pixel points in each preset feature window, where: In the formula, A j is the adaptive threshold corresponding to multiple pixels in the j-th preset feature window, is the correction factor of the jth preset feature window, G j is the brightness change intensity factor of the j-th preset feature window, L x is the sum of the wavelet coefficients in the jth preset feature window in the horizontal edge feature sub-image, L y is the sum of the wavelet coefficients in the jth preset feature window in the longitudinal edge feature sub-image, G max is the maximum value of the brightness change intensity factor in multiple preset feature windows; For a plurality of pixels in any preset feature window in any high-frequency sub-image, the wavelet coefficients corresponding to the pixels whose wavelet coefficients are not greater than the adaptive threshold are set to 0, thereby obtaining the first feature region image of each high-frequency sub-image at each scale based on the adaptive threshold extraction; Fusing multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image at each scale of each second grayscale image; Performing inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; Perform feature fusion on multiple feature area images to obtain a brightness abnormality image, and determine multiple target areas from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image; The brightness anomaly detection result of the LED special-shaped screen to be detected is generated according to the target pixel point, target line area and target area.

2. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: For the first grayscale image and the second grayscale image, the method further includes: Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform, determining multiple target line regions in the edge binary image and mapping them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing an erosion process on the first line region, and performing an expansion process on the second line region; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

3. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: A plurality of first feature region images at the same scale of each second grayscale image are fused to obtain a second feature region image at each scale of each second grayscale image, including: For each second grayscale image at the same scale of multiple first feature area images, the wavelet coefficient standard deviation of each first feature area image in the multiple first feature area images is calculated, the mean of the wavelet coefficient standard deviations of the multiple first feature area images is calculated, the ratio between the wavelet coefficient standard deviation of the first feature area image and the mean of the wavelet coefficient standard deviations of the multiple first feature area images is used as the fusion weight of the first feature area image, and based on the fusion weight corresponding to each first feature area image, the multiple first feature area images at the same scale of each second grayscale image are weightedly fused to obtain the second feature area image of each second grayscale image at each scale.

4. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: The brightness abnormality image is obtained by performing feature fusion on multiple feature region images, and multiple target regions are determined from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image, including: Traversing each feature region image based on a preset feature window, calculating an average value of the grayscale values ​​of the pixels in each preset feature window, calculating a standard deviation of the grayscale values ​​of the pixels in each preset feature window, calculating the average value of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in each feature region image, taking a ratio between an average value of the grayscale values ​​of the pixels in the preset feature window and an average value of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in the feature region image to which the preset feature window belongs as a local weight of the preset feature window, normalizing the local weights corresponding to the multiple preset feature windows in each feature region image, and performing weighted fusion on the multiple preset feature windows at the same position in the multiple feature region images based on the normalized local weights to obtain a brightness abnormality image; The brightness abnormality image is subjected to edge detection according to the gray value of each pixel in the brightness abnormality image to determine multiple brightness abnormality areas, which are recorded as target areas.

5. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: For the calculation of feature thresholds, it also includes: C i =u i +εδ i , where C i is the feature threshold corresponding to multiple pixels in the i-th preset feature window, u i is the mean value of the wavelet coefficients in the i-th preset feature window, δ i is the standard deviation of the wavelet coefficients in the i-th preset feature window, and ε is the adjustment parameter.

6. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 4 is characterized in that: The Daubechies wavelet is used as the wavelet basis function to perform wavelet transform on multiple second grayscale images.

7. A device for detecting surface defects of LED special-shaped screens based on image analysis, characterized in that: The device is used to implement the method for detecting surface defects of LED special-shaped screens based on image analysis as described in any one of claims 1 to 6, comprising: A data acquisition module is used to collect image data of the special-shaped LED screen to be detected under a brightness test pattern and construct a data set to be detected containing multiple test images; A geometric correction module, used for performing geometric correction on a plurality of test images in the detection data set according to the three-dimensional structural parameters of the LED special-shaped screen to be detected, and performing grayscale conversion on the images obtained after the geometric correction to obtain a plurality of first grayscale images; A grayscale image processing module is used to perform line detection on multiple first grayscale images based on Hough transform to determine multiple target line areas, traverse each first grayscale image based on a preset feature window, determine multiple target pixel points in the first grayscale image, and process the multiple target line areas and multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; A feature region extraction module is used to perform wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, extract a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image, traverse each high-frequency sub-image based on a preset feature window, determine a plurality of feature thresholds of each high-frequency sub-image, and extract a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds; A feature region fusion module is used to fuse multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image of each second grayscale image at each scale, and to perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; The brightness anomaly detection module is used to perform feature fusion on multiple feature area images to obtain a brightness anomaly image, determine multiple target areas from the brightness anomaly image according to the grayscale value of each pixel in the brightness anomaly image, and generate brightness anomaly detection results of the LED special-shaped screen to be detected according to the target pixel points, target line areas and target areas.

8. The device for detecting surface defects of LED special-shaped screens based on image analysis according to claim 7, characterized in that: For the grayscale image processing module, it also includes: Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform, determining multiple target line regions in the edge binary image and mapping them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing an erosion process on the first line region, and performing an expansion process on the second line region; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

Images