LED special-shaped screen surface defect detection method and device based on image analysis

Abstract

The invention provides an LED special-shaped screen surface defect detection method and device based on image analysis, and relates to the technical field of image processing. The LED special-shaped screen surface defect detection device based on image analysis comprises a data acquisition module, a geometric correction module, a grayscale image processing module, a feature region extraction module, a feature region fusion module and a brightness anomaly detection module. According to the invention, geometric correction processing is carried out on a plurality of collected test images to obtain a plurality of first grayscale images, edge detection and line detection are carried out on the first grayscale images, and visual angle deformation and noise interference are effectively eliminated based on morphological operation. According to the method, the brightness abnormal area on the LED special-shaped screen is accurately extracted and recognized in combination with wavelet transform and multi-scale analysis, high-precision detection of the brightness abnormal area on the surface of the LED special-shaped screen is achieved, particularly, the detection effect on the brightness gradient area is remarkable, and false detection and missing detection are effectively reduced while the detection precision is improved and the detection robustness is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for detecting surface defects of LED special-shaped screens based on image analysis. Background Art

[0002] With the increasing demand for personalized display, LED special-shaped screens have gradually become a hot spot in the market. These LED special-shaped screens not only have the display functions of conventional LED screens, but can also be customized according to different scenes and needs, showing a more unique visual effect.

[0003] Due to the complex structure and irregular arrangement of modules of LED special-shaped screens, surface defects are more likely to occur during manufacturing and use. Affected by various factors, such as the aging of LED lamp beads themselves, failure of the drive circuit or uneven heat dissipation, the brightness of LED lamp beads in a certain area of ​​the LED special-shaped screen may gradually decrease from the center to the periphery, forming a visually blurred brightness gradient area. This abnormal brightness area is an area where the brightness gradually changes from the center to the edge, and usually has no obvious boundaries. The blurred boundary features make it difficult for conventional edge detection and simple threshold segmentation methods to accurately identify brightness defect areas. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method and device for detecting surface defects of LED special-shaped screens based on image analysis, which can effectively extract brightness defect areas with gradually decaying brightness, and significantly improve the accuracy of surface defect detection of LED special-shaped screens.

[0005] To achieve the above-mentioned object, the first aspect of the present invention provides a method for detecting surface defects of LED special-shaped screens based on image analysis, comprising: Collect image data of the special-shaped LED screen to be detected under the brightness test pattern and construct a data set to be detected containing multiple test images; According to the three-dimensional structural parameters of the special-shaped LED screen to be detected, a plurality of test images in the detection data set are geometrically corrected, and the images obtained after the geometric correction are gray-scale converted to obtain a plurality of first gray-scale images; Performing line detection on the multiple first grayscale images based on Hough transform to determine multiple target line regions, traversing each first grayscale image based on a preset feature window to determine multiple target pixel points in the first grayscale image, and processing the multiple target line regions and the multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; Performing wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, and extracting a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image; Traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds; Fusing multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image at each scale of each second grayscale image; Performing inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; Perform feature fusion on multiple feature area images to obtain a brightness abnormality image, and determine multiple target areas from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image; The brightness anomaly detection result of the LED special-shaped screen to be detected is generated according to the target pixel point, target line area and target area.

[0006] Preferably, for the first grayscale image and the second grayscale image, the method further includes: Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform, determining multiple target line regions in the edge binary image and mapping them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing an erosion process on the first line region, and performing an expansion process on the second line region; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

[0007] Preferably, traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds include: Traversing each high-frequency sub-image based on a preset feature window, calculating the mean and standard deviation of the wavelet coefficients in each preset feature window, and determining the feature threshold corresponding to multiple pixel points in each preset feature window; in, , where For the The feature threshold corresponding to multiple pixels in a preset feature window, For the The mean value of the wavelet coefficients in a preset feature window, For the The standard deviation of the wavelet coefficients within a preset feature window, To adjust the parameters; According to the horizontal edge feature sub-image and the vertical edge feature sub-image in the multiple high-frequency sub-images, the correction factor of each preset feature window is calculated, and the feature threshold corresponding to the multiple pixel points in each preset feature window is corrected based on the correction factor to obtain the adaptive threshold corresponding to the multiple pixel points in each preset feature window, wherein the adaptive threshold corresponding to the multiple pixel points in each preset feature window is determined by the following formula: In the formula, For the The adaptive threshold corresponding to multiple pixels in a preset feature window, For the Correction factor for a preset characteristic window, For the The brightness change intensity factor of a preset feature window, , is the first The sum of the wavelet coefficients in a preset feature window, is the first The sum of the wavelet coefficients in a preset feature window, is the maximum value of the brightness change intensity factor in multiple preset feature windows; For multiple pixels in any preset feature window in any high-frequency sub-image, the wavelet coefficients corresponding to the pixels whose wavelet coefficients are not greater than the adaptive threshold are set to 0, thereby obtaining the first feature area image of each high-frequency sub-image at each scale based on the adaptive threshold extraction.

[0008] Preferably, a plurality of first feature region images at the same scale of each second grayscale image are fused to obtain a second feature region image at each scale of each second grayscale image, including: For each second grayscale image at the same scale of multiple first feature area images, the wavelet coefficient standard deviation of each first feature area image in the multiple first feature area images is calculated, the mean of the wavelet coefficient standard deviations of the multiple first feature area images is calculated, the ratio between the wavelet coefficient standard deviation of the first feature area image and the mean of the wavelet coefficient standard deviations of the multiple first feature area images is used as the fusion weight of the first feature area image, and based on the fusion weight corresponding to each first feature area image, the multiple first feature area images at the same scale of each second grayscale image are weightedly fused to obtain the second feature area image of each second grayscale image at each scale.

[0009] Preferably, feature fusion is performed on multiple feature region images to obtain a brightness abnormality image, and multiple target regions are determined from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image, including: Traversing each feature region image based on a preset feature window, calculating an average value of the grayscale values ​​of the pixels in each preset feature window, calculating a standard deviation of the grayscale values ​​of the pixels in each preset feature window, calculating the average value of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in each feature region image, taking a ratio between an average value of the grayscale values ​​of the pixels in the preset feature window and an average value of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in the feature region image to which the preset feature window belongs as a local weight of the preset feature window, normalizing the local weights corresponding to the multiple preset feature windows in each feature region image, and performing weighted fusion on the multiple preset feature windows at the same position in the multiple feature region images based on the normalized local weights to obtain a brightness abnormality image; The brightness abnormality image is subjected to edge detection according to the gray value of each pixel in the brightness abnormality image to determine multiple brightness abnormality areas, which are recorded as target areas.

[0010] Preferably, wavelet transform processing is performed on the plurality of second grayscale images using Daubechies wavelet as a wavelet basis function.

[0011] The second aspect of the present invention provides an LED special-shaped screen surface defect detection device based on image analysis, which is used to implement the above-mentioned LED special-shaped screen surface defect detection method based on image analysis, comprising: A data acquisition module is used to collect image data of the special-shaped LED screen to be detected under a brightness test pattern and construct a data set to be detected containing multiple test images; A geometric correction module, used for performing geometric correction on a plurality of test images in the detection data set according to the three-dimensional structural parameters of the LED special-shaped screen to be detected, and performing grayscale conversion on the images obtained after the geometric correction to obtain a plurality of first grayscale images; A grayscale image processing module is used to perform line detection on multiple first grayscale images based on Hough transform to determine multiple target line areas, traverse each first grayscale image based on a preset feature window, determine multiple target pixel points in the first grayscale image, and process the multiple target line areas and multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; A feature region extraction module is used to perform wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, extract a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image, traverse each high-frequency sub-image based on a preset feature window, determine a plurality of feature thresholds of each high-frequency sub-image, and extract a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds; A feature region fusion module is used to fuse multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image of each second grayscale image at each scale, and to perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; The brightness anomaly detection module is used to perform feature fusion on multiple feature area images to obtain a brightness anomaly image, determine multiple target areas from the brightness anomaly image according to the grayscale value of each pixel in the brightness anomaly image, and generate brightness anomaly detection results of the LED special-shaped screen to be detected according to the target pixel points, target line areas and target areas.

[0012] Preferably, the grayscale image processing module further includes: Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform, determining multiple target line regions in the edge binary image and mapping them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing an erosion process on the first line region, and performing an expansion process on the second line region; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

[0013] The present invention has the following beneficial effects: The present invention obtains multiple first grayscale images by performing geometric correction processing on multiple collected test images, performs edge detection and line detection on the first grayscale images, and effectively eliminates the interference of perspective deformation and noise based on morphological operations, and combines wavelet transform and multi-scale analysis to accurately extract and identify abnormal brightness areas on the LED special-shaped screen, including areas with gradually changing brightness, bright spots, dark spots, etc., thereby achieving high-precision detection of abnormal brightness areas on the surface of the LED special-shaped screen, especially the detection effect on the brightness gradient area is significant, which improves the detection accuracy and enhances the detection robustness while effectively reducing false detections and missed detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic flow chart of a method for detecting surface defects of LED special-shaped screens based on image analysis provided in an embodiment of the present invention.

[0015] Figure 2 A schematic structural diagram of a device for detecting surface defects of LED special-shaped screens based on image analysis provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make those skilled in the art better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0017] See also Figure 1 The embodiment of the present invention provides a method for detecting surface defects of LED special-shaped screens based on image analysis, comprising the following steps: S10, collecting image data of the LED special-shaped screen to be detected under the brightness test pattern and constructing a data set to be detected containing multiple test images, performing geometric correction on the multiple test images in the data set to be detected according to the three-dimensional structural parameters of the LED special-shaped screen to be detected, and performing grayscale conversion on the images obtained after the geometric correction to obtain multiple first grayscale images.

[0018] In this embodiment, a high-resolution camera or image sensor device is used to shoot the LED special-shaped screen to be detected under a preset brightness test pattern, wherein the brightness test pattern can be a series of pure color patterns with different grayscale values, such as 0% (black), 25%, 50%, 75%, and 100% (white), so as to effectively stimulate the brightness characteristics of the LED special-shaped screen to be detected, thereby acquiring a series of test images, and constructing a data set to be detected containing multiple test images. Since the LED special-shaped screen may have a complex geometric shape, such as an arc, a sphere, or other irregular shapes, geometric correction of the test image can convert these images into a standard two-dimensional plane projection, thereby eliminating the deformation caused by the viewing angle and perspective effect in the image. Geometric correction usually uses image registration technology or camera calibration method to correct the geometric shape of the image by matching control points or feature points, and specifically implements the correction process based on the three-dimensional structural parameters of the LED special-shaped screen (such as curvature, bending angle, module arrangement, etc.). Geometric correction is a technical means well known to those skilled in the art and will not be repeated here. The image obtained after geometric correction is subjected to grayscale conversion to generate a plurality of first grayscale images, so as to simplify the color information of each pixel in the image into a single brightness value, which is convenient for subsequent image processing and analysis.

[0019] S20. Perform line detection on multiple first grayscale images based on Hough transform to determine multiple target line areas, traverse each first grayscale image based on a preset feature window to determine multiple target pixel points in the first grayscale image, and process the multiple target line areas and multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image.

[0020] In this embodiment, Hough transform is a commonly used line detection algorithm. By mapping the straight line in the image space to the parameter space, the straight line features in the image can be effectively detected. Hough transform is performed on each first grayscale image to detect the line structure in the image. The detected target line area may represent the boundary between LED modules, the edge of the defect area, such as bright lines and dark lines, etc. Based on the preset feature window, the multiple target pixel points determined by traversing each first grayscale image may represent potential brightness abnormal areas, such as bright spots and dark spots, etc., wherein the preset feature window is a rectangular area of ​​a fixed size (for example, 3×3 or 5×5). Those skilled in the art can select a suitable window size according to actual needs, and this embodiment does not specifically limit it. Using morphological operations (such as corrosion, expansion, opening operation, closing operation, etc.) to process the detected multiple target line areas and target pixel points can eliminate noise, smooth edges, and enhance the connectivity of the target area to improve the detection accuracy, reduce the interference of pseudo defects or isolated pixels in the image on subsequent analysis, and finally generate a second grayscale image corresponding to each first grayscale image.

[0021] S30, performing wavelet transform and multi-scale wavelet decomposition on the multiple second grayscale images, extracting a low-frequency sub-image and multiple high-frequency sub-images at each scale of each second grayscale image, traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds.

[0022] In this embodiment, the wavelet transform detects the area where the brightness changes gradually through multi-scale analysis. It decomposes the image into wavelet coefficients of different scales and directions. These coefficients can reflect the brightness change information of the image at different scales. Wavelet transform is performed on each second grayscale image to decompose the image into different frequency components at multiple scales to obtain a low-frequency sub-image and multiple high-frequency sub-images, wherein the low-frequency sub-image represents the low-frequency component of the image, contains most of the energy and structural data of the image, and reflects the overall brightness information of the image. The multiple high-frequency detail sub-images include a horizontal edge feature sub-image, a vertical edge feature sub-image, and a diagonal edge feature sub-image to capture the edge features and detail changes in the horizontal, vertical, and diagonal directions of the image.

[0023] Specifically, the wavelet basis function can be Haar wavelet, Daubechies wavelet, Symlet, etc. In this embodiment, Daubechies wavelet is selected as the wavelet basis function. Daubechies wavelet has good spatiotemporal localization ability and smoothness, and can better capture the brightness gradient characteristics of the image. In the multi-scale wavelet decomposition process, the number of decomposition layers determines the degree of decomposition of the transformed image details and low-frequency components. Those skilled in the art can select a suitable number of decomposition layers (also called decomposition level or scale number) based on the resolution of the image, for example, select the number of decomposition layers as 3, so as to obtain a low-frequency sub-image and multiple high-frequency sub-images corresponding to each layer. Through a preset feature window, each high-frequency sub-image is traversed to determine multiple feature thresholds of each sub-image, specifically, to determine the feature thresholds corresponding to each local window, and each high-frequency sub-image is processed through the feature threshold to extract the first feature area image of each high-frequency sub-image at each scale, in order to remove insignificant (usually representing noise or irrelevant details) wavelet coefficients, and the ultimate goal is to enhance and highlight the area in the image where the brightness gradually darkens from the center to the surroundings.

[0024] S40. Fuse the multiple first feature area images at the same scale of each second grayscale image to obtain the second feature area image of each second grayscale image at each scale, and perform inverse wavelet transform on the low-frequency subimage and the second feature area image of each second grayscale image at each scale to obtain the target feature area image corresponding to each second grayscale image.

[0025] In this embodiment, multiple first feature region images at the same scale of each second grayscale image are fused, with the purpose of summarizing the brightness changes highlighted in each first feature region image to reduce the influence of noise, and obtaining the second feature region image of each second grayscale image at each scale.

[0026] Optionally, for each second grayscale image at the same scale of multiple first feature region images, the wavelet coefficient standard deviation of each first feature region image is calculated, and the wavelet coefficient standard deviation reflects the intensity and distribution of the brightness change of the image in the region; and the mean of the wavelet coefficient standard deviation of the multiple first feature region images is calculated, and the mean of the wavelet coefficient standard deviation reflects the overall brightness change degree of all feature region images at the scale. The weight of each first feature region image is determined according to the standard deviation of the wavelet coefficient and the mean of the wavelet coefficient standard deviation. Specifically, the ratio between the wavelet coefficient standard deviation of the first feature region image and the mean of the wavelet coefficient standard deviation of the multiple first feature region images is used as the fusion weight of the first feature region image. The feature region image with a larger standard deviation represents a more significant brightness change, and therefore is given a higher weight in the fusion process. Finally, according to the fusion weight corresponding to each first feature region image, the multiple first feature region images at the same scale of each second grayscale image are weightedly fused to obtain the second feature region image of each second grayscale image at each scale. The fused second feature area images more accurately reflect the areas with significant brightness changes, especially those areas where there may be gradual brightness decay. These images retain the feature information of each scale in the multi-scale analysis and can better reflect the potential brightness defects on the surface of LED special-shaped screens.

[0027] It is worth noting that this method of fusing multiple first feature area images pays more attention to the overall characteristics of the image. The fusion weight is allocated by calculating the standard deviation of the wavelet coefficients of the entire image, highlighting the global brightness change characteristics of different images, and the weight allocation is more global.

[0028] After completing the fusion of feature information at each scale in the multi-scale analysis, an inverse wavelet transform is performed on the low-frequency sub-image and the second feature area image of each second grayscale image at each scale to reconstruct the target feature area image corresponding to each second grayscale image. The reconstructed target feature area image discards some interference data, which can better reflect the area with gradually attenuated brightness that may exist on the surface of the LED special-shaped screen.

[0029] S50, feature fusion is performed on multiple feature area images to obtain a brightness abnormality image, multiple target areas are determined from the brightness abnormality image according to the grayscale value of each pixel in the brightness abnormality image, and a brightness abnormality detection result of the LED special-shaped screen to be detected is generated according to the target pixel points, target line areas and target areas.

[0030] In this embodiment, feature fusion is performed on multiple target feature area images to obtain a final brightness abnormality image, which contains enhanced brightness gradient features and can highlight all possible brightness abnormality areas. Edge detection is performed based on the grayscale value of each pixel in the brightness abnormality image to identify and extract multiple target areas. The target area represents the area with abnormal brightness in the image, such as the area where the brightness gradually decays. The target pixel point, the target line area and the target area are combined to generate the brightness abnormality detection result of the LED special-shaped screen to be detected. The brightness abnormality detection result may include the position, shape, area and other information of each brightness abnormality area in the LED special-shaped screen to be detected, as well as the position of the bright spot / dark spot and bright line / dark line in the LED special-shaped screen to be detected, so as to provide data reference for further repair or maintenance of the LED special-shaped screen to be detected.

[0031] In an optional implementation, for the first grayscale image and the second grayscale image in step S20, the method further includes: Based on the Canny operator, edge detection is performed on each first grayscale image to obtain an edge binary image corresponding to each first grayscale image, wherein edge pixels in the edge binary image are represented as white (grayscale value is 1), and non-edge pixels are represented as black (grayscale value is 0), and these edges may correspond to bright lines or dark lines. Based on the Hough transform, line detection is performed on the edge binary image to determine multiple target line areas in the edge binary image and map them to the first grayscale image to mark the positions and ranges of these line areas in the first grayscale image, wherein the multiple target line areas may be bright line areas (high grayscale value) or dark line areas (low grayscale value).

[0032] The multiple target line regions mapped to the first grayscale image are divided into a first line region and a second line region according to their grayscale features, representing a bright line region and a dark line region, respectively, wherein the bright line region refers to a line with a higher grayscale value in the image, usually corresponding to a high-brightness boundary or transition in a brightness abnormality region, and the dark line region refers to a line with a lower grayscale value in the image, usually corresponding to a dim boundary or shadow transition in a brightness abnormality region. The bright line region, i.e., the first line region, is subjected to corrosion processing to reduce these high-brightness edge regions, remove noise and small high-brightness boundaries, so as to avoid interference with other brightness abnormality regions; the dark line region, i.e., the second line region, is subjected to dilation processing to fill the dark lines by expanding the higher-brightness portion of the image.

[0033] After morphological processing of bright lines and dark lines, interference of bright lines and dark lines on the detection of bright spots or dark spots that may exist in the image can be avoided. Based on the preset feature window, each high-frequency sub-image is traversed, and the maximum and minimum values ​​of the grayscale values ​​in each preset feature window are determined. Based on the maximum and minimum values ​​of the grayscale values, multiple target pixels in each first grayscale image are determined. Specifically, the multiple target pixels contain bright spots or dark spots. The multiple target pixels are divided into first pixels and second pixels, representing bright spots and dark spots defects respectively. Similarly, the bright spots, i.e., the first pixels, are corroded to reduce the high-brightness edge areas, and the dark spots, i.e., the second pixels, are expanded to fill the dark spots according to the brighter parts of the image. This process helps to enhance the features of the brightness abnormality areas and reduce the interference of noise and irrelevant details. The second grayscale image finally generated further highlights the possible brightness abnormality areas, providing a more accurate basis for subsequent detection and analysis.

[0034] In an optional implementation, for step S30, traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature threshold, specifically includes: Based on the preset feature window, each high-frequency sub-image is traversed, the mean and standard deviation of the wavelet coefficients in each preset feature window are calculated, and the feature thresholds corresponding to multiple pixels in each preset feature window are determined. The mean and standard deviation reflect the average level and amplitude of the image brightness change in the window.

[0035] Among them, for the calculation of feature threshold: , where For the The feature threshold corresponding to multiple pixels in a preset feature window, For the The mean value of the wavelet coefficients in a preset feature window, For the The standard deviation of the wavelet coefficients within a preset feature window, It is an adjustment parameter used to adjust the sensitivity of the feature threshold to the standard deviation. The larger the adjustment parameter, the more dependent the feature threshold will be on the amplitude of brightness change, and the better it can capture areas with significant brightness anomalies.

[0036] After calculating the feature thresholds corresponding to multiple pixel points in each preset feature window, the correction factor of each preset feature window is calculated according to the horizontal edge feature sub-image and the vertical edge feature sub-image in the multiple high-frequency sub-images, and the feature thresholds corresponding to the multiple pixel points in each preset feature window are corrected based on the correction factors to obtain the adaptive thresholds corresponding to the multiple pixel points in each preset feature window.

[0037] The following formula is used to determine the adaptive threshold corresponding to multiple pixels in each preset feature window: In the formula, For the The adaptive threshold corresponding to multiple pixels in a preset feature window, For the Correction factor for a preset characteristic window, For the The brightness change intensity factor of a preset feature window, , is the first The sum of the wavelet coefficients in the preset feature windows represents the intensity of brightness changes in the horizontal and vertical directions. is the first The sum of the wavelet coefficients in a preset feature window, It is the maximum value of the brightness change intensity factor in multiple preset feature windows, ensuring that the relative scale of the brightness change intensity factor is between [1,2], and dynamically adjusting the sensitivity of the threshold.

[0038] For multiple pixels in any preset feature window in any high-frequency sub-image, the wavelet coefficients corresponding to the pixels whose wavelet coefficients are not greater than the adaptive threshold are set to 0, so as to extract the first feature region image of each high-frequency sub-image at each scale based on the adaptive threshold. By correcting the adaptive threshold, the selection of wavelet coefficients can more accurately reflect the local brightness abnormality area, reduce false detection and missed detection, and because the adaptive threshold takes into account the local gradient changes, it can enhance the edge details, make the boundaries of the brightness abnormality area clearer, and help to accurately locate these areas. The extracted feature area can better reflect the brightness abnormality area in the image. This method combines multi-scale analysis and adaptive threshold calculation, which helps to improve the accuracy and robustness of brightness anomaly detection.

[0039] In an optional implementation, for step S50, multiple target areas are determined from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image, including: Based on the preset feature window, each feature region image is traversed. During the traversal process, the average grayscale value of the pixels in each preset feature window is calculated, and the standard deviation of the grayscale value of the pixels in each preset feature window is calculated based on the average value. Then, the mean of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in each feature region image is calculated, and then the ratio between the average grayscale value of the pixels in the preset feature window and the mean of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in the feature region image to which the preset feature window belongs is used as the local weight of the preset feature window. This method pays more attention to the local features of the image. By analyzing the brightness features of each window, it can effectively highlight the areas with abnormal local brightness in the image.

[0040] After determining the local weight of each preset feature window, the local weights corresponding to multiple preset feature windows in each feature region image are normalized, and multiple preset feature windows at the same position in multiple feature region images are weighted fused based on the normalized local weights to obtain a brightness abnormality image. Brightness abnormality areas (e.g., areas with gradually changing brightness) in the feature region images corresponding to the test patterns with different grayscale values ​​usually have significant brightness change characteristics (e.g., higher standard deviation or contrast), that is, areas with gradually changing brightness. Since the brightness changes in these areas are more significant (e.g., the brightness gradually decreases from the center to the surroundings), their brightness contrast or standard deviation is usually larger. When calculating the local weights, these areas will be assigned larger weights. In other words, the fused image will further magnify these brightness abnormal areas, which can better highlight these areas and make these abnormal areas more clearly visible in the image.

[0041] After the brightness abnormality image is obtained through fusion, an edge detection algorithm can be used to perform edge detection on the brightness abnormality image according to the grayscale value of each pixel in the brightness abnormality image. For example, edge detection can be performed based on the Canny operator to detect the boundary of the brightness abnormality area, so as to determine multiple brightness abnormality areas, which are recorded as target areas.

[0042] An embodiment of the present invention provides a method for detecting surface defects of LED special-shaped screens based on image analysis. The method performs geometric correction processing on multiple collected test images to obtain multiple first grayscale images, performs edge detection and line detection on the first grayscale images, and effectively eliminates the interference of perspective deformation and noise based on morphological operations. The method combines wavelet transform and multi-scale analysis to accurately extract and identify abnormal brightness areas on the LED special-shaped screen, including areas with gradually changing brightness, bright spots, dark spots, etc., thereby achieving high-precision detection of abnormal brightness areas on the surface of the LED special-shaped screen, especially the detection effect on areas with gradually changing brightness is significant, which improves the detection accuracy and enhances the detection robustness while effectively reducing false detections and missed detections.

[0043] See also Figure 2 The embodiment of the present invention provides an LED special-shaped screen surface defect detection device based on image analysis, which is used to implement the above-mentioned LED special-shaped screen surface defect detection method based on image analysis, including: A data acquisition module is used to collect image data of the special-shaped LED screen to be detected under a brightness test pattern and construct a data set to be detected containing multiple test images; A geometric correction module, used for performing geometric correction on a plurality of test images in the detection data set according to the three-dimensional structural parameters of the LED special-shaped screen to be detected, and performing grayscale conversion on the images obtained after the geometric correction to obtain a plurality of first grayscale images; A grayscale image processing module is used to perform line detection on multiple first grayscale images based on Hough transform to determine multiple target line areas, traverse each first grayscale image based on a preset feature window, determine multiple target pixel points in the first grayscale image, and process the multiple target line areas and multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; Specifically, edge detection is performed on each first grayscale image based on a Canny operator to obtain an edge binary image corresponding to each first grayscale image, line detection is performed on the edge binary image based on a Hough transform, multiple target line regions in the edge binary image are determined and mapped to the first grayscale image, the multiple target line regions in the first grayscale image are divided into a first line region and a second line region, the first line region is eroded, and the second line region is expanded; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

[0044] A feature region extraction module is used to perform wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, extract a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image, traverse each high-frequency sub-image based on a preset feature window, determine a plurality of feature thresholds of each high-frequency sub-image, and extract a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds; Specifically, traverse each high-frequency sub-image based on a preset feature window, calculate the mean and standard deviation of the wavelet coefficients in each preset feature window, and determine the feature threshold corresponding to multiple pixel points in each preset feature window; in, , where For the The feature threshold corresponding to multiple pixels in a preset feature window, For the The mean value of the wavelet coefficients in a preset feature window, For the The standard deviation of the wavelet coefficients within a preset feature window, To adjust the parameters; According to the horizontal edge feature sub-image and the vertical edge feature sub-image in the multiple high-frequency sub-images, the correction factor of each preset feature window is calculated, and the feature threshold corresponding to the multiple pixel points in each preset feature window is corrected based on the correction factor to obtain the adaptive threshold corresponding to the multiple pixel points in each preset feature window, wherein the adaptive threshold corresponding to the multiple pixel points in each preset feature window is determined by the following formula: In the formula, For the The adaptive threshold corresponding to multiple pixels in a preset feature window, For the Correction factor for a preset characteristic window, For the The brightness change intensity factor of a preset feature window, , is the first The sum of the wavelet coefficients in a preset feature window, is the first The sum of the wavelet coefficients in a preset feature window, is the maximum value of the brightness change intensity factor in multiple preset feature windows; For multiple pixels in any preset feature window in any high-frequency sub-image, the wavelet coefficients corresponding to the pixels whose wavelet coefficients are not greater than the adaptive threshold are set to 0, thereby obtaining the first feature area image of each high-frequency sub-image at each scale based on the adaptive threshold extraction.

[0045] A feature region fusion module is used to fuse multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image of each second grayscale image at each scale, and to perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; The brightness anomaly detection module is used to perform feature fusion on multiple feature area images to obtain a brightness anomaly image, determine multiple target areas from the brightness anomaly image according to the grayscale value of each pixel in the brightness anomaly image, and generate brightness anomaly detection results of the LED special-shaped screen to be detected according to the target pixel points, target line areas and target areas.

[0046] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A method for detecting surface defects of LED special-shaped screens based on image analysis, characterized in that: include: Collect image data of the special-shaped LED screen to be detected under the brightness test pattern and construct a data set to be detected containing multiple test images; According to the three-dimensional structural parameters of the special-shaped LED screen to be detected, a plurality of test images in the detection data set are geometrically corrected, and the images obtained after the geometric correction are gray-scale converted to obtain a plurality of first gray-scale images; Performing line detection on the multiple first grayscale images based on Hough transform to determine multiple target line regions, traversing each first grayscale image based on a preset feature window to determine multiple target pixel points in the first grayscale image, and processing the multiple target line regions and the multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; Performing wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, and extracting a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image; Traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds; Fusing multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image at each scale of each second grayscale image; Performing inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; Perform feature fusion on multiple feature area images to obtain a brightness abnormality image, and determine multiple target areas from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image; The brightness anomaly detection result of the LED special-shaped screen to be detected is generated according to the target pixel point, target line area and target area.

2. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: For the first grayscale image and the second grayscale image, the method further includes: Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform, determining multiple target line regions in the edge binary image and mapping them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing an erosion process on the first line region, and performing an expansion process on the second line region; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

3. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: Traversing each high-frequency sub-image based on a preset feature window, determining multiple feature thresholds of each high-frequency sub-image, and extracting a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds, including: Traversing each high-frequency sub-image based on a preset feature window, calculating the mean and standard deviation of the wavelet coefficients in each preset feature window, and determining the feature threshold corresponding to multiple pixel points in each preset feature window; in, , where For the The feature threshold corresponding to multiple pixels in a preset feature window, For the The mean value of the wavelet coefficients in a preset feature window, For the The standard deviation of the wavelet coefficients within a preset feature window, To adjust the parameters; According to the horizontal edge feature sub-image and the vertical edge feature sub-image in the multiple high-frequency sub-images, the correction factor of each preset feature window is calculated, and the feature threshold corresponding to the multiple pixel points in each preset feature window is corrected based on the correction factor to obtain the adaptive threshold corresponding to the multiple pixel points in each preset feature window, wherein the adaptive threshold corresponding to the multiple pixel points in each preset feature window is determined by the following formula: In the formula, For the The adaptive threshold corresponding to multiple pixels in a preset feature window, For the Correction factor for a preset characteristic window, For the The brightness change intensity factor of a preset feature window, , is the first The sum of the wavelet coefficients in a preset feature window, is the first The sum of the wavelet coefficients in a preset feature window, is the maximum value of the brightness change intensity factor in multiple preset feature windows; For multiple pixels in any preset feature window in any high-frequency sub-image, the wavelet coefficients corresponding to the pixels whose wavelet coefficients are not greater than the adaptive threshold are set to 0, thereby obtaining the first feature area image of each high-frequency sub-image at each scale based on the adaptive threshold extraction.

4. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: A plurality of first feature region images at the same scale of each second grayscale image are fused to obtain a second feature region image at each scale of each second grayscale image, including: For each second grayscale image at the same scale of multiple first feature area images, the wavelet coefficient standard deviation of each first feature area image in the multiple first feature area images is calculated, the mean of the wavelet coefficient standard deviations of the multiple first feature area images is calculated, the ratio between the wavelet coefficient standard deviation of the first feature area image and the mean of the wavelet coefficient standard deviations of the multiple first feature area images is used as the fusion weight of the first feature area image, and based on the fusion weight corresponding to each first feature area image, the multiple first feature area images at the same scale of each second grayscale image are weightedly fused to obtain the second feature area image of each second grayscale image at each scale.

5. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 1, characterized in that: The brightness abnormality image is obtained by performing feature fusion on multiple feature region images, and multiple target regions are determined from the brightness abnormality image according to the gray value of each pixel in the brightness abnormality image, including: Traversing each feature region image based on a preset feature window, calculating an average value of the grayscale values ​​of the pixels in each preset feature window, calculating a standard deviation of the grayscale values ​​of the pixels in each preset feature window, calculating the average value of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in each feature region image, taking a ratio between an average value of the grayscale values ​​of the pixels in the preset feature window and an average value of the standard deviations of the grayscale values ​​of the pixels in multiple preset feature windows in the feature region image to which the preset feature window belongs as a local weight of the preset feature window, normalizing the local weights corresponding to the multiple preset feature windows in each feature region image, and performing weighted fusion on the multiple preset feature windows at the same position in the multiple feature region images based on the normalized local weights to obtain a brightness abnormality image; The brightness abnormality image is subjected to edge detection according to the gray value of each pixel in the brightness abnormality image to determine multiple brightness abnormality areas, which are recorded as target areas.

6. The method for detecting surface defects of LED special-shaped screens based on image analysis according to claim 5 is characterized in that: The Daubechies wavelet is used as the wavelet basis function to perform wavelet transform on multiple second grayscale images.

7. A device for detecting surface defects of LED special-shaped screens based on image analysis, characterized in that: The device is used to implement the method for detecting surface defects of LED special-shaped screens based on image analysis as described in any one of claims 1 to 6, comprising: A data acquisition module is used to collect image data of the special-shaped LED screen to be detected under a brightness test pattern and construct a data set to be detected containing multiple test images; A geometric correction module, used for performing geometric correction on a plurality of test images in the detection data set according to the three-dimensional structural parameters of the LED special-shaped screen to be detected, and performing grayscale conversion on the images obtained after the geometric correction to obtain a plurality of first grayscale images; A grayscale image processing module is used to perform line detection on multiple first grayscale images based on Hough transform to determine multiple target line areas, traverse each first grayscale image based on a preset feature window, determine multiple target pixel points in the first grayscale image, and process the multiple target line areas and multiple target pixel points based on morphological operations to obtain a second grayscale image corresponding to each first grayscale image; A feature region extraction module is used to perform wavelet transform and multi-scale wavelet decomposition on the plurality of second grayscale images, extract a low-frequency sub-image and a plurality of high-frequency sub-images at each scale of each second grayscale image, traverse each high-frequency sub-image based on a preset feature window, determine a plurality of feature thresholds of each high-frequency sub-image, and extract a first feature region image of each high-frequency sub-image at each scale based on the feature thresholds; A feature region fusion module is used to fuse multiple first feature region images at the same scale of each second grayscale image to obtain a second feature region image of each second grayscale image at each scale, and to perform inverse wavelet transform on the low-frequency sub-image and the second feature region image of each second grayscale image at each scale to obtain a target feature region image corresponding to each second grayscale image; The brightness anomaly detection module is used to perform feature fusion on multiple feature area images to obtain a brightness anomaly image, determine multiple target areas from the brightness anomaly image according to the grayscale value of each pixel in the brightness anomaly image, and generate brightness anomaly detection results of the LED special-shaped screen to be detected according to the target pixel points, target line areas and target areas.

8. The device for detecting surface defects of LED special-shaped screens based on image analysis according to claim 7, characterized in that: For the grayscale image processing module, it also includes: Performing edge detection on each first grayscale image based on the Canny operator to obtain an edge binary image corresponding to each first grayscale image, performing line detection on the edge binary image based on the Hough transform, determining multiple target line regions in the edge binary image and mapping them to the first grayscale image, dividing the multiple target line regions in the first grayscale image into a first line region and a second line region, performing an erosion process on the first line region, and performing an expansion process on the second line region; Based on the preset feature window, each high-frequency sub-image is traversed, the maximum and minimum grayscale values ​​in each preset feature window are determined, and multiple target pixel points in each first grayscale image are determined based on the maximum and minimum grayscale values. The multiple target pixel points are divided into first pixel points and second pixel points, the first pixel points are corroded, and the second pixel points are expanded to obtain a second grayscale image corresponding to each first grayscale image.

Images