A fault monitoring intelligent positioning system for liquid crystal display screen

Abstract

The present application belongs to the technical field of computer vision detection, and discloses a kind of intelligent positioning system for fault monitoring of liquid crystal display screen, comprising: preliminary detection module, obtains LCD panel front view image and detects preliminary defect area;Angle prediction module, extract multidimensional feature vector from preliminary defect area, and predict the best observation angle through feature-angle mapping model;Auxiliary image module, obtain the LCD panel image of best observation angle and carry out enhancement processing;Auxiliary detection module, detect side view defect area;Coordinate mapping module, map side view defect area to front view coordinate system;Report module, output defect detection report.The present application establishes the mapping relationship between feature and observation angle based on the physical characteristics of Mura defect, effectively solves the problem of incomplete defect marking caused by front view image alone, and also improves the system detection efficiency by obtaining the auxiliary observation image of the best observation angle.

Description

Technical Field

[0001] This invention relates to the field of computer vision inspection technology, and more specifically, to an intelligent positioning system for fault monitoring of liquid crystal displays. Background Technology

[0002] With the widespread application of LCD technology and increasing consumer demands for display quality, quality control of LCD panels has become increasingly important. Mura defects are common visual defects in LCD panel manufacturing, manifesting as low contrast, areas of non-uniform brightness, and blurred edges. These defects are typically larger than one pixel and can cause visual discomfort to the viewer. Mura defects are also among the most complex and difficult to detect of all visual defects. With the development of computer vision and image processing technologies, automated Mura defect detection systems have gradually become a research hotspot and a mainstream direction for industrial applications.

[0003] Currently, most automated Mura defect detection systems employ image processing-based methods, acquiring front-view images of the LCD panel using high-precision cameras and then applying various image processing algorithms for defect detection. However, existing Mura defect detection technologies still have some significant shortcomings. First, most detection systems only use the front-view image of the LCD panel for analysis, ignoring the differences in the visibility of Mura defects at different viewing angles. Due to the optical anisotropy of liquid crystal molecules, some Mura defects may not be obvious at the front view angle but become very obvious at specific angles. This leads to the risk of missed detections in detection methods based solely on the front-view image, failing to comprehensively reflect the actual quality condition of the LCD panel. Second, to address the above problem, a multi-angle Mura defect detection method has been proposed, acquiring images of the LCD panel at multiple preset angles and then analyzing each image separately. While this method can improve the comprehensiveness of detection, blindly performing detection at all preset angles leads to a significant waste of computational resources because the visibility of different types of Mura defects varies at different angles. Summary of the Invention

[0004] To overcome the aforementioned problems of the prior art, this invention proposes an intelligent fault monitoring and positioning system for liquid crystal displays, which solves the above-mentioned problems.

[0005] This invention provides the following technical solution:

[0006] A fault monitoring and intelligent location system for liquid crystal displays includes:

[0007] The preliminary inspection module is used to acquire a front view image of the LCD panel, perform defect detection on the front view image of the LCD panel to obtain the defect area and record it as the preliminary defect area;

[0008] An angle prediction module is used to extract multi-dimensional feature vectors from the preliminary defect area; and input the multi-dimensional feature vectors into a pre-trained feature-angle mapping model to predict the optimal viewing angle.

[0009] An auxiliary image module is used to acquire an LCD panel image at the optimal viewing angle, denoted as the auxiliary observation image; the auxiliary observation image is then enhanced to obtain an enhanced observation image.

[0010] The auxiliary detection module is used to perform defect detection on the enhanced observation image to obtain the defect region and record it as the side-view defect region;

[0011] The coordinate mapping module is used to map the side-view defect area to the front-view coordinate system through perspective transformation to obtain the final defect area;

[0012] The reporting module is used to output a defect detection report based on the final defect area.

[0013] Preferably, the defect detection includes:

[0014] The system receives the image to be detected for defects, converts it to grayscale, and records it as the image to be detected. The image to be detected is then preprocessed with noise suppression and illumination equalization.

[0015] For the preprocessed image to be detected, based on the grayscale image of the corresponding LCD panel display image in the image to be detected, the filtered image is subjected to differential processing to obtain a differential image;

[0016] The difference image is divided into several regions, and the regional statistical features of each region are calculated. The regional statistical features include the region's average gray value, the region's standard deviation of gray value, and the region's average gradient. An adaptive threshold is set for each region based on the regional statistical features.

[0017] The gray value of each pixel in the difference image is compared with the adaptive threshold of the corresponding region, and pixels with gray values ​​greater than the threshold are marked as candidate defect points.

[0018] Connectivity analysis is applied to the candidate defect points to obtain connected regions;

[0019] Select connected regions whose area, contrast, and shape characteristics meet predetermined conditions as defect regions.

[0020] Preferably, the multidimensional feature vector includes spatial features, texture features, frequency domain features, and brightness features;

[0021] The spatial features include location, area, shape factor, orientation, and boundary gradient;

[0022] The texture features include local binary pattern features and gray-level co-occurrence matrix features;

[0023] The frequency domain features include wavelet transform coefficients and Fourier descriptors;

[0024] The brightness characteristics include contrast, mean, variance, skewness, and kurtosis.

[0025] Preferably, the step of obtaining the feature-angle mapping model includes:

[0026] Collect a sample set of LCD panels containing known Mura defects;

[0027] For each sample in the LCD panel sample set, multi-angle images are acquired at preset angle intervals within a preset angle range to form a multi-angle image set;

[0028] Calculate the feature enhancement score for each angle image in the multi-angle image set;

[0029] The angle with the highest feature enhancement score for each sample is determined as the optimal observation angle for that sample.

[0030] Extract the multidimensional feature vector of the Mura defect region from the front view image of each sample;

[0031] Using the multidimensional feature vector as input and the optimal observation angle as output, a gradient boosting tree model is trained as the feature-angle mapping model.

[0032] Preferably, the calculation of feature enhancement score for each angle image in the multi-angle image set includes:

[0033] For the sample image at the frontal viewing angle, preprocessing and grayscale conversion are performed, and then the difference is performed with the grayscale image of the sample display image to obtain the baseline difference image; for the sample image at each angle other than the frontal viewing angle, preprocessing and grayscale conversion are performed, and the difference is performed with the grayscale image of the sample display image to obtain the comparison difference image.

[0034] In the baseline difference image and the contrast difference image, all defect pixels are marked according to the Mura defect region of the sample to form a defect pixel set;

[0035] For each defective pixel in the reference difference image, calculate the absolute difference between its gray value and the average gray value of the reference difference image; sum up the absolute differences of all defective pixels to obtain the total difference of the reference pixels;

[0036] For each defective pixel in the comparison difference image, calculate the absolute difference between its gray value and the average gray value of the comparison difference image; sum up the absolute differences of all defective pixels to obtain the total difference of the comparison pixels;

[0037] The feature enhancement score for each angle image is obtained based on the ratio between the sum of the differences between the contrasting pixels and the sum of the differences between the baseline pixels.

[0038] Preferably, the enhancement process includes illumination compensation, filtering, and histogram equalization.

[0039] Preferably, mapping the side-view defect region to the front-view coordinate system through perspective transformation to obtain the final defect region includes:

[0040] Calculate the perspective transformation matrix from the side view angle to the front view angle based on the pre-calibrated camera parameters;

[0041] The side-view defect area is processed into a side-view defect area mask image;

[0042] The side-view defect region mask is transformed to the front-view coordinate system using the perspective transformation matrix;

[0043] The transformed mask image is then fused with the initial defect area to obtain the final defect area.

[0044] Preferably, the step of fusing the transformed mask image with the preliminary defect region to obtain the final defect region includes:

[0045] Calculate the intersection and union of the transformed mask image and the initial defect area;

[0046] When the ratio of intersection to union is greater than the preset fusion threshold, the transformed mask image is used as the final defect area.

[0047] When the ratio of intersection to union is less than or equal to the preset fusion threshold, the union of the transformed mask image and the initial defect area is taken as the final defect area.

[0048] Preferably, the defect detection report includes the location, shape, and severity of the final defect area;

[0049] The location includes the center coordinates and boundary coordinates of the final defect area; the shape includes the area, perimeter, and roundness; and the severity is scored according to the location and shape based on preset rules.

[0050] This invention provides an intelligent fault monitoring and positioning system for liquid crystal displays, which has the following advantages:

[0051] This invention achieves intelligent prediction from a frontal view image to the optimal viewing angle by extracting multi-dimensional feature vectors from the initial defect area through an angle prediction module and inputting them into a pre-trained feature-angle mapping model. Based on the physical properties of the Mura defect, this module establishes a mapping relationship between defect features and the optimal viewing angle, effectively solving the problem of incomplete defect marking caused by relying solely on the frontal view image in existing technologies. Due to the optical anisotropy of liquid crystal molecules, the path length and refraction angle of light through the liquid crystal layer change when viewed from a specific angle, making defects that are originally inconspicuous at a frontal view angle clearly visible. By analyzing the spatial, textural, frequency domain, and brightness characteristics of the defect, the invention accurately predicts the viewing angle that maximizes defect visibility, significantly improving the comprehensiveness and accuracy of detection.

[0052] By selectively acquiring auxiliary observation images from the optimal viewing angle, rather than blindly performing detection at multiple preset angles, this invention significantly improves the system's detection efficiency. Only the frontal view image and the predicted optimal viewing angle image need to be acquired, drastically reducing image acquisition and processing time. Furthermore, since the optimal viewing angle is intelligently predicted based on defect features, rather than blindly selected, the acquired auxiliary observation images possess the highest information value, ensuring that detection quality is not sacrificed while reducing computational load, achieving a dual improvement in detection efficiency and accuracy. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a module of an intelligent positioning system for fault monitoring of a liquid crystal display screen according to the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0055] Please see Figure 1 In this embodiment, a fault monitoring and intelligent positioning system for a liquid crystal display screen includes:

[0056] The preliminary inspection module is used to acquire a front view image of the LCD panel, perform defect detection on the front view image of the LCD panel to obtain the defect area and record it as the preliminary defect area;

[0057] The defect detection includes:

[0058] The system receives the image to be detected for defects, converts it to grayscale, and records it as the image to be detected. The image to be detected is then preprocessed with noise suppression and illumination equalization.

[0059] For the preprocessed image to be detected, based on the grayscale image of the corresponding LCD panel display image in the image to be detected, the filtered image is differentially processed to obtain a differential image;

[0060] The difference image is divided into several regions, and the regional statistical features of each region are calculated. The regional statistical features include the region's average gray value, the region's standard deviation of gray value, and the region's average gradient. An adaptive threshold is set for each region based on the regional statistical features.

[0061] The gray value of each pixel in the difference image is compared with the adaptive threshold of the corresponding region, and pixels with gray values ​​greater than the threshold are marked as candidate defect points.

[0062] Connectivity analysis is applied to the candidate defect points to obtain connected regions;

[0063] Select connected regions whose area, contrast, and shape characteristics meet predetermined conditions as defect regions.

[0064] In this embodiment, an industrial-grade CCD camera can be used to acquire a front view image of the LCD panel. The camera is fixed above the detection platform, parallel to the LCD panel. The acquired raw image is first converted to grayscale, transforming the RGB color image into an 8-bit grayscale image, which is denoted as the image to be detected. Then, the image to be detected is preprocessed, including noise suppression and illumination equalization. Noise suppression uses a Gaussian filtering method, and illumination equalization uses an adaptive histogram equalization method.

[0065] Mura defects typically manifest as uneven brightness on an LCD panel when displaying a uniform image. Based on this characteristic, a differential method is used to highlight Mura defects. First, the LCD panel displays a standard test image (usually a grayscale image), and its grayscale image is acquired (this is the original image in grayscale). A mean-mode filter is applied to the preprocessed image to suppress noise while preserving the edge and regional features of the Mura defect. The difference between the filtered image and the standard grayscale image is calculated to obtain the differential image. The differential operation effectively highlights Mura defect areas because, ideally, the difference value of defect-free areas should be close to zero, while Mura defect areas, due to their abnormal brightness, will produce a significant difference value.

[0066] Due to the potential brightness uniformity in different areas of the LCD panel and the regional characteristics of Mura defects, the differential image was uniformly divided into 8×6 rectangular regions, each approximately 1 / 48 the size of the original image. For each region, three statistical features were calculated: the average gray level, the standard deviation of the gray level, and the average gradient. The average gray level reflects the overall brightness deviation of the region; the standard deviation reflects the dispersion of brightness distribution within the region, with Mura defect areas typically exhibiting a higher standard deviation; and the average gradient reflects the drastic brightness changes within the region, with Mura defect edges typically showing higher gradient values. These statistical features effectively describe the regional characteristics of Mura defects, providing a basis for subsequent adaptive threshold setting.

[0067] Since Mura defects can exhibit different brightness characteristics in different regions, an adaptive threshold is used for detection. Based on regional statistical characteristics, an adaptive threshold is set for each region. This adaptive threshold can automatically adjust the detection sensitivity according to the characteristics of different regions; for example, a higher threshold is used for regions with large brightness variations, and a lower threshold is used for regions with uniform brightness, thereby improving the accuracy of Mura defect detection.

[0068] The grayscale value of each pixel in the difference image is compared with an adaptive threshold for its region. For each pixel in the difference image, its region is determined, and an adaptive threshold for that region is obtained. When the grayscale value of a pixel is greater than the threshold, it is marked as a candidate defect point; otherwise, it is marked as a background point. A binary image is generated, where candidate defect points are represented by 1 (white) and background points by 0 (black). This step initially separates the Mura defect from the background, laying the foundation for subsequent connected component analysis.

[0069] Mura defects typically appear as continuous regions; therefore, connected component analysis is used to cluster candidate defect points. The 8-connectivity criterion is applied to label connected components in the binary image, resulting in a series of connected regions. For each connected region, area, contrast, and shape features are calculated. The area is the number of pixels within the connected region; the contrast is the absolute value of the difference between the average gray value of the pixels within the connected region in the original difference image and the average gray value of the surrounding background region; shape features include perimeter, roundness, and aspect ratio. Connected component analysis can organize scattered candidate defect points into meaningful regions, facilitating subsequent defect feature extraction and classification.

[0070] Based on the typical characteristics of Mura defects, connected regions are filtered. For example, areas larger than a preset threshold are used to filter out excessively small noise. Since Mura defects typically manifest as abnormal brightness, a certain level of contrast is crucial; the contrast must exceed a preset threshold to ensure sufficient brightness difference between the defect area and the background. Regarding shape conditions, Mura defects are usually irregular in shape and therefore have low roundness. A roundness threshold below a preset threshold can be set to retain irregularly shaped areas. Simultaneously, an appropriate aspect ratio range is set to exclude excessively long and thin areas, as these are typically bright or dark lines rather than Mura defects. These conditions, based on the typical morphological characteristics of Mura defects, effectively distinguish them from other types of defects. Connected regions that meet all the above conditions are marked as defect areas.

[0071] An angle prediction module is used to extract multi-dimensional feature vectors from the preliminary defect area; and input the multi-dimensional feature vectors into a pre-trained feature-angle mapping model to predict the optimal viewing angle.

[0072] The multidimensional feature vector includes spatial features, texture features, frequency domain features, and brightness features;

[0073] The spatial features include location, area, shape factor, orientation, and boundary gradient;

[0074] The texture features include local binary pattern features and gray-level co-occurrence matrix features;

[0075] The frequency domain features include wavelet transform coefficients and Fourier descriptors;

[0076] The brightness characteristics include contrast, mean, variance, skewness, and kurtosis.

[0077] The steps for obtaining the feature-angle mapping model include:

[0078] Collect a sample set of LCD panels containing known Mura defects;

[0079] For each sample in the LCD panel sample set, multi-angle images are acquired at preset angle intervals within a preset angle range to form a multi-angle image set;

[0080] Calculate the feature enhancement score for each angle image in the multi-angle image set;

[0081] The angle with the highest feature enhancement score for each sample is determined as the optimal observation angle for that sample.

[0082] Extract the multidimensional feature vector of the Mura defect region from the front view image of each sample;

[0083] Using the multidimensional feature vector as input and the optimal observation angle as output, a gradient boosting tree model is trained as the feature-angle mapping model.

[0084] The calculation of feature enhancement score for each angle image in the multi-angle image set includes:

[0085] For the sample image at the frontal viewing angle, preprocessing and grayscale conversion are performed, and then the difference is performed with the grayscale image of the sample display image to obtain the baseline difference image; for the sample image at each angle other than the frontal viewing angle, preprocessing and grayscale conversion are performed, and the difference is performed with the grayscale image of the sample display image to obtain the comparison difference image.

[0086] In the baseline difference image and the contrast difference image, all defect pixels are marked according to the Mura defect region of the sample to form a defect pixel set;

[0087] For each defective pixel in the reference difference image, calculate the absolute difference between its gray value and the average gray value of the reference difference image; sum up the absolute differences of all defective pixels to obtain the total difference of the reference pixels;

[0088] For each defective pixel in the comparison difference image, calculate the absolute difference between its gray value and the average gray value of the comparison difference image; sum up the absolute differences of all defective pixels to obtain the total difference of the comparison pixels;

[0089] The feature enhancement score for each angle image is obtained based on the ratio between the sum of the differences between the contrasting pixels and the sum of the differences between the baseline pixels.

[0090] In this embodiment, the angle prediction module is mainly used to extract multi-dimensional feature vectors from the initial defect area and predict the optimal viewing angle through a pre-trained feature-angle mapping model. Mura defects are usually caused by factors such as uneven stress distribution inside the LCD panel, abnormal liquid crystal molecule arrangement, or inconsistent backlight transmittance. For example, if uneven pressure is applied during the panel manufacturing process, it will cause the liquid crystal molecules to shift in the alignment direction in some areas. This shift may not be obvious at a frontal viewing angle, but the defect will become more obvious when viewed from a specific side viewing angle due to the optical anisotropy of the liquid crystal molecules.

[0091] The multidimensional feature vector extracted from the initial defect region includes spatial features, texture features, frequency domain features, and brightness features. Spatial features include location, area, shape factor, directionality, and boundary gradient, used to describe the geometric characteristics of the defect region; texture features include local binary pattern features and gray-level co-occurrence matrix features, used to describe the texture complexity of the defect region; frequency domain features include wavelet transform coefficients and Fourier descriptors, used to describe the performance of the defect at different frequencies; brightness features include contrast, mean, variance, skewness, and kurtosis, used to describe the brightness distribution characteristics of the defect region.

[0092] The process of obtaining the feature-angle mapping model first requires collecting a sample set of LCD panels containing known Mura defects. For example, 200 LCD panel samples with Mura defects of different causes are collected. For each sample, multi-angle images are acquired within a preset angle range and at preset angle intervals to form a multi-angle image set. For example, in both the horizontal and vertical directions, a range of -60° to 60° can be selected, with images acquired at 15° intervals.

[0093] By calculating the feature enhancement score, the enhancement effect of different viewing angles on the visibility of Mura defects can be quantified. The feature enhancement score is essentially the ratio of the contrast of the defect region in the contrast difference image to the contrast of the defect region in the baseline difference image. A higher score indicates a greater contrast between the defect region and the background at that viewing angle, making the defect easier to observe. Due to the optical anisotropy of liquid crystal molecules, the path length and refraction angle of light through the liquid crystal layer change when viewed from different angles, causing defect regions that are not obvious at a frontal viewing angle to have significantly enhanced contrast at a specific side viewing angle.

[0094] After determining the angle with the highest feature enhancement score for each sample as the optimal viewing angle for that sample, multi-dimensional feature vectors of the Mura defect region are extracted from the front view image of each sample. Using these feature vectors as input and the optimal viewing angle as output, a gradient boosting tree model is trained as a feature-angle mapping model. Since different types of Mura defects have different physical causes and optical properties, their optimal viewing angles also differ, but this difference is intrinsically related to the characteristics of the defect. For example, for large-area, irregularly shaped Mura defects caused by uneven stress distribution, the optimal viewing angle often has a large shift in the vertical direction; while for directional striped Mura defects caused by abnormal liquid crystal molecule arrangement, the optimal viewing angle is closely related to the direction of the stripes. The gradient boosting tree model can effectively learn this complex nonlinear relationship and accurately predict the optimal viewing angle based on the characteristics of the defect.

[0095] During model training, a 5-fold cross-validation method can be used to evaluate model performance. The final model's average angle prediction error on the test set is less than 10°, meeting practical application requirements. This feature-based angle prediction method has significant practical value, especially on high-speed production lines where limited inspection time typically limits the acquisition of frontal images for preliminary inspection. Through the angle prediction module in this embodiment, the system can intelligently determine the optimal viewing angle and then selectively acquire images at that angle for further analysis, significantly improving the accuracy and comprehensiveness of Mura defect detection and providing more reliable technical support for LCD panel quality control.

[0096] An auxiliary image module is used to acquire an LCD panel image at the optimal viewing angle, denoted as the auxiliary observation image; the auxiliary observation image is then enhanced to obtain an enhanced observation image.

[0097] The enhancement process includes illumination compensation, filtering, and histogram equalization.

[0098] In this embodiment, based on the optimal observation angle predicted by the angle prediction module, the camera position is adjusted or the corresponding camera in the multi-camera array is used to acquire an image of the LCD panel at that angle, which is denoted as the auxiliary observation image. Since side-view shooting may present issues such as uneven illumination and reduced contrast, the auxiliary observation image needs to undergo illumination compensation, filtering, and histogram equalization to improve defect visibility. Through the above enhancement processing, the resulting enhanced observation image has more uniform illumination conditions, clearer edge features, and higher local contrast, providing high-quality image input for subsequent auxiliary detection.

[0099] The auxiliary detection module is used to perform defect detection on the enhanced observation image to obtain the defect region and record it as the side-view defect region;

[0100] In this embodiment, the auxiliary detection module employs the same detection method as the preliminary detection module. Although the detection method is the same, due to the change in viewing angle, the defect area detected at the side viewing angle may differ significantly from the area detected at the front viewing angle. This is mainly because of the optical anisotropy of liquid crystal molecules, which makes some Mura defects that are not obvious at the front viewing angle more obvious at specific side viewing angles. For example, Mura defects caused by abnormal alignment of liquid crystal molecules may only show slight brightness abnormalities in some areas at the front viewing angle, while at the optimal viewing angle, the entire defect area will show obvious brightness differences. At the same time, defect edge areas that are not detectable at the front viewing angle may become clearly visible at the side viewing angle. These changes allow the side-viewed defect area to provide supplementary information that cannot be obtained from the front-viewed defect area, helping to more comprehensively determine the actual range and severity of the defect.

[0101] Through the auxiliary detection module, the system can obtain defect area information at the optimal viewing angle. This information complements the preliminary defect area at the frontal viewing angle, providing a more comprehensive basis for subsequent coordinate mapping and final defect area determination.

[0102] The coordinate mapping module is used to map the side-view defect area to the front-view coordinate system through perspective transformation to obtain the final defect area;

[0103] The step of mapping the side-view defect region to the front-view coordinate system through perspective transformation to obtain the final defect region includes:

[0104] Calculate the perspective transformation matrix from the side view angle to the front view angle based on the pre-calibrated camera parameters;

[0105] The side-view defect area is processed into a side-view defect area mask image;

[0106] The side-view defect region mask is transformed to the front-view coordinate system using the perspective transformation matrix;

[0107] The transformed mask image is then fused with the initial defect area to obtain the final defect area.

[0108] The step of fusing the transformed mask image with the preliminary defect region to obtain the final defect region includes:

[0109] Calculate the intersection and union of the transformed mask image and the initial defect area;

[0110] When the ratio of intersection to union is greater than the preset fusion threshold, the transformed mask image is used as the final defect area.

[0111] When the ratio of intersection to union is less than or equal to the preset fusion threshold, the union of the transformed mask image and the initial defect area is taken as the final defect area.

[0112] In this embodiment, firstly, the perspective transformation matrix from the side-view angle to the front-view angle is calculated based on pre-calibrated camera parameters. The transformation matrix can be obtained by calibrating the camera using a standard checkerboard calibration board to acquire the camera's intrinsic parameter matrix and distortion coefficients. For each preset viewing angle, the camera's extrinsic parameter matrix (rotation matrix and translation vector) is recorded. Based on these parameters, the perspective transformation matrix from the side-view angle to the front-view angle can be calculated. In this embodiment, taking the same camera shooting from different angles as an example, the intrinsic parameter matrix remains unchanged.

[0113] Next, the side-view defect region is processed into a side-view defect region mask image. Specifically, the side-view defect region detected by the auxiliary detection module is converted into a binary image, with the defect region pixel value being 1 and the background pixel value being 0, thus forming a mask image.

[0114] Then, the side-view defect area mask is transformed to the front-view coordinate system using the calculated perspective transformation matrix. The transformation process employs bilinear interpolation to ensure the quality of the transformed image. Through perspective transformation, the defect area from the side view angle is mapped to the same coordinate system as the front-view image, allowing for direct comparison and fusion between the two.

[0115] Finally, the transformed mask image is fused with the preliminary defect region to obtain the final defect region. The fusion process first calculates the intersection and union of the transformed mask image and the preliminary defect region, and then calculates the ratio of the intersection to the union, i.e., the Jaccard similarity coefficient.

[0116] When the similarity coefficient is greater than a preset fusion threshold (e.g., 0.6), it indicates that the two defect areas highly overlap. In this case, the transformed mask image is used as the final defect area. This is because defect areas obtained from a side view angle usually have higher contrast and clearer boundaries, and can more accurately reflect the actual extent of the defect. When the similarity coefficient is less than or equal to the preset fusion threshold, it indicates that the two defect areas do not overlap significantly, possibly because some defects exhibit different characteristics at different angles. In this case, the union of the transformed mask image and the initial defect area is used as the final defect area to ensure that no potential defect areas are missed.

[0117] The reporting module is used to output a defect detection report based on the final defect area.

[0118] The defect detection report includes the location, shape, and severity of the final defect area;

[0119] The location includes the center coordinates and boundary coordinates of the final defect area; the shape includes the area, perimeter, and roundness; and the severity is scored according to the location and shape based on preset rules.

[0120] In this embodiment, for location information, the reporting module calculates the center coordinates and boundary coordinates of the final defect area. The center coordinates are calculated using the region centroid method, and the boundary coordinates include the coordinates of the upper left and lower right corners of the smallest bounding rectangle. The location information is represented in pixel coordinates and converted into a percentage position relative to the LCD panel size for easy comparison between panels of different sizes.

[0121] For shape information, the reporting module calculates the area, perimeter, and roundness of the defect region. These shape features effectively describe the geometric characteristics of the Mura defect, aiding in defect type identification and severity assessment. For example, defects with a roundness close to 1 are typically point-like Muras, while defects with lower roundness may be linear or irregularly shaped Muras.

[0122] For severity assessment, the reporting module scores defects based on location and shape according to preset rules. The scoring rules primarily consider the following factors: location (defects in the center are more severe than those at the edges), area (larger areas indicate higher severity), shape (irregularly shaped defects are generally more noticeable), and contrast (higher contrast makes them more visually apparent). Based on a comprehensive evaluation of these factors, the system can classify defect severity into four levels: minor, moderate, severe, and extremely severe.

[0123] The reporting module presents defect detection results in graphical and tabular formats, including marked defect areas on the original LCD panel image, magnified images of the defect areas, and detailed tables containing information on location, shape, and severity. For multiple defect areas, they can also be sorted by severity from highest to lowest. Through the detailed defect information output of the reporting module, production personnel can quickly understand the quality status of the LCD panels, adjust production parameters in a timely manner, and improve product yield.

[0124] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0125] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0126] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fault monitoring and intelligent positioning system for liquid crystal displays, characterized in that, include: The preliminary inspection module is used to acquire a front view image of the LCD panel, perform defect detection on the front view image of the LCD panel to obtain the defect area and record it as the preliminary defect area; An angle prediction module is used to extract multi-dimensional feature vectors from the preliminary defect region; The multidimensional feature vector is input into a pre-trained feature-angle mapping model to predict the optimal viewing angle. The steps for obtaining the feature-angle mapping model include: Collect a sample set of LCD panels containing known Mura defects; For each sample in the LCD panel sample set, multi-angle images are acquired at preset angle intervals within a preset angle range to form a multi-angle image set; Calculate the feature enhancement score for each angle image in the multi-angle image set; The angle with the highest feature enhancement score for each sample is determined as the optimal observation angle for that sample. Extract the multidimensional feature vector of the Mura defect region from the front view image of each sample; Using the multidimensional feature vector as input and the optimal observation angle as output, a gradient boosting tree model is trained as the feature-angle mapping model. An auxiliary image module is used to acquire an LCD panel image at the optimal viewing angle, denoted as the auxiliary observation image; the auxiliary observation image is then enhanced to obtain an enhanced observation image. The auxiliary detection module is used to perform defect detection on the enhanced observation image to obtain the defect region and record it as the side-view defect region; The coordinate mapping module is used to map the side-view defect area to the front-view coordinate system through perspective transformation to obtain the final defect area; The reporting module is used to output a defect detection report based on the final defect area.

2. The intelligent fault monitoring and positioning system for a liquid crystal display screen according to claim 1, characterized in that, The defect detection includes: The system receives the image to be detected for defects, converts it to grayscale, and records it as the image to be detected. The image to be detected is then preprocessed with noise suppression and illumination equalization. For the preprocessed image to be detected, based on the grayscale image of the corresponding LCD panel display image in the image to be detected, the filtered image is differentially processed to obtain a differential image; The difference image is divided into several regions, and the regional statistical features of each region are calculated. The regional statistical features include the region's average gray value, the region's standard deviation of gray value, and the region's average gradient. An adaptive threshold is set for each region based on the regional statistical features. The gray value of each pixel in the difference image is compared with the adaptive threshold of the corresponding region, and pixels with gray values ​​greater than the threshold are marked as candidate defect points. Connectivity analysis is applied to the candidate defect points to obtain connected regions; Select connected regions whose area, contrast, and shape characteristics meet predetermined conditions as defect regions.

3. The intelligent fault monitoring and positioning system for a liquid crystal display screen according to claim 2, characterized in that, The multidimensional feature vector includes spatial features, texture features, frequency domain features, and brightness features; The spatial features include location, area, shape factor, orientation, and boundary gradient; The texture features include local binary pattern features and gray-level co-occurrence matrix features; The frequency domain features include wavelet transform coefficients and Fourier descriptors; The brightness characteristics include contrast, mean, variance, skewness, and kurtosis.

4. The intelligent fault monitoring and positioning system for a liquid crystal display screen according to claim 3, characterized in that, The calculation of feature enhancement score for each angle image in the multi-angle image set includes: For the sample image at the frontal viewing angle, preprocessing and grayscale conversion are performed, and then the difference is performed with the grayscale image of the sample display image to obtain the baseline difference image; for the sample image at each angle other than the frontal viewing angle, preprocessing and grayscale conversion are performed, and the difference is performed with the grayscale image of the sample display image to obtain the comparison difference image. In the baseline difference image and the contrast difference image, all defect pixels are marked according to the Mura defect region of the sample to form a defect pixel set; For each defective pixel in the reference difference image, calculate the absolute difference between its gray value and the average gray value of the reference difference image; sum up the absolute differences of all defective pixels to obtain the total difference of the reference pixels; For each defective pixel in the comparison difference image, calculate the absolute difference between its gray value and the average gray value of the comparison difference image; sum up the absolute differences of all defective pixels to obtain the total difference of the comparison pixels; The feature enhancement score for each angle image is obtained based on the ratio between the sum of the differences between the contrasting pixels and the sum of the differences between the baseline pixels.

5. The intelligent fault monitoring and positioning system for a liquid crystal display screen according to claim 1, characterized in that, The enhancement process includes illumination compensation, filtering, and histogram equalization.

6. The intelligent positioning system for fault monitoring of a liquid crystal display screen according to claim 5, characterized in that, The step of mapping the side-view defect region to the front-view coordinate system through perspective transformation to obtain the final defect region includes: Calculate the perspective transformation matrix from the side view angle to the front view angle based on the pre-calibrated camera parameters; The side-view defect area is processed into a side-view defect area mask image; The side-view defect region mask is transformed to the front-view coordinate system using the perspective transformation matrix; The transformed mask image is then fused with the initial defect area to obtain the final defect area.

7. A fault monitoring and intelligent positioning system for a liquid crystal display screen according to claim 6, characterized in that, The step of fusing the transformed mask image with the preliminary defect region to obtain the final defect region includes: Calculate the intersection and union of the transformed mask image and the initial defect area; When the ratio of intersection to union is greater than the preset fusion threshold, the transformed mask image is used as the final defect area. When the ratio of intersection to union is less than or equal to the preset fusion threshold, the union of the transformed mask image and the initial defect area is taken as the final defect area.

8. The intelligent fault monitoring and positioning system for a liquid crystal display screen according to claim 7, characterized in that, The defect detection report includes the location, shape, and severity of the final defect area; The location includes the center coordinates and boundary coordinates of the final defect area; the shape includes the area, perimeter, and roundness; and the severity is scored according to the location and shape based on preset rules.

Images