Liquid crystal screen Mura defect detection system based on convolutional neural network

Abstract

The invention belongs to the technical field of visual inspection, and particularly relates to a liquid crystal screen Mura defect detection system based on a convolutional neural network, which comprises the following steps: establishing an XY coordinate system on a liquid crystal screen, and obtaining a to-be-detected image by scrolling display of a target area and control of brightness circulation; the method further comprises the following steps of: preprocessing the image, wherein the preprocessing comprises denoising and brightness balancing; extracting the frequency domain characteristics of the image through Fourier transform, and calculating the edge attribute of the defect in combination with an edge impact algorithm; and inputting the preprocessed image into the convolutional neural network, optimizing the network weight through dynamic iteration, generating a defect detection result, and outputting the defect detection result. According to the invention, through dynamic rolling and brightness control, the contrast ratio of the defect area is enhanced, and the frequency domain is combined with the spatial domain, so that the detection rate of the tiny Mura defect is greatly improved, and compared with a traditional detection mode, the efficiency and the precision are higher.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to a liquid crystal screen Mura defect detection system based on a convolutional neural network. Background Art

[0002] LCD panel inspection is a critical quality control process that ensures that display devices are defect-free during production, thus ensuring the visual quality and user experience of the final product.

[0003] The manufacturing process of liquid crystal screens (LCD) and organic light-emitting semiconductor (OLED) screens is complex, and various defects may be introduced during the production process. In practical applications, machine vision inspection systems usually include image acquisition modules, lighting modules, and processing software. The image acquisition module is responsible for acquiring images of the LCD screen, the lighting module provides appropriate lighting to highlight defects, and the processing software analyzes the image and identifies defects through algorithms.

[0004] The biggest difficulty in LCD screen inspection projects is the detection of Mura defects. Currently, the widely used defect detection method is based on image processing. There are problems such as misjudgment and missed detection. The inspected products need to be manually re-judged, which is inefficient and wastes a lot of manpower. Summary of the invention

[0005] The purpose of the present invention is to provide a liquid crystal screen Mura defect detection system based on a convolutional neural network to solve the problems raised in the background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a liquid crystal screen mura defect detection system based on convolutional neural network, comprising the following steps: Establish an XY coordinate system on the LCD screen, and obtain the image to be detected by scrolling the target area and controlling the brightness cycle; Preprocessing the image, including denoising and brightness equalization; The frequency domain features of the image are extracted through Fourier transform, and the edge properties of the defect are calculated by combining the edge impact algorithm; The preprocessed image is input into the convolutional neural network, and the network weights are optimized through dynamic iteration to generate and output the defect detection results.

[0007] In the liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, the calculation formula of edge impact in the Fourier transform is: ; Among them, sgn is the sign function, is the second-order directional derivative of the image grayscale value, is the image gradient.

[0008] The liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, wherein the iterative step length of the edge impact Dynamic adjustment, including: Initialization step size =0.1; Update according to the loss function descent rate , where α=10; The iteration stops when the loss function change is less than the threshold 1E-5.

[0009] In the liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, the calculation formula of the expansion coefficient d and the erosion coefficient e is: And sharpen the edges through morphological operations.

[0010] The liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, wherein the convolutional neural network is an improved ResNet-18 structure, comprising: Input layer (224×224×3); 4 residual blocks, each containing 2 convolutional layers (kernel 3×3, stride 1, Padding=1); Global average pooling layer and fully connected layer.

[0011] The liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, wherein the training of the convolutional neural network adopts a two-stage strategy: Initial stage: Combine traditional image processing to generate auxiliary labels, and fuse them with CNN prediction results at a weight of 7:3; Stable stage: When the accuracy of the validation set is >85%, only CNN independent detection is used.

[0012] The liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, wherein a space-frequency domain attention module is embedded in the residual block, comprising: Spatial branch: extract local texture features through 3×3 convolution; Frequency domain branch: Fast Fourier transform (FFT) is performed on the feature map and key frequencies are enhanced through learnable filters; The fusion formula is: ; Where σ is the Sigmoid function, W s , W f are trainable weights.

[0013] The liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, wherein the following steps are added before the Fourier transform: Perform wavelet decomposition on the preprocessed image to extract high-frequency components and low-frequency components; The high-frequency component is Fourier transformed and fused with the original image Fourier features according to the weights. The formula is: .

[0014] The liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, wherein the training data enhancement of the system includes: random rotation (±5°); Brightness jitter (±10%); Add Gaussian noise (σ=0.01).

[0015] In the liquid crystal screen mura defect detection system based on convolutional neural network of the present invention, in the process of dynamically iteratively optimizing the network weights, the Adam optimizer update rule is adopted: Among them, m t and v t The first-order and second-order momentum of the Adam optimizer are used to adaptively adjust the learning rate and accelerate model convergence; η is the learning rate, which determines the step size of weight update; 1E-8 is a numerical stability constant to prevent the denominator from being zero; W t+1 is the updated weight, which is used to adaptively adjust the learning rate, accelerate convergence and reduce oscillation. The calculation formula is: .

[0016] Compared with the prior art, the present invention has the following beneficial effects: the present invention enhances the contrast of the defect area through dynamic scrolling and brightness control, and combines the frequency domain with the spatial domain, thereby greatly improving the detection rate of tiny Mura defects. Compared with the traditional detection method, the present invention has higher efficiency and accuracy; Specifically, an XY coordinate system is established on the surface of the LCD screen, and the target area (such as Xa×Yb pixels) is scrolled along the X-axis direction by controlling the power-on state of the screen, and the brightness of the target area is periodically adjusted during the scrolling process, such as N times of bright-dark-bright cycle; after the image acquisition module obtains the screen image, it uses median filtering to denoise, and adjusts the brightness distribution through histogram equalization, performs Fourier transform on the pre-processed image, extracts frequency domain features, and calculates edge attributes in combination with the edge impact algorithm; then the processed image is input into the improved convolutional neural network, and the network weights are optimized through dynamic iteration to generate and output defect detection results, thereby achieving high-precision and high-speed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 The figure is a flow chart of the method steps of the present invention. DETAILED DESCRIPTION

[0019] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present invention and the drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] "Multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0022] Moreover, the terms "up, down, left, right, upper end, lower end, longitudinal" and the like indicating directions are all based on the posture and position of the device or equipment described in this solution during normal use.

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the following will be described clearly and completely in combination with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0024] This embodiment discloses Figure 1 The LCD screen Mura defect detection system based on convolutional neural network shown in the figure includes the following steps: Step S10: Establish an XY coordinate system on the LCD screen, and obtain the image to be detected by scrolling the target area (the size of which is X*Y pixels, and the value of XY can be selected according to actual needs) and controlling the brightness cycle, wherein the scrolling cycle is to scroll the display in sequence along the X-axis direction, and the brightness of the displayed target area is synchronously changed from bright to dark and then to bright; Step S20: preprocessing the image, including denoising and brightness equalization, achieving brightness equalization through histogram equalization to eliminate interference from uneven illumination; Step S30: extracting the frequency domain features of the image by Fourier transform, and calculating the edge attributes of the defect by combining the edge impact algorithm; Step S40: Input the preprocessed image into the convolutional neural network, optimize the network weights through dynamic iteration, generate and output the defect detection results.

[0025] The present invention enhances the contrast of defective areas through dynamic scrolling and brightness control. Dynamic brightness cycle and scrolling acquisition improve the development effect of defective areas, and combine the frequency domain with the spatial domain, thereby greatly improving the detection rate of tiny Mura defects. Compared with traditional detection methods, the efficiency and accuracy are higher. Specifically, an XY coordinate system is established on the surface of the LCD screen, and the target area (such as Xa×Yb pixels) is scrolled along the X-axis direction by controlling the power-on state of the screen, and the brightness of the target area is periodically adjusted during the scrolling process, such as N times of bright-dark-bright cycle; after the image acquisition module acquires the screen image, it uses median filtering to denoise, and adjusts the brightness distribution through histogram equalization, performs Fourier transform on the pre-processed image, extracts frequency domain features, and calculates edge attributes in combination with the edge impact algorithm; then the processed image is input into the improved convolutional neural network, and the network weights are optimized through dynamic iteration to generate and output defect detection results, thereby achieving high-precision and high-speed detection. In this embodiment, in Fourier transform, the calculation formula of edge impact is: ; Among them, sgn is the sign function, is the second-order directional derivative of the image grayscale value, It is the image gradient; the edge impact algorithm can clarify the edge polarity and reduce misjudgment. The second-order derivative can be used to enhance the geometric features of the defect edge, which is suitable for detection under complex backgrounds.

[0026] In this embodiment, the iterative step length of edge impact Dynamic adjustment, including: initialization step length =0.1, calculate the initial loss function L0; Update according to the loss function descent rate , where α1 is the attenuation coefficient, the default value is 10, which is used to control the sensitivity of step size adjustment. The larger α1 is, the smoother the step size changes; The formula for calculating the descent rate is: ; When the change in the loss function (that is, the rate of decline) is less than the threshold 1E-5, the iteration is stopped. By setting the adaptive step size adjustment to improve the convergence speed of the algorithm, the cost of manual parameter adjustment can be reduced, which is suitable for large-scale production lines.

[0027] In this embodiment, the calculation formulas of the expansion coefficient d and the erosion coefficient e are: And sharpen the edge through morphological operations (dilation u+d: fill edge breaks; erosion ue: eliminate isolated noise points), specifically according to the gradient amplitude Dynamically adjust the expansion intensity. The larger the gradient (obvious edge), the higher the expansion amplitude. When the gradient is small, reduce the erosion amplitude to avoid over-smoothing. During the process, the setting of dynamic parameters can adapt to different defect types (such as linear Mura and point Mura), so that this detection method can be applied to PCB solder joint detection to solve the limitations of traditional morphological fixed parameters.

[0028] In this embodiment, the convolutional neural network is an improved ResNet-18 structure, including: The input layer is 224×224×3, which is suitable for high-resolution images on LCD screens; 4 residual blocks, each containing 2 convolutional layers (kernel 3×3, stride 1, Padding=1), extracting multi-scale features through residual blocks (including 3×3 convolutions); Global average pooling layer and fully connected layer (output dimension 2), while the global average pooling layer compresses the feature dimension, and the fully connected layer outputs the "defect" or "normal" classification results; The lightweight network structure can reduce the demand for computing resources and reduce the resource usage of the computer GPU, while the residual connection alleviates the gradient disappearance, making the system more suitable for high-resolution image processing and supporting deeper network expansion.

[0029] In this embodiment, the training of the convolutional neural network adopts a two-stage strategy: Initial stage: Combine traditional image processing (Canny edge detection + morphological operation) to generate auxiliary labels, and fuse them with CNN prediction results at a weight of 7:3. The formula is: Final label = 0.7*CNN output + 0.3*traditional method label, * represents the multiplication relationship; Stable stage: When the accuracy of the validation set is > 85%, only CNN independent detection is used; The two-stage training strategy can alleviate the model cold start problem, improve the initial accuracy, and make it more suitable for industrial inspection scenarios with scarce data (such as the research and development stage of new screens).

[0030] In this embodiment, a space-frequency attention module (SFAM) is embedded in the residual block, including: Spatial branch: extract local texture features through 3×3 convolution; Frequency domain branch: Perform fast Fourier transform FFT on the feature map to generate frequency domain features F freq , and enhance key frequencies through learnable filters; The fusion formula is: ; Where σ is the Sigmoid function, W s , W f is the trainable weight; There are two benefits of dividing the feature map into the spatial branch (3×3 convolution) and the frequency domain branch (FFT+learnable filtering). First, the frequency domain focuses on the periodic signal of the defect, which improves the detection rate. Second, the training efficiency is improved, which is suitable for real-time online detection systems.

[0031] In this embodiment, the following steps are added before Fourier transform: Use Daubechies wavelet to decompose the image into high frequency components W high (details) and low-frequency components W low (contour); The high-frequency component is Fourier transformed and fused with the original image Fourier features according to the weights. The formula is: The fused features are input into CNN for learning. The advantage is that the high-frequency component enhances the periodic characteristics of tiny defects, the low-frequency component suppresses the background brightness fluctuation, the false detection rate is reduced by 15%, and it can be extended to textile defect detection to solve the texture interference problem.

[0032] In this embodiment, the system's training data enhancement includes: Random rotation: ±5° simulates screen installation angle deviation; Brightness jitter: ±10% to simulate different lighting conditions; Add Gaussian noise: σ=0.01 to simulate sensor noise; The benefit of the above data augmentation strategy is to enhance the model's generalization ability for complex environments, reduce the risk of overfitting, and support cross-brand and cross-model screen detection.

[0033] In this embodiment, during the process of dynamic iterative optimization of network weights, the Adam optimizer update rule is adopted: Among them, m t and v tThe first-order and second-order momentum of the Adam optimizer are used to adaptively adjust the learning rate and accelerate model convergence; η is the learning rate, which determines the step size of weight update; 1E-8 is a numerical stability constant to prevent the denominator from being zero; W t+1 The updated weights are used to adaptively adjust the learning rate, accelerate convergence and reduce oscillation.

[0034] The Adam optimizer update rule can adapt the learning rate to prevent the model from falling into the local optimum. It is robust to noisy data and is suitable for low-quality image input scenarios.

[0035] This solution establishes an XY coordinate system on the surface of the LCD screen, controls the power-on state of the screen, scrolls the target area (such as Xa×Yb pixels) along the X-axis direction, and periodically adjusts the brightness of the target area during the scrolling process (bright-dark-bright cycle N times); after the image acquisition module obtains the screen image, it uses median filtering to denoise, and adjusts the brightness distribution through histogram equalization, and performs Fourier transform on the pre-processed image to extract frequency domain features, and calculates edge attributes in combination with the edge impact algorithm; then the processed image is input into the improved ResNet-18 network, and the weights are dynamically updated through the Adam optimizer to output the defect classification results, which greatly facilitates the subsequent operation and management of production supervisors.

[0036] The following table shows the results of this solution compared with the traditional solution: Detection Methods Accuracy (%) False positive rate (%) Missed detection rate (%) Traditional image processing (Canny+Morphology) 82.3 15.6 17.8 Ordinary CNN (without fusion of frequency domain features) 89.1 8.9 10.2 This solution (Fourier + CNN) 95.4 3.2 4.1 The above effect comparison is based on a test data set: 10,000 LCD screen images (5,000 normal images and 5,000 defective images), covering point, line, and cloud-like Mura defects. This solution can significantly improve the detection rate of small defects with an area of ​​<0.1mm² in actual process through frequency domain feature fusion. In addition, in the initial stage, the cold start accuracy can be improved by 19.3% through the auxiliary labeling of traditional methods (weight 7:3), thereby avoiding the defects of the traditional single learning mechanism and enhancing the model's autonomous optimization capability.

[0037] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A liquid crystal screen mura defect detection system based on convolutional neural network, characterized in that: The following steps are involved: Establish an XY coordinate system on the LCD screen, and obtain the image to be detected by scrolling the target area and controlling the brightness cycle; Preprocessing the image, including denoising and brightness equalization; The frequency domain features of the image are extracted through Fourier transform, and the edge properties of the defect are calculated by combining the edge impact algorithm; The preprocessed image is input into the convolutional neural network, and the network weights are optimized through dynamic iteration to generate and output the defect detection results.

2. According to claim 1, the liquid crystal screen mura defect detection system based on convolutional neural network is characterized in that: In the Fourier transform, the calculation formula of edge impact is: ; Among them, sgn is the sign function, is the second-order directional derivative of the image gray value, is the image gradient.

3. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 2, characterized in that: The iteration step size of the edge impact Dynamic adjustment, including: initialization step length =0.1; Update according to the loss function descent rate , where α1=10; The iteration stops when the loss function change is less than the threshold 1E-5.

4. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 3, characterized in that: The calculation formulas for the expansion coefficient d and the erosion coefficient e are: And sharpen the edges through morphological operations.

5. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 1, characterized in that: The convolutional neural network is an improved ResNet-18 structure, including: Input layer (224×224×3); 4 residual blocks, each containing 2 convolutional layers (kernel 3×3, stride 1, Padding=1); Global average pooling layer and fully connected layer.

6. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 5, characterized in that: The training of the convolutional neural network adopts a two-stage strategy: Initial stage: Combine traditional image processing to generate auxiliary labels, and fuse them with CNN prediction results at a weight of 7:3; Stable stage: When the accuracy of the validation set is >85%, only CNN independent detection is used.

7. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 5, characterized in that: Embedding a spatial-frequency domain attention module in the residual block includes: Spatial branch: extract local texture features through 3×3 convolution; Frequency domain branch: Fast Fourier transform (FFT) is performed on the feature map and key frequencies are enhanced through learnable filters; The fusion formula is: ; Where σ is the Sigmoid function, W s , W f are trainable weights.

8. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 7, characterized in that: Before the Fourier transform, the following steps are added: Perform wavelet decomposition on the preprocessed image to extract high-frequency components and low-frequency components; The high-frequency component is Fourier transformed and fused with the original image Fourier features according to the weights. The formula is: .

9. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 7, characterized in that: The training data augmentation of the system includes: random rotation (±5°); Brightness jitter (±10%); Add Gaussian noise (σ=0.01).

10. The liquid crystal screen mura defect detection system based on convolutional neural network according to claim 1, characterized in that: In the process of dynamic iterative optimization of network weights, the Adam optimizer update rule is used: Among them, m t and v t The first-order and second-order momentum of the Adam optimizer are used to adaptively adjust the learning rate and accelerate model convergence; η is the learning rate, which determines the step size of weight update; 1E-8 is a numerical stability constant to prevent the denominator from being zero; W t+1 The updated weights are used to adaptively adjust the learning rate, accelerate convergence and reduce oscillation.

Images