Image Processing-Based Method and System for Display Screen Anomaly Detection

Abstract

This invention belongs to the field of display panel quality inspection technology, specifically a display screen anomaly detection method based on image processing. It captures the physical channel image of the display screen's self-illuminating components in real time using a polarization filter array, while simultaneously generating a virtual channel image using a pre-trained ambient light masking GAN network. Pixel-level difference operations are performed on the two channel images to eliminate ambient light noise, outputting an illumination-invariant corrected image. The corrected image is then input into a pyramid feature distillation network (PFD) to extract cross-scale defect features. The PFD network contains a three-level feature processing flow path, dynamically calibrating the transmission axis angle using a polarization filter array to capture the screen's intrinsic radiation, and combining this with ambient light masking to generate a virtual channel. Pixel-level difference and morphological filtering are then performed to improve the ambient light noise suppression rate and dark spot detection rate, solving the problems of low micro-defect detection rate and high false alarm rate in high-reflectivity areas caused by fluctuations in production line illumination in traditional grayscale thresholding methods.

Description

Technical Field

[0001] This invention belongs to the field of display panel quality inspection technology, specifically a method and system for detecting abnormal points on a display screen based on image processing. Background Technology

[0002] With the continuous advancement of technology, display panels have become an indispensable part of modern electronic devices. Whether it's smartphones, tablets, televisions, or computer monitors, the quality of the display panel directly affects the user's visual experience.

[0003] In current mainstream display panel quality testing, the traditional grayscale threshold method is affected by ambient light fluctuations, causing the false negative rate to drift with the production line lighting conditions; the texture distortion caused by the deformation of foldable screens leads to the global threshold failure problem; the classification model relies on manually defined features and cannot correlate pixel anomalies with the failure mechanism of the panel driving circuit.

[0004] Therefore, the present invention provides a method and system for detecting abnormal points on a display screen based on image processing. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0006] The technical solution adopted by this invention to solve its technical problem is: the display screen abnormality detection method based on image processing described in this invention includes...

[0007] The physical channel image of the self-illuminating display screen is captured in real time by a polarization filter array, while a virtual channel image is generated by a pre-trained ambient light occlusion GAN network. Pixel-level difference operation is performed on the two channel images to eliminate ambient light noise and output an illumination invariance corrected image. The transmission axis direction of the polarization filter array is dynamically adjusted according to the screen normal vector.

[0008] The corrected image is input into the Pyramid Feature Distillation Network (PFD) to extract cross-scale defect features. The PFD consists of three-level feature processing pathways: the first pathway extracts a 64×64 high-resolution feature map using a 3×3 convolutional kernel to capture micron-level point defects; the second pathway extracts a 32×32 mid-scale feature map using a 5×5 dilated convolution to perceive sub-pixel-level texture anomalies; and the third pathway extracts a 16×16 low-resolution feature map using a 7×7 deformable convolution to identify Mura patches. The three feature paths are then weighted and aggregated by a dynamic feature fusion layer, with the weight coefficients adaptively generated by the regional information entropy.

[0009] Based on the Pyramid Feature Distillation Network (PFD) outputting a defect probability heatmap, a threshold generation network (TG-Net) is used to calculate pixel-level dynamic segmentation thresholds. The threshold generation network (TG-Net) takes the local statistical features of the heatmap as input, including kurtosis K, skewness S, and information entropy E within the local statistical period, and generates a threshold matrix through a fully connected layer.

[0010] Abnormal pixels are located based on pixel-level dynamic segmentation thresholds, and circuit source analysis and online model updates are performed.

[0011] Preferably, the polarization filter array comprises M×N independently rotatable polarization units, each equipped with a micro stepper motor. The motor control signal is generated by the screen curvature detection module, and the curvature detection adopts the laser triangulation method with a measurement point density of not less than 100 points / cm².

[0012] Preferably, the weight coefficients of the dynamic feature fusion layer for:

[0013]

[0014] in, Let Hi be the information entropy of the i-th level feature map, and Hj be the information entropy of the j-th level feature map. It is the Sigmoid activation function. This is the temperature coefficient.

[0015] Preferably, the pyramid feature distillation network (PFD) is injected with a physically constrained attention module before feature fusion. This module performs the following operations:

[0016] An RGB sub-pixel geometric mask is applied to the first flow path features, and the mask template is consistent with the actual pixel arrangement of the display screen.

[0017] Apply directional gradient histogram constraints to the second flow path features to suppress edge responses that are not inherent to the panel orientation;

[0018] The third flow path features are subjected to frequency domain band-stop filtering, and the stopband range corresponds to the fundamental frequency of the periodic texture of the panel backplane circuit.

[0019] Preferably, the fully connected layers of the threshold generation network TG-Net adopt a residual structure:

[0020]

[0021] in, This is a pre-defined basic threshold matrix based on the panel type. , Here is the weight matrix, and BN is the batch normalization layer.

[0022] Preferably, the online model update includes:

[0023] Candidate negative samples with feature cosine distance > 0.7 are selected; 15%-30% random block erasure enhancement is performed on the samples; the classifier is updated using momentum contrast loss.

[0024] Preferably, a display screen anomaly detection system based on image processing includes:

[0025] Optical acquisition module: Industrial camera with polarization filter array;

[0026] Preprocessing module: integrates FPGA chip to perform dual-channel illumination correction and feature map pre-extraction in real time;

[0027] AI processing module: Equipped with an ASIC chip, it integrates the Pyramid Feature Distillation Network (PFD) and the Threshold Generation Network (TG-Net) inference engine.

[0028] Defect analysis terminal: Locates the coordinates of abnormal pixels based on the threshold segmentation results, and looks up the corresponding driving circuit pins through a pre-built circuit mapping database.

[0029] Preferably, the FPGA chip and the ASIC chip are interconnected via a high-speed serial bus:

[0030] Image data: Only feature map blocks required by the Pyramid Feature Distillation Network (PFD) are transmitted, and the blocks are dynamically divided according to the information entropy threshold.

[0031] Control commands: 4-byte compressed encoding, including polarization angle, laser coordinates, and model update flag.

[0032] Preferably, the circuit mapping database contains a three-level index structure:

[0033] Level 1: Panel Model → Driver IC Model Mapping Table;

[0034] Level 2: Driver IC Model → Pin Function Allocation Table;

[0035] Level 3: Pixel coordinates → row and column drive circuit coordinate transformation matrix, matrix coefficients are calibrated through panel electrical simulation.

[0036] Preferably, the ASIC chip integrates a two-stage pipeline architecture:

[0037] Level 1: Parallel computation of the pyramid feature distillation network (PFD) with three levels of feature flow paths;

[0038] The second stage: The TG-Net feature fusion and threshold generation network is used for computation, and matrix multiplication and addition operations are implemented using a systolic array.

[0039] The two-stage production lines are connected by a double buffer mechanism.

[0040] The beneficial effects of this invention are as follows:

[0041] 1. The image processing-based display screen abnormality detection method and system of the present invention captures the intrinsic radiation of the screen by dynamically calibrating the transmission axis angle of the polarization filter array, and generates a virtual channel by combining ambient light occlusion, and performs pixel-level differential and morphological filtering to improve the ambient light noise suppression rate and dark spot detection rate, and solve the problems of micro-defect missed detection rate and high reflectivity false alarm rate caused by the fluctuation of production line illumination in the traditional grayscale threshold method;

[0042] 2. The image processing-based display screen anomaly detection method and system of the present invention uses the TG-Net residual threshold framework, which binds the panel model's base noise distribution and is dynamically generated by local kurtosis K, skewness S, and information entropy E, to reduce the false alarm rate and false negative rate in the bending area of ​​flexible screens and solve the problem of global threshold failure caused by texture distortion caused by deformation of foldable screens. Attached Figure Description

[0043] The invention will now be further described with reference to the accompanying drawings.

[0044] Figure 1 This is a flowchart of the display screen anomaly detection method based on image processing according to the present invention;

[0045] Figure 2 This is a system flowchart of the display screen anomaly detection system based on image processing according to the present invention. Detailed Implementation

[0046] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0047] Example 1

[0048] like Figure 1 As shown in the embodiment of the present invention, the display screen anomaly detection method based on image processing includes:

[0049] (a) Dual-channel illumination correction: The physical channel image of the display screen's self-illuminating light is captured in real time using a polarization filter array, while a virtual channel image is generated using a pre-trained ambient light masking GAN network. Pixel-level difference operations are performed on the two channel images to eliminate ambient light noise, and an illumination-invariant corrected image is output. The transmission axis direction of the polarization filter array is dynamically adjusted according to the screen normal vector, and the adjustment algorithm satisfies: (θ is the rotation angle of the transmission axis,) To characterize the rate of change of curvature of the screen surface in the X direction, This represents the displacement along the normal direction of the screen surface. (This refers to a small displacement of the screen plane in the X direction).

[0050] (b) Multi-scale defect perception: The corrected image is input into the Pyramid Feature Distillation Network (PFD) to extract cross-scale defect features. The PFD contains three-level feature processing paths: the first path extracts a 64×64 high-resolution feature map through a 3×3 convolutional kernel to capture micron-level point defects; the second path extracts a 32×32 mid-scale feature map through a 5×5 dilated convolution to perceive sub-pixel-level texture anomalies; and the third path extracts a 16×16 low-resolution feature map through a 7×7 deformable convolution to identify Mura patches. The three features are weighted and aggregated by a dynamic feature fusion layer, and the weight coefficients are adaptively generated by the regional information entropy.

[0051] (c) Adaptive Threshold Decision: Based on the defect probability heatmap output by the Pyramid Feature Distillation Network (PFD), a threshold generation network (TG-Net) is used to calculate the pixel-level dynamic segmentation threshold. The threshold generation network (TG-Net) takes the local statistical features of the heatmap as input, including kurtosis K, skewness S, and information entropy E within the local statistical period, and generates a threshold matrix through a fully connected layer. , where α, β, γ are learnable parameters;

[0052] (d) Defect handling: Locate abnormal pixels based on pixel-level dynamic segmentation thresholds, perform circuit source analysis, and trigger online model updates.

[0053] The coordinates of abnormal pixels are located based on the threshold segmentation results. The corresponding driver circuit pins are then looked up using a pre-built pixel-driver IC mapping database. This database stores the topological connection relationship between each pixel and the driver chip pins, and the mapping rules satisfy the following: ,in The row and column numbers are pixels. N represents the total number of panel columns, and N represents the number of pins on the driver IC.

[0054] Incremental model update: When an unknown category defect is detected, a few-shot learning protocol is triggered: images of abnormal regions with confidence > 0.9 are selected, and a hard sample mining strategy is used to screen high-information negative samples. Based on the contrastive loss function, online model fine-tuning is performed on the embedded AI chip, with an update cycle of ≤ 50 samples.

[0055] In this embodiment, the polarization filter array includes M×N independently rotatable polarization units, each equipped with a micro stepper motor. The motor control signal is generated by the screen curvature detection module. The curvature detection adopts the laser triangulation method, and the measurement point density is not less than 100 points / cm².

[0056] In this embodiment, the weight coefficients of the dynamic feature fusion layer for:

[0057]

[0058] in, The information entropy of the i-th level feature map. The information entropy of the j-th level feature map, It is the Sigmoid activation function. This is the temperature coefficient (initial value 0.5, exponentially decaying to a final value of 0.1 during training).

[0059] Information entropy The calculation formula is: in, represents the probability of grayscale value k appearing in the i-th level feature map. The higher the entropy value, the greater the texture complexity, and it is given higher weight during fusion.

[0060] It should be explained that the Pyramid Feature Distillation Network (PFD) injects a physically constrained attention module before feature fusion. This module performs the following operations:

[0061] An RGB sub-pixel geometric mask is applied to the first flow path features, and the mask template is consistent with the actual pixel arrangement of the display screen.

[0062] Apply directional gradient histogram constraints to the second flow path features to suppress edge responses that are not inherent to the panel orientation;

[0063] The third flow path features are subjected to frequency domain band-stop filtering, and the stopband range corresponds to the fundamental frequency of the periodic texture of the panel backplane circuit.

[0064] In this embodiment, the fully connected layers of the threshold generation network TG-Net adopt a residual structure:

[0065]

[0066] in, This is a pre-defined basic threshold matrix based on the panel type. , Here is the weight matrix, and BN is the batch normalization layer.

[0067] It needs to be explained that the input feature layer is: Local statistical eigenvectors;

[0068] K: Kurtosis, quantifies the sharpness of the distribution in a local window of a heatmap. Its function is to indicate potential point defects (such as bright / dark spots); it is used in display screen inspection to identify abnormal point defects. Referring to Table 1, kurtosis K is calculated using the following formula:

[0069]

[0070] Where Xi is the pixel grayscale value of the i-th level feature map, and N is the number of pixels in the local window. The standard deviation of the pixel grayscale values ​​in the local window. This represents the average grayscale value of a local window.

[0071] Kurtosis range Distribution characteristics Defect type indication K>1 Peak distribution Point defects (bright / dark spots) -1<K<1 Approximate normal Normal area K<-1 Flat-top distribution Mura plaques

[0072] Table 1

[0073] S: Skewness, measures the asymmetry of the distribution. Its function is to detect bright spot clusters with positive skewness and dark areas (mura) with negative skewness. Skewness S is a statistical measure of probability distribution asymmetry, used in display defect detection to quantify the direction of brightness distribution skewness. Referring to Table 2, skewness S is calculated using the following formula:

[0074]

[0075] Skewness range Distribution characteristics Defect type indication S>0.5 Rightward deviation (slant) Highlights / highlight area defects -0.5<S<0.5 Approximately symmetric Normal area S<-0.5 Negative bias (left bias) Dark spot / low-brightness area defects

[0076] Table 2

[0077] E: Information entropy, which represents texture complexity. Its function is to suppress misjudgment of the base texture in high-entropy regions.

[0078] Feature Transformation Layer Fully connected weight matrix mapping; Function: Upgrades 3D statistical features to a higher-dimensional space to capture nonlinear relationships.

[0079] Normalization and activation layer BN(·): Batch normalization solves the feature distribution drift caused by different panel bases and outputs standardized features with mean 0 and variance 1.

[0080] ReLU(·): Modified linear unit activation function. Its function is to filter out negative interference and retain effective features.

[0081] The residual output layer W2·(·) is a fully connected weight matrix (dimension 1×d) that compresses the hidden layer features into a threshold adjustment. : Basic threshold matrix related to panel type.

[0082] Innovation points of residual mechanism:

[0083]

[0084] in, Ensure the stability of basic testing. It dynamically adapts to abnormal features in local areas, avoiding the failure of traditional global thresholds to address the deformation of flexible screens.

[0085] In this embodiment, the online model update includes:

[0086] Candidate negative samples with feature cosine distance > 0.7 are selected; 15%-30% random block erasure enhancement is performed on the samples; the classifier is updated using momentum contrast loss.

[0087] It needs to be explained that the incremental model update involves triggering a few-shot learning protocol when an unknown category defect is detected. This protocol selects images of anomalous regions with a confidence level > 0.9, employs a hard sample mining strategy to filter high-information negative samples, and performs online model fine-tuning on an embedded AI chip based on a contrastive loss function. The update cycle is ≤ 50 samples. The few-shot learning protocol includes a negative sample mining strategy.

[0088] Step 1: Calculate the feature cosine distance between the candidate negative sample and the known defect category, and filter samples with a distance > 0.7;

[0089] Step 2: Perform random block erasure data augmentation on the selected samples, with the erasure area accounting for 15%-30%;

[0090] Step 3: Update the classifier using momentum contrast loss.

[0091] Example 2

[0092] like Figure 2 As shown in Example 1, another embodiment of the present invention is as follows:

[0093] A display screen anomaly detection system based on image processing, used to implement the above-mentioned display screen anomaly detection method, comprising:

[0094] Optical acquisition module: Industrial camera with polarization filter array, frame rate ≥120fps, pixel size ≤2.4μm;

[0095] Preprocessing module: integrates FPGA chip to perform dual-channel illumination correction and feature map pre-extraction in real time;

[0096] AI processing module: Equipped with an ASIC chip, it integrates the Pyramid Feature Distillation Network (PFD) and the Threshold Generation Network (TG-Net) inference engine.

[0097] Laser calibration module: Fiber laser including galvanometer system;

[0098] Defect analysis terminal: includes circuit mapping database.

[0099] In this embodiment, the FPGA chip and the ASIC chip are interconnected via a high-speed serial bus:

[0100] Image data: Only feature map blocks required by the Pyramid Feature Distillation Network (PFD) are transmitted, and the blocks are dynamically divided according to the information entropy threshold.

[0101] Control commands: 4-byte compressed encoding, including polarization angle, laser coordinates, and model update flag.

[0102] In this embodiment, the circuit mapping database includes a three-level index structure:

[0103] Level 1: Panel Model → Driver IC Model Mapping Table;

[0104] Level 2: Driver IC Model → Pin Function Allocation Table;

[0105] Level 3: Pixel coordinates → row and column drive circuit coordinate transformation matrix, matrix coefficients are calibrated through panel electrical simulation.

[0106] In this embodiment, the ASIC chip integrates a two-stage pipeline architecture:

[0107] Level 1: Parallel computation of the pyramid feature distillation network (PFD) with three levels of feature flow paths;

[0108] The second stage: The TG-Net feature fusion and threshold generation network is used for computation, and matrix multiplication and addition operations are implemented using a systolic array.

[0109] The two-stage pipeline is connected by a double buffering mechanism, with a latency jitter of ≤1.2ms.

[0110] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A display screen anomaly detection method based on image processing, characterized in that: include The physical channel image of the self-illuminating display screen is captured in real time by a polarization filter array, while a virtual channel image is generated by a pre-trained ambient light occlusion GAN network. Pixel-level difference operation is performed on the two channel images to eliminate ambient light noise and output an illumination invariance corrected image. The transmission axis direction of the polarization filter array is dynamically adjusted according to the screen normal vector. The corrected image is input into the Pyramid Feature Distillation Network (PFD) to extract cross-scale defect features. The PFD contains a three-level feature processing flow: the first flow extracts a 64×64 high-resolution feature map through a 3×3 convolution kernel to capture micron-level point defects; the second flow extracts a 32×32 mid-scale feature map through a 5×5 dilated convolution to perceive sub-pixel-level texture anomalies; and the third flow extracts a 16×16 low-resolution feature map through a 7×7 deformable convolution to identify Mura patches. The three features are weighted and aggregated by a dynamic feature fusion layer, and the weight coefficients are adaptively generated by the regional information entropy. Based on the Pyramid Feature Distillation Network (PFD) outputting a defect probability heatmap, a threshold generation network (TG-Net) is used to calculate pixel-level dynamic segmentation thresholds. The threshold generation network (TG-Net) takes the local statistical features of the heatmap as input, including kurtosis K, skewness S, and information entropy E within the local statistical period, and generates a threshold matrix through a fully connected layer. Kurtosis K refers to the distribution sharpness of the local window of the heatmap, skewness S is a statistic that measures the asymmetry of the probability distribution, and information entropy E characterizes the texture complexity. Abnormal pixels are located based on pixel-level dynamic segmentation thresholds, and circuit source analysis and online model updates are performed.

2. The display screen anomaly detection method based on image processing according to claim 1, characterized in that: The polarization filter array contains M×N independently rotatable polarization units, each equipped with a micro stepper motor. The motor control signal is generated by the screen curvature detection module. Curvature detection adopts the laser triangulation method, and the measurement point density is not less than 100 points / cm².

3. The display screen anomaly detection method based on image processing according to claim 1, characterized in that: The weight coefficients of the dynamic feature fusion layer for: in, For the first Information entropy of the level feature map For the first Information entropy of the level feature map It is the Sigmoid activation function. This is the temperature coefficient.

4. The display screen anomaly detection method based on image processing according to claim 1, characterized in that: The pyramid feature distillation network (PFD) injects a physically constrained attention module before feature fusion. This module performs the following operations: An RGB sub-pixel geometric mask is applied to the first flow path features, and the mask template is consistent with the actual pixel arrangement of the display screen. Apply directional gradient histogram constraints to the second flow path features to suppress edge responses that are not inherent to the panel orientation; The third flow path features are subjected to frequency domain band-stop filtering, and the stopband range corresponds to the fundamental frequency of the periodic texture of the panel backplane circuit.

5. The display screen anomaly detection method based on image processing according to claim 1, characterized in that: The fully connected layers of the threshold generation network TG-Net employ a residual structure: in, This is a pre-defined basic threshold matrix based on the panel type. , is the weight matrix, and BN is the batch normalization layer.

6. The display screen anomaly detection method based on image processing according to claim 1, characterized in that: The online model update includes: Candidate negative samples with feature cosine distance > 0.7 are selected; 15%-30% random block erasure enhancement is performed on the samples; the classifier is updated using momentum contrast loss.

7. A display screen anomaly detection system based on image processing, used to implement the display screen anomaly detection method as described in any one of claims 1-6, characterized in that: Include: Optical acquisition module: Industrial camera with polarization filter array; Preprocessing module: integrates FPGA chip to perform dual-channel illumination correction and feature map pre-extraction in real time; AI processing module: Equipped with an ASIC chip, it integrates the Pyramid Feature Distillation Network (PFD) and the Threshold Generation Network (TG-Net) inference engine. Defect analysis terminal: Locates the coordinates of abnormal pixels based on the threshold segmentation results, and looks up the corresponding driving circuit pins through a pre-built circuit mapping database.

8. The display screen anomaly detection system based on image processing according to claim 7, characterized in that: The FPGA chip and the ASIC chip are interconnected via a high-speed serial bus. Image data: Only feature map blocks required by the Pyramid Feature Distillation Network (PFD) are transmitted, and the blocks are dynamically divided according to the information entropy threshold. Control commands: 4-byte compressed encoding, including polarization angle, laser coordinates, and model update flag.

9. The display screen anomaly detection system based on image processing according to claim 7, characterized in that: The circuit mapping database contains a three-level index structure: Level 1: Panel Model → Driver IC Model Mapping Table; Level 2: Driver IC Model → Pin Function Allocation Table; Level 3: Pixel coordinates → row and column drive circuit coordinate transformation matrix, matrix coefficients are calibrated through panel electrical simulation.

10. The display screen anomaly detection system based on image processing according to claim 7, characterized in that: The ASIC chip integrates a two-stage pipeline architecture: Level 1: Parallel computation of the pyramid feature distillation network (PFD) with three levels of feature flow paths; The second stage: The TG-Net feature fusion and threshold generation network is used for computation, and matrix multiplication and addition operations are implemented using a systolic array. The two-stage production lines are connected by a double buffer mechanism.

Images