Display screen abnormal point detection method and system based on image processing

Abstract

The invention belongs to the technical field of display panel quality detection, and particularly relates to a display screen abnormal point detection method based on image processing, which comprises the following steps: capturing a self-luminous physical channel image of a display screen in real time through a polarization filtering array, and meanwhile, generating a virtual channel image by adopting a pre-trained ambient light shielding GAN network; pixel-level differential operation is carried out on the two-channel image to eliminate ambient light noise, and an illumination invariance correction image is output; the corrected image is input into a pyramid feature distillation network PFD to extract cross-scale defect features, the pyramid feature distillation network PFD comprises a three-level feature processing flow path, the angle of a light-transmitting axis is dynamically calibrated through a polarization filtering array to capture intrinsic radiation of a screen, a virtual channel is generated in combination with ambient light shielding, pixel-level difference and morphological filtering are executed, and the cross-scale defect features are obtained. The environment light noise suppression rate and the dark spot detection rate are improved, and the problems of the micro-defect omission rate and the high-reflection-area false alarm rate caused by illumination fluctuation of a production line in a traditional gray threshold value method are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of display panel quality detection, and particularly relates to a display screen abnormal point detection method and system based on image processing. BACKGROUND

[0002] With the continuous progress of science and technology, display panels have become an indispensable part of modern electronic devices. Whether it is a smartphone, tablet computer, or television and computer display, the quality of the display panel directly affects the user's visual experience.

[0003] Currently, when detecting the quality of mainstream display panels, the traditional gray threshold method is disturbed by environmental light fluctuations, resulting in a drift in the missed detection rate with the production line lighting conditions; texture distortion caused by the deformation of foldable screens leads to global threshold failure problems; and the classification model relies on artificial definition of features and cannot associate pixel abnormalities with the failure mechanism of the panel driving circuit.

[0004] Therefore, the application provides a display screen abnormal point detection method and system based on image processing. SUMMARY

[0005] In order to make up for the deficiencies of the prior art and solve at least one technical problem raised in the background art.

[0006] The technical scheme adopted by the application to solve its technical problems is: the display screen abnormal point detection method based on image processing comprises

[0007] The physical channel image of the self-luminous display screen is captured in real time through a polarized filter array, and a pre-trained environmental light shielding GAN network is used to generate a virtual channel image, pixel-level difference operation is performed on the two channel images to eliminate environmental light noise, and an illumination invariance correction image is output; wherein the light transmission axis direction of the polarized filter array is dynamically adjusted according to the screen normal vector;

[0008] The correction image is input into a pyramid feature distillation network PFD to extract cross-scale defect features, the pyramid feature distillation network PFD comprises three levels of feature processing flow paths: the first flow path extracts a 64x64 high-resolution feature map through a 3x3 convolution kernel to capture micron-level point defects, the second flow path extracts a 32x32 medium-scale feature map through a 5x5 hollow convolution to perceive sub-pixel-level texture abnormalities, and the third flow path extracts a 16x16 low-resolution feature map through a 7x7 deformable convolution to identify Mura patches; the three features are weighted and aggregated by a dynamic feature fusion layer, and the weight coefficient is adaptively generated from the regional information entropy;

[0009] The defect probability heat map is output based on a pyramid feature distillation network PFD, and a pixel-level dynamic segmentation threshold is calculated by a threshold generation network TG-Net, the threshold generation network TG-Net taking local statistical features of the heat map as input, including kurtosis K, skewness S and information entropy E in a local statistical period, and generating a threshold matrix through a fully connected layer mapping;

[0010] An abnormal pixel is located according to the pixel-level dynamic segmentation threshold, circuit traceability analysis is performed, and online model updating is triggered.

[0011] Preferably, the polarization filter array comprises MxN independently rotatable polarization units, each unit is equipped with a micro stepping motor, and a motor control signal is generated by a screen curvature detection module, curvature detection is performed by a laser triangulation method, and the density of measurement points is not less than 100 points / cm 2 .

[0012] Preferably, the weight coefficient wi of the dynamic feature fusion layer is:

[0013]

[0014] Wherein, Hi is the information entropy of the i-th feature map, Hj is the information entropy of the j-th feature map, sigma is a Sigmoid activation function, and tau is a temperature coefficient.

[0015] Preferably, the pyramid feature distillation network PFD injects a physical constraint attention module before feature fusion, and the module performs the following operations:

[0016] An RGB sub-pixel geometric mask is applied to the first flow path feature, and the mask template is consistent with the actual pixel arrangement of the display screen;

[0017] A direction gradient histogram constraint is performed on the second flow path feature to suppress the edge response of non-panel inherent direction;

[0018] A frequency domain band-stop filter is performed on the third flow path feature, and the stop band range corresponds to the fundamental frequency of the panel back plate circuit period texture.

[0019] Preferably, the fully connected layer of the threshold generation network TG-Net adopts a residual structure:

[0020] Tout = Tbase + W2 ReLU (BN (W1 [K, S, E]))

[0021] Wherein, Tbase is a preset basic threshold matrix according to the panel type, W1 and W2 are weight matrices, and BN is a batch normalization layer.

[0022] Preferably, the online model updating comprises:

[0023] Screening candidate negative samples with cosine distance of features greater than 0.7; performing 15%-30% random block erasing enhancement on the samples; updating the classifier using momentum contrast loss.

[0024] Preferably, an image processing-based display screen abnormal point detection system comprises:

[0025] Optical acquisition module: industrial camera with polarization filter array;

[0026] Preprocessing module: integrated FPGA chip, real-time execution of two-way light correction and feature map pre-extraction;

[0027] AI processing module: equipped with special ASIC chip, solidified pyramid feature distillation network PFD and threshold generation network TG-Net inference engine;

[0028] Defect analysis terminal: according to the threshold segmentation result, the abnormal pixel coordinates are located, and the corresponding drive circuit pins are inquired through the pre-constructed circuit mapping database.

[0029] Preferably, the FPGA chip and the ASIC chip are interconnected through a high-speed serial bus:

[0030] Image data: only the feature map blocks required by the pyramid feature distillation network PFD are transmitted, and the blocks are dynamically divided according to the information entropy threshold;

[0031] Control instruction: 4-byte compression encoding is adopted, including polarization angle, laser coordinate, model update flag.

[0032] Preferably, the circuit mapping database comprises a three-level index structure:

[0033] First level: panel model→drive IC model mapping table;

[0034] Second level: drive IC model→pin function allocation table;

[0035] Third level: pixel coordinates→row and column drive circuit coordinate conversion matrix, and the matrix coefficients are calibrated through panel electrical simulation.

[0036] Preferably, the ASIC chip integrates a two-stage pipeline architecture:

[0037] First stage: parallel computing pyramid feature distillation network PFD three-level feature flow path;

[0038] Second stage: executing feature fusion and threshold generation network TG-Net calculation, and adopting pulsating array to realize matrix multiplication and addition operation;

[0039] The two-stage pipeline is connected through a double buffering mechanism.

[0040] The beneficial effects of the present application are as follows:

[0041] 1. The image processing-based display screen abnormal point detection method and system of the present application, by dynamically calibrating the light transmission axis angle through a polarized light filter array to capture the intrinsic radiation of the screen, and generating a virtual channel combined with ambient light shielding, performing pixel-level difference and morphological filtering, to improve the ambient light noise suppression rate and dark point detection rate, solving the problem of micro-defect missed detection rate and high false alarm rate in high light reflection area caused by traditional gray threshold method due to production line illumination fluctuation;

[0042] 2. The image processing-based display screen abnormal point detection method and system of the present application, by threshold generation network TG-Net residual threshold framework, wherein the panel model base noise distribution is dynamically generated by local kurtosis K, skewness S and information entropy E, to reduce the false alarm rate and missed detection rate of flexible screen bending area, and solve the problem of global threshold failure caused by texture distortion due to foldable screen deformation. BRIEF DESCRIPTION OF DRAWINGS

[0043] The present application will be further described below with reference to the accompanying drawings.

[0044] Figure 1 is the method flowchart of the image processing-based display screen abnormal point detection method of the present application;

[0045] Figure 2 is the system flowchart of the image processing-based display screen abnormal point detection system of the present application. DETAILED DESCRIPTION

[0046] In order to make the technical means, creative features, purposes and effects realized by the present application easy to understand, the present application will be further described below with reference to the specific embodiments.

[0047] Embodiment one

[0048] As shown in Figure 1 , the image processing-based display screen abnormal point detection method of the present application embodiment comprises:

[0049] (a) Dual-path light correction: real-time capture the physical channel image of the display screen self-luminous through the polarized light filter array, and simultaneously generate a virtual channel image by using a pre-trained ambient light shielding GAN network, perform pixel-level difference operation on the two channel images to eliminate ambient light noise, and output the light invariance correction image; wherein the light transmission axis direction of the polarized light filter array is dynamically adjusted according to the screen normal vector, and the adjustment algorithm satisfies: θ = arctan(dz / dx) + π / 4 (θ is the rotation angle of the light transmission axis, dz / dx is the curvature change rate of the screen surface in the X direction, dz is the displacement amount of the screen surface normal direction; dx is the small displacement in the X direction of the screen plane);

[0050] (b) Multi-scale defect perception: The corrected image is input into a pyramid feature distillation network PFD to extract cross-scale defect features, which includes a three-level feature processing flow: the first flow extracts a 64x64 high-resolution feature map through a 3x3 convolution kernel to capture micron-level point defects, the second flow extracts a 32x32 medium-scale feature map through a 5x5 hollow convolution to perceive sub-pixel-level texture abnormalities, and the third flow extracts a 16x16 low-resolution feature map through a 7x7 deformable convolution to identify Mura patches; the three features are weighted and aggregated by a dynamic feature fusion layer, and the weight coefficient is adaptively generated from the regional information entropy;

[0051] (c) Adaptive threshold decision: Based on the defect probability heat map output by the pyramid feature distillation network PFD, a threshold generation network TG-Net is used to calculate the pixel-level dynamic segmentation threshold, which takes the local statistical features of the heat map as input, including kurtosis K, skewness S and information entropy E in the local statistical period, and generates a threshold matrix T(x,y) = a*tanh(β1*K+β2*S+β3*E)+γ through a fully connected layer, where a, β, γ are learnable parameters.

[0052] (d) Defect processing: According to the pixel-level dynamic segmentation threshold, abnormal pixels are located, circuit traceability analysis is performed, and online model updating is triggered.

[0053] According to the threshold segmentation result, the abnormal pixel coordinates are located, and the corresponding drive circuit pin is searched through the pre-constructed pixel-drive IC mapping database, which stores the topological connection relationship between each pixel point and the drive chip pin, and the mapping rule satisfies: PinID=(RowxColMax+Col)modN, where Row / Col is the pixel row / column number, ColMax is the total number of panel columns, and N is the number of drive IC pins.

[0054] Incremental model updating: When unknown class defects are detected, a few-shot learning protocol is triggered: select abnormal area images with confidence > 0.9, use difficult sample mining strategy to select high information negative samples, perform online model fine-tuning based on contrast loss function on embedded AI chip, and update period ≤ 50 samples.

[0055] In this embodiment, the polarization filter array includes MxN independently rotatable polarization units, each unit is equipped with a micro stepping motor, and the motor control signal is generated by a screen curvature detection module. The curvature detection uses a laser triangulation method, and the measurement point density is not less than 100 points / cm 2 .

[0056] In this embodiment, the weight coefficient wi of the dynamic feature fusion layer is:

[0057]

[0058] Where Hi is the information entropy of the i-th level feature map, Hj is the information entropy of the j-th level feature map, σ is the Sigmoid activation function, and τ is the temperature coefficient (initial value 0.5, which decays exponentially to a final value of 0.1 during training).

[0059] The formula for calculating information entropy Hi is:

[0060]

[0061] Where pk is the probability of gray value k appearing in the i-th level feature map. The higher the entropy value, the greater the texture complexity, and the higher the weight is given during fusion.

[0062] It should be explained that the Pyramid Feature Distillation Network (PFD) injects a Physically Constrained Attention (PCA) module before feature fusion, and this module performs the following operations:

[0063] An RGB sub-pixel geometric mask is applied to the first flow path features, and the mask template is consistent with the actual pixel arrangement of the display screen.

[0064] Apply directional gradient histogram constraints to the second flow path features to suppress edge responses that are not inherent to the panel orientation;

[0065] The third flow path features are subjected to frequency domain band-stop filtering, and the stopband range corresponds to the fundamental frequency of the periodic texture of the panel backplane circuit.

[0066] In this embodiment, the fully connected layers of the threshold generation network TG-Net adopt a residual structure:

[0067] Tout=Tbase+W2·ReLU(BN(W1·[K,S,E]))

[0068] Where Tbase is the base threshold matrix preset according to the panel type, W1 and W2 are weight matrices, and BN is the batch normalization layer.

[0069] It needs to be explained that the input feature layer: [K,S,E]: local statistical feature vectors;

[0070] K: Kurtosis, quantifies the sharpness of the distribution in a local window of a heatmap. Its function is to indicate potential point defects (such as bright / dark spots); it is used in display screen inspection to identify abnormal point defects. Referring to Table 1, kurtosis K is calculated using the following formula:

[0071]

[0072] Where Xi is the pixel gray value of the i-th level feature map, N is the number of pixels in the local window, ρ is the standard deviation of the pixel gray values ​​in the local window, and μ is the mean gray value of the local window.

[0073] Kurtosis value range Distribution characteristic Defect type indication $K>1$ Spiked distribution Point defects (bright / dark spots) $-1<K<1$ Approximately normal Normal area $K<-1$ Flat-topped distribution Mura patches

[0074] Table 1

[0075] S: Skewness, measures the asymmetry of the distribution, function: positive skewness detects bright spot clusters, negative skewness detects dark Mura. Among them, the skewness S is a statistical quantity for measuring the asymmetry of the probability distribution, which is used to quantify the skew direction of the brightness distribution in the display screen defect detection. Referring to Table 2, the skewness S is calculated by the following formula:

[0076]

[0077] Skewness value range Distribution characteristic Defect type indication S>0.5 Positive skew (right skew) Bright spot / high area defects -0.5<S<0.5 Approximately symmetric Normal area S<-0.5 Negative skew (left skew) Dark spot / low area defects

[0078] Table 2

[0079] E: Entropy, represents the complexity of the texture, function: high entropy area inhibits false judgment of the base texture.

[0080] Feature transformation layer W1·[K, S, E]: full connection weight matrix mapping; function: upgrade 3-dimensional statistical features to high-dimensional space, capture nonlinear relationship.

[0081] Normalization and activation layer BN(·): batch normalization, solve feature distribution drift caused by different panel bases, output standardized features with mean 0 and variance 1;

[0082] ReLU(·): rectified linear unit activation function, function: filter out negative interference and retain effective features.

[0083] Residual output layer W2·(·): full connection weight matrix (dimension 1xd), compresses hidden layer features into threshold adjustment quantity, Tbase: panel type related basic threshold matrix.

[0084] Residual mechanism innovation point: ΔT = W2·ReLU(BN(W1·[K, S, E]))

[0085] Tout = Tbase + ΔT

[0086] Among them, Tbase guarantees the stability of the basic detection, and ΔT dynamically adapts to the abnormal features of the local area, avoiding the failure of the traditional global threshold to the deformation of the flexible screen.

[0087] In the embodiment, the online model update comprises:

[0088] Screening candidate negative samples with a feature cosine distance greater than 0.7; performing 15%-30% random block erasing enhancement on the samples; and updating the classifier using momentum contrast loss.

[0089] Need to explain, incremental model update: when the unknown class defect is checked out, trigger the few-shot learning protocol: select the abnormal area image with confidence > 0.9, use the difficult sample mining strategy to select high information negative sample, based on the contrast loss function in the embedded AI chip to perform online model fine-tuning, update cycle ≤ 50 samples. Few-shot learning protocol includes negative sample mining strategy:

[0090] Step 1: Calculate the cosine distance of the candidate negative sample and the known defect class feature, and select the sample with distance > 0.7;

[0091] Step 2: Random block erasing data enhancement is performed on the selected sample, and the erasing area accounts for 15%-30%;

[0092] Step 3: Update the classifier using momentum contrast loss.

[0093] Example two

[0094] As Figure 2 shown, compared with example one, another embodiment of the present application is:

[0095] A display screen abnormal point detection system based on image processing is used to implement the above-mentioned display screen abnormal point detection method, which comprises:

[0096] Optical acquisition module: industrial camera containing polarization filter array, frame rate ≥ 120fps, pixel size ≤ 2.4μm;

[0097] Pretreatment module: integrated FPGA chip, real-time execution of double-channel light correction and feature map pre-extraction;

[0098] AI processing module: equipped with special ASIC chip, solidified pyramid feature distillation network PFD and threshold generation network TG-Net inference engine;

[0099] Laser calibration module: fiber laser containing galvanometer system;

[0100] Defect analysis terminal: containing circuit mapping database.

[0101] In this embodiment, the FPGA chip and the ASIC chip are interconnected through a high-speed serial bus:

[0102] Image data: only the feature map block required by the pyramid feature distillation network PFD is transmitted, and the block is dynamically divided according to the information entropy threshold;

[0103] Control command: using 4-byte compression encoding, including polarization angle, laser coordinate, model update flag.

[0104] In this embodiment, the circuit mapping database comprises a three-level index structure:

[0105] First level: panel model number to driving IC model number mapping table;

[0106] Second level: driving IC model number to pin function assignment table;

[0107] Third level: pixel coordinates to row-column driving circuit coordinate conversion matrix, matrix coefficients calibrated through panel electrical simulation.

[0108] In this embodiment, the ASIC chip integrates two-stage pipeline architecture:

[0109] First level: parallel computing pyramid feature distillation network PFD three-level feature flow path;

[0110] Second level: execute feature fusion and threshold generation network TG-Net calculation, adopt pulsating array to realize matrix multiplication and addition operation;

[0111] Wherein the two-stage pipeline is connected through double buffering mechanism, and the time delay jitter is less than or equal to 1.2 ms.

[0112] The above shows and describes the basic principles, main features and advantages of the present application. It should be understood by those skilled in the art that the present application is not limited to the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A display screen anomaly point detection method based on image processing, characterized in that: Comprising A physical channel image of the display screen self-luminance is captured in real time through a polarized filter array, while a pre-trained ambient light masking GAN network is used to generate a virtual channel image, pixel-level difference operation is performed on the two channel images to eliminate ambient light noise, and an illumination invariance correction image is output; wherein the transmission axis direction of the polarized filter array is dynamically adjusted according to the screen normal vector; The correction image is input into a pyramid feature distillation network PFD to extract cross-scale defect features, and the pyramid feature distillation network PFD includes three levels of feature processing flow: the first flow extracts a 64x64 high-resolution feature map through a 3x3 convolution kernel to capture micron-level point defects, the second flow extracts a 32x32 medium-scale feature map through a 5x5 hollow convolution to perceive sub-pixel-level texture abnormalities, and the third flow extracts a 16x16 low-resolution feature map through a 7x7 deformable convolution to identify Mura patches; The three features are weighted and aggregated through a dynamic feature fusion layer, and the weight coefficient is adaptively generated from the regional information entropy; A defect probability heat map is output based on the pyramid feature distillation network PFD, a pixel-level dynamic segmentation threshold is calculated using a threshold generation network TG-Net, the threshold generation network TG-Net takes local statistical features of the heat map as input, including kurtosis K, skewness S and information entropy E in the local statistical period, and generates a threshold matrix through a fully connected layer mapping; Abnormal pixels are located according to the pixel-level dynamic segmentation threshold, circuit traceability analysis is performed, and online model updating is triggered.

2. The image processing based display panel anomaly point detection method of claim 1, wherein: The polarization filter array comprises MxN independently rotatable polarization units, each unit is equipped with a micro stepping motor, and the motor control signal is generated by a screen curvature detection module, the curvature detection adopts a laser triangulation method, and the measuring point density is not less than 100 points / cm 2 .

3. The image processing based display panel anomaly point detection method of claim 1, wherein: The weight coefficient wi of the dynamic feature fusion layer is: Wherein, Hi is the information entropy of the i-th feature map, Hj is the information entropy of the j-th feature map, sigma is the Sigmoid activation function, and tau is the temperature coefficient.

4. The image processing based display panel anomaly point detection method of claim 1, wherein: The pyramid feature distillation network PFD injects a physical constraint attention module before feature fusion, which performs the following operations: Apply an RGB sub-pixel geometric mask to the first flow feature, and the mask template is consistent with the actual pixel arrangement of the display screen; Perform a histogram of oriented gradients constraint on the second flow feature to suppress edge responses in non-panel intrinsic directions; Perform frequency domain band-stop filtering on the third flow feature, and the stop band range corresponds to the fundamental frequency of the panel backplate circuit periodic texture.

5. The image processing based display panel anomaly point detection method of claim 1, wherein: The fully connected layer of the threshold generation network TG-Net uses a residual structure: Tout=Tbase+W2·ReLU(BN(W1·[K,S,E])) Wherein, Tbase is a preset base threshold matrix according to the panel type, W1 and W2 are weight matrices, and BN is a batch normalization layer.

6. The image processing based display panel anomaly point detection method of claim 1, wherein: The online model updating includes: Screening candidate negative samples with a feature cosine distance greater than 0.7; performing 15%-30% random block erasing enhancement on the samples; and updating the classifier using momentum contrast loss.

7. An image processing based display abnormal point detection system for implementing the display abnormal point detection method according to any one of claims 1-6, characterized in that: Comprising Optical acquisition module: industrial camera containing polarized filter array; Preprocessing module: integrated FPGA chip, real-time execution of two-way light correction and feature map pre-extraction; AI processing module: equipped with a special ASIC chip, solidifying the pyramid feature distillation network PFD and threshold generation network TG-Net inference engine; Defect analysis terminal: locate abnormal pixel coordinates according to threshold segmentation results, and reverse query corresponding driving circuit pins through pre-constructed circuit mapping database.

8. The image processing based display screen anomaly point detection system of claim 7, wherein: The FPGA chip and the ASIC chip are interconnected through a high-speed serial bus: Image data: only the feature map blocks required by the pyramid feature distillation network PFD are transmitted, and the blocks are dynamically divided according to an information entropy threshold; Control instruction: 4-byte compression encoding is adopted, and the control instruction includes a polarization angle, a laser coordinate and a model update flag.

9. The image processing based display screen anomaly point detection system of claim 7, wherein: The circuit mapping database comprises a three-level index structure: First level: panel model to driving IC model mapping table; Second level: driving IC model to pin function allocation table; Third level: pixel coordinate to row-column driving circuit coordinate conversion matrix, and the matrix coefficients are calibrated through panel electrical simulation.

10. The image processing based display screen anomaly point detection system of claim 7, wherein: The ASIC chip integrates a two-stage pipeline architecture: First stage: parallel computing pyramid feature distillation network PFD three-stage feature flow path; Second stage: executing feature fusion and threshold generation network TG-Net calculation, and adopting a systolic array to realize matrix multiplication and addition operation; The two-stage pipeline is connected through a double buffering mechanism.

Images