A machine vision-based liquid crystal panel pixel abnormality recognition method and system

Abstract

The application discloses a kind of liquid crystal panel pixel abnormality identification method and system based on machine vision, it is related to liquid crystal panel technical field.The method steps include: the visible light image and luminance signal of liquid crystal panel are collected and preprocessed, obtain standardized image and luminance time series data;Image segmentation is carried out to standardized image by improving image cascade semantic segmentation algorithm, and pixel abnormality candidate area is obtained;Based on pixel abnormality candidate area, feature extraction is carried out to standardized image and luminance time series data, and pixel abnormality feature vector is obtained;Pixel abnormality feature vector is input into SE-ResNet classifier, and abnormal type result is output, and pixel abnormality grade is divided based on abnormal type result, so as to realize liquid crystal panel pixel abnormality identification.

Description

Technical Field

[0001] This invention relates to the field of liquid crystal panel technology, specifically to a method for identifying pixel anomalies in liquid crystal panels based on machine vision. Background Technology

[0002] As a core component of terminal devices such as televisions, smartphones, and monitors, the display quality of LCD panels directly affects user experience and product competitiveness. During the panel production process, factors such as material characteristics, manufacturing process fluctuations, and equipment precision can easily lead to pixel abnormalities such as bright spots, color shifts, and flickering. These defects not only damage the integrity of the image but may also trigger a chain of failures and shorten the panel's lifespan.

[0003] Traditional methods often employ image segmentation based on fixed thresholds. This method first acquires images of the LCD panel in solid color display mode using an industrial camera, and then sets a fixed brightness threshold based on historical detection data or human experience. Areas in the image whose pixel brightness values ​​exceed the threshold range are directly identified as abnormal areas, thus completing the screening of pixel anomalies.

[0004] However, in actual production, LCD panels need to be adapted to various display modes such as solid color, grayscale gradient, and complex images. The brightness difference between normal and abnormal areas varies significantly under different modes. Fixed thresholds cannot dynamically adapt to such scene changes, which leads to deviations in the segmentation of abnormal areas with tiny pixel anomalies or blurred boundaries. Ultimately, it is difficult to accurately identify low-contrast and small-sized pixel defects, and it cannot meet the high-precision and high-consistency testing requirements in industrial production. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a machine vision-based method for identifying pixel anomalies in liquid crystal panels, thereby resolving the problems existing in the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a machine vision-based method for identifying pixel anomalies in a liquid crystal panel, comprising the following steps: Step S1: Acquire the visible light image and brightness signal of the LCD panel and perform preprocessing to obtain standardized image and brightness timing data; Step S2: Perform image segmentation on the standardized image using an improved image concatenated semantic segmentation algorithm to obtain candidate regions for pixel anomalies; Step S3: Based on the candidate regions of pixel anomalies, perform feature extraction on the standardized image and brightness time series data to obtain pixel anomaly feature vectors; Step S4: Input the pixel anomaly feature vector into the SE-ResNet classifier, output the anomaly type result, classify the pixel anomaly level based on the anomaly type result, and thus realize the pixel anomaly identification of the LCD panel.

[0007] Preferably, the step of acquiring the visible light image and brightness signal of the liquid crystal panel and performing preprocessing to obtain standardized image and brightness time-series data includes the following specific steps: Visible light images of the LCD panel are captured by an industrial camera; the brightness signal of the LCD panel is collected by a photometer to obtain a brightness time sequence; and the trigger synchronization between the industrial camera and the photometer is achieved by a synchronous trigger signal generator. The acquired visible light image is filtered using a Butterworth low-pass filter to obtain a filtered image; the filtered image is then calibrated for color and brightness using an adaptive histogram matching algorithm to obtain a standardized image. The acquired luminance time series was denoised by Gaussian filtering to obtain luminance time series data.

[0008] Preferably, the step of segmenting the standardized image using an improved image concatenated semantic segmentation algorithm to obtain candidate regions for pixel anomalies specifically involves: The improved image cascaded semantic segmentation algorithm uses Res2Net50 as the backbone feature extraction network and adopts an artifact removal and dilated residual network. After Res2Net50 feature extraction, it obtains abnormal candidate regions through a bidirectional pyramid pooling module, an attention pyramid enhancement module, and a cascaded feature fusion module. The specific calculation of the artifact removal and holed residual network is as follows: ; in, The input feature map is the DDRN dilated convolution. Indicates the expansion rate The dilated convolution operation, is the dilated convolution kernel of DDRN, and p is the pixel coordinate in the output feature map. convolution kernel The set of pixels, where b is the convolution kernel. The pixel coordinates in the image. For the expansion rate of DDRN, This represents the pixel value at the dilated sampling position corresponding to the output pixel p in the input feature map.

[0009] Preferably, the bidirectional pyramid pooling module is as follows: The bidirectional pyramid pooling module uses bidirectional propagation, following a top-down path, with intermediate feature layers. The formula for calculation is: ; in, Input weights for the i-th layer, Pass weights to the intermediate layer. For upsampling operation, The minimum value is 0.0001 to avoid the denominator being zero. This represents the intermediate feature map of the i-th layer in the top-down path. This is a 3×3 convolution operation. This is the original pooling feature map of the i-th layer output by the bidirectional pyramid pooling module. This is the intermediate feature map of the (i+1)th layer; Output feature layer along bottom-up path The formula for calculation is: ; in, This is the output feature map of the j-th layer in the bottom-up path. This is the original pooling feature map of the j-th layer output by the bidirectional pyramid pooling module. Input weights for the j-th layer, For intermediate layer weights, This is the intermediate feature map of the j-th layer output by the top-down path. To pass weights to lower layers, For downsampling operation, This is the output feature map of the (j-1)th layer.

[0010] Preferably, the attention pyramid enhancement module is as follows: The attention pyramid enhancement module employs three mechanisms: self-attention, downward attention, and upward attention. The downward attention mechanism calculates vector similarity independently for each attention head, and the attention graph is as follows: ; in, For the attention graph output function of the downward attention mechanism, For the nth learnable aggregation weight of the downward attention mechanism, Find the squared negative Euclidean distance between the query matrix and the key matrix for the nth feature block.

[0011] Preferably, the step of extracting features from standardized images and brightness time-series data based on pixel anomaly candidate regions to obtain pixel anomaly feature vectors includes the following steps: Based on pixel anomaly candidate regions, local images of each pixel anomaly candidate region are cropped from normalized images and brightness temporal data. and local brightness time sequence ; Local image of each pixel anomaly candidate region and local brightness time sequence Feature extraction is performed, and the extracted features include spatial features, chromaticity features, temporal features, and luminance features, as well as spatial distribution index, luminance difference index, temporal stability index, and chromaticity deviation index. The extracted feature rows are normalized to obtain pixel anomaly feature vectors. .

[0012] Preferably, the chromaticity features are as follows: Calculate the overall colorimetric deviation : ; in, This refers to the overall deviation of the chromaticity components. Due to color deviation, and For correction factors, , and These are the weighting coefficients. This is to account for overall color deviation.

[0013] Preferably, the step of inputting the pixel anomaly feature vector into the SE-ResNet classifier and outputting the anomaly type result includes the following specific steps: The SE-ResNet classifier uses the SE-ResNet18 network, whose structure consists of an input layer, four residual block groups, a global average pooling layer, and a fully connected classification layer. Each residual block group contains several SE-ResNet basic modules. The core computations of SE-ResNet include compression, excitation, and recalibration. The SE-ResNet classifier is trained. After training, the pixel anomaly feature vector is input into the SE-ResNet classifier, and the anomaly type result is output.

[0014] Preferably, the step of classifying pixel anomaly levels based on anomaly type results specifically involves: The spatial distribution index, luminance difference index, temporal stability index, and chromaticity deviation index are normalized to obtain normalized index values. ; The weighting coefficients of each index are adjusted based on the anomaly type results. The weighting allocation formula is as follows: ; in, The basic weights for each index, This represents the correlation coefficient between the anomaly type results and the index. These are the final weighting coefficients; Based on the final weighting coefficient and the normalized exponent value A weighted sum is performed to obtain the anomaly level score, and the pixel anomaly level is classified according to the anomaly level score.

[0015] A machine vision-based liquid crystal panel pixel anomaly recognition system includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the above method.

[0016] This invention provides a machine vision-based method for identifying pixel anomalies in liquid crystal panels, involving machine learning and deep learning technologies, which has the following beneficial effects: (1) In the production of LCD panels, pixel anomalies often manifest as small isolated or continuous clusters. The pixel-level segmentation network based on Res2Net50-ICNet captures multi-scale features through hierarchical residual connections, which can accurately locate 1-2 pixel-level micro anomalies. This solves the problem of fuzzy identification of small component anomalies by traditional segmentation, and provides a precise regional basis for subsequent feature extraction and classification, which is suitable for the need for efficient positioning of panel defects in industrial scenarios.

[0017] (2) An improved image segmentation algorithm based on artifact removal hollow residual network + bidirectional pyramid pooling + attention pyramid enhancement is based on Res2Net50-ICNet. The algorithm eliminates grid artifacts through artifact removal hollow residual network, strengthens feature fusion through bidirectional pyramid pooling, and establishes cross-dimensional interaction through attention pyramid enhancement module. This further improves the robustness and boundary accuracy of segmentation, makes subsequent feature extraction more targeted, makes classification and grading results more reliable, effectively adapts to panel quality detection in complex production environments, and reduces production losses caused by misjudgment. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the steps of a machine vision-based method for identifying pixel anomalies in a liquid crystal panel, as proposed in this invention. Figure 2 This is a step hierarchy diagram of obtaining candidate regions for pixel anomalies in a machine vision-based liquid crystal panel pixel anomaly identification method proposed in this invention. Figure 3 This is a step hierarchy diagram of obtaining the anomaly type result in a machine vision-based liquid crystal panel pixel anomaly identification method proposed in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1-3 The present invention provides a technical solution: a method for identifying pixel anomalies in liquid crystal panels based on machine vision.

[0022] Step S1: Acquire the visible light image and brightness signal of the LCD panel and perform preprocessing to obtain standardized image and brightness timing data; A high-resolution CCD industrial camera with a resolution of at least 8 megapixels (model TDS-VCXU-201C.R recommended) is used, paired with a 25mm fixed-focus low-distortion lens. Under dark conditions, it captures visible light images of the LCD panel in standard display modes. These standard display modes include solid color modes (red, green, blue, white), grayscale gradient modes, and complex image modes (comprehensive test images including faces, color marks, and grayscale transition bars), ensuring coverage of triggering scenarios for different pixel anomalies. Simultaneously, an accuracy of ≤±0.1cd / m² is employed. 2 The high-precision photometer acquires the brightness signal of the same detection area at a sampling frequency of 10ms / time, and continuously acquires 100 time points to form a brightness time sequence I(t). The hardware trigger synchronization between the CCD industrial camera and the high-precision photometer is realized through a synchronous trigger signal generator. The trigger signal generator outputs a synchronous pulse signal with a frequency of 100Hz, which is connected to the external trigger interface of the camera and the photometer respectively, ensuring that the two start acquisition simultaneously under the trigger of the same pulse signal, and realizing strict synchronous acquisition of visible light image and brightness time sequence data.

[0023] During the data acquisition process, the relative distance between the CCD camera and the LCD panel was fixed at 30-50cm using a position controller, the camera exposure time was set to 10-20ms, and the white balance parameter was calibrated to a 6500K standard light source to ensure the consistency of the acquired data.

[0024] The acquired visible light image is filtered using a Butterworth low-pass filter to eliminate periodic texture interference from the liquid crystal panel array, resulting in a filtered image. The core filter transfer function is:

[0025] in, The filter transfer function, The angular frequency of the visible light image. The cutoff angular frequency (derived from the cutoff frequency) The result of the conversion is ), The pixel arrangement period of the LCD panel determines the mainstream 100-200ppi panel. The value is 50-80Hz, and n is the filter order, which is 2-4, to balance the texture suppression effect and the fidelity of pixel abnormal signals.

[0026] Subsequently, the filtered image is calibrated for color and brightness using an adaptive histogram matching algorithm. Using the image of a defect-free standard panel as a reference template, the cumulative histogram distribution function of each color channel (R, G, B) of the image to be processed is calculated. Through mapping transformation, the color distribution of the image to be processed is made consistent with the reference template, eliminating color and brightness deviations caused by ambient light fluctuations and camera parameter differences, thus obtaining a standardized image.

[0027] The acquired luminance time-series sequence was denoised using Gaussian filtering to obtain luminance time-series data, which is as follows:

[0028] in, Here, k is the temporal neighborhood window offset, and K is the window size (ranging from 3 to 5, corresponding to a window length of 7 to 11 time points). The standard deviation is Gaussian, set according to the noise intensity of the luminance signal, with a value ranging from 0.8 to 1.2. This ensures that random noise is smoothly suppressed while preserving the temporal variation characteristics of pixel dynamic anomalies. This is the brightness timing data.

[0029] After preprocessing, standardized images (with a uniform resolution of 3840×2160 pixels) and luminance time-series data are output. This provides high-quality, low-interference input data for subsequent abnormal region segmentation.

[0030] Step S2: Perform image segmentation on the standardized image using an improved image concatenated semantic segmentation algorithm to obtain candidate regions for pixel anomalies; By improving the image concatenated semantic segmentation algorithm, image segmentation is performed on the standardized image to obtain candidate regions for pixel anomalies, as detailed below: The input consists of a standardized image and an improved image cascade semantic segmentation algorithm based on Res2Net50-ICNet. This algorithm uses Res2Net50 as the backbone feature extraction network and replaces the basic feature extraction module of the original ICNet. Its core improvement lies in building hierarchical residual connections in a single Bottleneck residual structure to achieve finer-grained multi-scale feature capture.

[0031] Specifically, each residual module of Res2Net50 reduces the input feature map through 1×1 convolution and then splits it into g feature subsets along the channel dimension (the splitting scale g takes the value of 4, adapting to the scale range of pixel anomalies in the liquid crystal panel), where the first feature subset is directly output, and the second feature subset is output after being processed by 3×3 convolution. The i-th (2 < i ≤ g) feature subset is first added to the output of the previous feature subset and then output through 3×3 convolution, that is , where is the output feature map of the i-th feature subset. The outputs of all feature subsets are concatenated and then the channel dimension is adjusted through 1×1 convolution, and a residual connection is added to obtain the module output. This structure enables the feature map to contain different combinations of receptive field scales. Compared with the feature extraction network of the original ICNet, the ability to capture features of 1-2 pixel-level micro-anomalies (such as isolated dead pixels) is significantly improved.

[0032] To eliminate the interference of grid artifacts introduced by dilated convolution on the boundaries of abnormal regions, the original dilated convolution structure in the network is replaced with an artifact-removing dilated residual network (DDRN). The network structure of DDRN is divided into 9 stages according to the stage. The operators, output channel numbers C, repetition times n, dilation rates d, and strides s of each stage are set as follows: stage1 uses the Conv7×7 operator (C = 16, n = 1, d = 1, s = 1), stage2 uses the Conv3×3 operator (C = 16, n = 1, d = 1, s = 1), stage3 uses the Conv3×3 operator (C = 32, n = 1, d = 1, s = 2), stage4-5 use the Bottleneck operator (C is 256 and 512 respectively, n is 3 and 4 respectively, d is 1 for both, s is 2 for both), stage6-7 use the Bottleneck operator (C is 1024 and 2048 respectively, n is 6 and 3 respectively, d is 2 and 4 respectively, s is 1 for both), stage8-9 use the Conv3×3 operator (C is 512 for both, n is 1 for both, d is 2 and 1 respectively, and there is no stride setting).

[0033] It should be noted that the above channel number settings take into account both the feature mapping rules of the native Res2Net50 network and the scenario requirements of LCD panel pixel anomaly detection: stages 4-5 use the native 256 and 512 channels to ensure compatibility of basic feature extraction, while stages 6-7 are adjusted to 768 and 1024 (instead of the native 2048), which avoids training instability caused by channel number jumps through gradient growth and reduces the computational redundancy brought by high channel number; at the same time, in response to the need for small target detection of pixel anomalies, stages 8-9 reduce the number of channels back to 512. The "increase-decrease" design of the number of channels focuses on key features and adapts to the real-time requirements of industrial inspection.

[0034] High-amplitude, high-frequency effects are eliminated by replacing the max-pooling layer with two 3×3 convolutional layers, and aliasing artifacts are filtered out by adding convolutional layers with different dilation rates at the end of the network. The dilated convolution is defined as follows:

[0035] in, The input feature map is the DDRN dilated convolution. Indicates the expansion rate The dilated convolution operation, is the dilated convolution kernel of DDRN, and p is the pixel coordinate in the output feature map. convolution kernel The set of pixels, where b is the convolution kernel. The pixel coordinates in the image. The dilation rate of DDRN is set to 1-4 based on the required fineness of the abnormal region boundaries. When d=1, it is a normal convolution; when d>1, it is a dilated convolution. This represents the pixel value at the dilated sampling position corresponding to the output pixel p in the input feature map.

[0036] This formula avoids mesh artifacts from traditional dilated convolution by adjusting the calculation logic of the dilation rate, effectively ensuring the segmentation accuracy of abnormal region boundaries.

[0037] After Res2Net50 feature extraction, a bidirectional pyramid pooling (BPPM) module is introduced to fuse multi-scale features. This module first performs global average pooling on the input feature map to obtain pooled feature maps of five different scales: 12×12, 6×6, 3×3, 2×2, and 1×1. , , , , Subsequently, bidirectional feature transfer is achieved through bidirectional pyramid pooling: along a top-down path, high-level semantic information is transferred to the lower and intermediate feature layers. The formula for calculation is:

[0038] Where i is the feature layer scale index of the top-down path. Input weights for the i-th layer (values ​​ranging from 0.3 to 0.7 are obtained through learning, based on the importance of the feature scale). Pass weights to the intermediate layer (values ​​and) Complementary, the sum of the two is 1). For upsampling operation, To avoid the denominator being zero, i is the minimum value of 0.0001. , This represents the intermediate feature map of the i-th layer in the top-down path. This is a 3×3 convolution operation. This is the original pooling feature map of the i-th layer output by the bidirectional pyramid pooling module. This is the intermediate feature map of the (i+1)th layer.

[0039] Following a bottom-up path, lower-level location information is passed to higher levels, outputting the feature layer. The formula for calculation is:

[0040] Where j is the feature layer scale index of the bottom-up path. This is the output feature map of the j-th layer in the bottom-up path. This is the original pooling feature map of the j-th layer output by the bidirectional pyramid pooling module. Input weights for the j-th layer, For intermediate layer weights, This is the intermediate feature map of the j-th layer output by the top-down path. Weights are passed down to lower layers, with values ​​ranging from 0.2 to 0.6. For downsampling operation, j , This is the output feature map of the (j-1)th layer.

[0041] It should be noted that this applies to the lowest level (12×12 scale). , For the highest level (1×1 scale).

[0042] in, Input a weight (value 0.5) for the highest layer. This is the original pooled feature map with a 1×1 scale. The weight is passed from the highest layer to the lowest layer (with a value of 0.5). The output feature map is a 2×2 scale.

[0043] The above calculations enhance the interaction of contextual information between feature layers of different scales through bidirectional transmission, thus solving the problem of easily missed detection of anomalies in small targets.

[0044] To further establish cross-space and cross-scale feature interactions, an Attention Pyramid Enhancement (APE) module is integrated after the output of the bidirectional pyramid pooling module. This module uses the feature maps from the five scales output by the bidirectional pyramid pooling module. , , , , (Corresponding to scales from 12×12 to 1×1) as input, feature enhancement is achieved through three mechanisms: self-attention (SA), downward attention (DA), and upward attention (UA). The self-attention mechanism captures spatial co-occurrence features of feature maps at the same scale, and inputs feature maps. After adjusting the size, the result is Through learnable matrices , , Generate query matrix Key matrix Value matrix After splitting into N feature blocks according to the channel dimension, the normalized attention map calculation formula is:

[0045] in, For the nth learnable aggregation weight (values ​​range from 0.1 to 0.3), This is the attention map output function for the SA mechanism. For normalization function, It is the inner product of the query matrix and the key matrix of the nth feature block.

[0046] Final SA output feature map ,in, The learnable weight matrix is ​​C×C' (C'=C, ensuring the number of channels remains unchanged). This is a dimension reshaping operation (restoring a one-dimensional feature to a two-dimensional feature map of C×H×W). It is the transpose of the value matrix.

[0047] The downward attention mechanism passes high-level semantic information to lower levels, and the high-level feature maps... After upsampling, the result is After adjusting the size, it is compared with the low-level feature map. The resized matrix is ​​split into N feature blocks (N=16, corresponding to 16 independent attention heads) according to the channel dimension. Each feature block is a one-dimensional vector, which is then used to generate the query matrix. Key matrix Value matrix The vector similarity is calculated independently for each attention head, and the attention graph is as follows:

[0048] in, The attention map output function of the DA mechanism. This is the nth learnable aggregation weight in the DA mechanism, with a value between 0.1 and 0.3. The negative Euclidean squared distance between the query matrix and the key matrix for the nth feature block is a scalar value; the smaller the distance, the higher the vector correlation and the larger the corresponding weight.

[0049] Output feature map ,in, Let be a C×C learnable weight matrix.

[0050] The upward attention mechanism integrates low-level positional information into high-level information, and the weights are obtained from the low-level feature map K through global average pooling. ( This is a global average pooling operation. The weight vector (of channel dimension, dimension C×1) is concatenated with the weighted concatenation of each channel of the high-level query feature map Q and then convolved to obtain the result. ,in, Let be the weight value of the i-th channel. Let Q be the feature map of the i-th channel. For channel splicing operations, the low-level feature map V is downsampled and then... Summation followed by 3×3 convolution for output ,in, This is a 3×3 convolution operation with a stride of 2, used to achieve downsampling. Output feature maps for the UA mechanism.

[0051] In the APE module After processing by SA and UA, the following results were obtained. , After processing with SA and DA, the following was obtained , and After processing by SA, DA, and UA respectively, we obtain , and The features are stacked along the channel dimension and then concatenated with the original input feature map. The number of channels is adjusted by a 1×1 convolution to output an enhanced feature map.

[0052] Finally, the enhanced feature map is fed into ICNet's Cascaded Feature Fusion (CFF) module, which upsamples the high-level coarse-grained features to the same size as the low-level fine-grained features. After refinement by 3×3 dilated convolution (dilation rate 2), it is then compared with the features obtained by 1... The low-level features with adjusted channel counts by convolution are summed to obtain the final fused feature map. The binary anomaly candidate region mask map is output by softmax classification. The region with a pixel value of 1 in the mask map is the pixel anomaly candidate region. The pixel coordinate range of each pixel anomaly candidate region is clearly marked (from the upper left corner coordinate (x1, y1) to the lower right corner coordinate (x2, y2)), which provides a precise region localization basis for subsequent classification feature extraction.

[0053] Step S3: Based on the candidate regions of pixel anomalies, perform feature extraction on the standardized image and brightness time series data to obtain pixel anomaly feature vectors; Based on the binarized anomaly candidate region mask image output in step S2, the pixel coordinate range marked in the mask image (from the top left corner coordinate (x1, y1) to the bottom right corner coordinate (x2, y2)) is used to analyze the normalized image and brightness temporal data. Precisely crop out local images of each anomaly candidate region and the corresponding local brightness time sequence , of which local images The resolution is adaptively adjusted based on the size of the candidate pixel anomaly region (minimum 3×3 pixels, maximum no more than 100×100 pixels, ensuring the targeting and efficiency of feature extraction), local brightness temporal sequence. The brightness of the candidate region center point changes over time (retain the 100 time points collected in step S1).

[0054] Based on the enhanced feature map output by the attention pyramid enhancement (APE) module integrated in step S2, the local image of the anomaly candidate region for each pixel is processed. and local brightness time sequence Enhanced extraction of classification-specific features is performed. The extracted features include four categories: spatial features, chromaticity features, temporal features, and luminance features, as well as spatial distribution index (SDI), luminance difference index (TWBDI), temporal stability index (TSI), and chromaticity deviation index (CDI). All features are weighted and enhanced through the attention mechanism of the APE module to highlight key information for distinguishing anomaly types.

[0055] In terms of spatial features, the focus is on capturing the connectivity and geometry of candidate regions for pixel anomalies: connectivity is calculated by computing local images. Number of connected regions with a pixel value of 1 The calculation formula is as follows:

[0056] in, Functions for labeling connected components. This is the connectivity threshold (a value of 1 represents 4-neighbor connectivity, adapted to the pixel arrangement characteristics of LCD panels). For counting functions, Represents isolated outliers (such as a single bad pixel). This represents a cluster of consecutive anomalies (such as a cluster of consecutive bad pixels).

[0057] Geometric morphology is determined by calculating the aspect ratio of the bounding rectangle of the abnormal region. and roundness To achieve this, the aspect ratio calculation formula is: ,in, The length of the bounding rectangle (taken as the maximum span of the candidate region's boundary coordinates). The width of the bounding rectangle. This represents an approximately circular anomaly (such as a pinhole defect). Represents bar-shaped or irregular anomalies (such as linear color shifts); the formula for calculating roundness is... ,in, The pixel area of ​​the abnormal region (i.e., the local image) (The total number of pixels with a value of 1) The perimeter of the abnormal region is calculated after the boundary is extracted using the Canny edge detection algorithm. The value range is (0, 1], and the closer it is to 1, the closer the area is to a circle.

[0058] Based on spatial characteristics, the Spatial Distribution Index (SDI) is further calculated using the following formula:

[0059] in, It is a spatial distribution characteristic index. This is the reference area for detecting pixel anomalies in the LCD panel. This serves as a reference perimeter for detecting anomalies in LCD panels. Reflecting the compactness of the area, Logarithmically scale the area of ​​the outlier region (to avoid excessively large areas causing exponential imbalance). This is the ratio of the abnormal area to the reference area. The value ranges from [0.5, 10]. The larger the value, the more complex the spatial distribution of the abnormal area (such as large areas of irregular and uneven brightness).

[0060] In terms of chromaticity features, by using the downward attention mechanism of the APE module in conjunction with color deviation information at different scales, the local image is first... Convert from RGB color space to ClE-Lab color space to obtain , , Three component images were generated, and then the deviation of each component image from the corresponding component image of the normal region at the same location (using a 10-pixel range of anomaly-free areas around the candidate region selected by reverse selection of the mask image as a reference) was calculated. , , The calculation formula is:

[0061] in, represent , , Any component in, To reference the component value corresponding to pixel p in the normal region, This represents the total number of pixels in the abnormal candidate region. This represents the average deviation value of the monochromaticity component.

[0062] Finally, the overall color deviation was calculated using the CIEDE2000 color difference formula. :

[0063] in, This refers to the overall deviation of the chromaticity components. Due to color deviation, and These are correction coefficients (all set to 1, to suit color evaluation scenarios for LCD panels). , and These are the weighting coefficients. To account for color deviation, The larger the value, the more severe the color cast, and it can be detected through... , Positive and negative judgment of color cast type (e.g.) and (Represents a red bias).

[0064] It should be noted that the formula for calculating the overall deviation of the chromaticity components is: The formula for calculating tone deviation is: ,in, The hue angle difference between the reference area and the abnormal area; the weighting coefficients are calculated as follows: , , ,in, and These are the average brightness and chromaticity values ​​of the reference area and the abnormal area, respectively. This is a color correction item.

[0065] Based on comprehensive color deviation The color deviation index (CDI) is calculated using the following formula: ,in, The maximum color difference in the training sample set (value 20, corresponding to severe color cast scenarios), The value range is [0.2, 8], with the lower limit of 0.2 determined based on the human visual perception threshold. Color shifts of less than 0.2 are considered minute deviations imperceptible to the human eye and are not included in the scope of pixel anomaly detection. A higher value indicates a more severe color cast, and this can be achieved through... , The positive or negative color cast type is determined.

[0066] In terms of temporal features, the upward attention mechanism of the APE module is used to associate local brightness temporal sequences. Extract stability indicators of brightness changes: First, calculate the standard deviation of the time series. The calculation formula is:

[0067] in, The length of the time series (value 100). The mean of the time series. The standard deviation of the time series. The smaller the value, the more stable the brightness (e.g., constant bad pixels); the larger the value, the more drastic the brightness fluctuation (e.g., flickering).

[0068] Secondly, the dominant frequency of the time series is extracted using Fourier transform. , Used to distinguish between periodic flickering (with a definite dominant frequency) and irregular fluctuations (without a definite dominant frequency).

[0069] Based on the above time series characteristics, the Time Stability Index (TSI) is calculated using the following formula:

[0070] in, This is the luminance variation coefficient (to eliminate the influence of luminance amplitude). This serves as a reference frequency for detecting pixel anomalies in LCD panels. The frequency penalty term (the higher the main frequency, the greater the penalty) has a TSI value range of [0.05, 3]. The smaller the value, the better the time stability.

[0071] Regarding brightness characteristics, the amplitude of the brightness difference between the abnormal region and the surrounding normal region is calculated by combining the cross-scale attention interaction of the APE module. The calculation formula is ,in, The average brightness value of the candidate anomaly region (from the local image) (obtained by calculating the grayscale image), The average brightness value is the average brightness value of the surrounding normal reference area (calculated by selecting a non-abnormal area within a 5-pixel range around the abnormal area). Used to distinguish between bright dead pixels, dark dead pixels, and color shift areas without brightness abnormalities.

[0072] based on Further calculation of the Tolerance for Brightness Difference Index (TWBDI) is performed first on local images. The grayscale image is subjected to three-level discrete wavelet decomposition. After each level of decomposition, multiple high-frequency sub-bands (horizontal, vertical, and diagonal directions) are obtained, where the first... Layer contains The high-frequency subband, for the first The u-th high-frequency subband of the layer decomposition ( =1, 2, 3 represent the number of decomposition layers), calculate their brightness differences. ,in, For the first The total number of pixels in the u-th high-frequency subband of layer . For the first In the u-th high-frequency subband of the layer The pixel value at the location, where m is the pixel index within that sub-band. For the first The mean of the u-th high-frequency subband of the layer; then calculate the mean of the th high-frequency subband. Layer brightness difference characteristic index ( For the first Number of high-frequency subbands per layer For the first (Layer high-frequency subband index); finally, summation yields the brightness difference characteristic index TWBDI, with a value range of [0.1, 5]. The larger the value, the more significant the brightness difference in the abnormal area.

[0073] The 15 feature parameters extracted above, including spatial features, chromaticity features, temporal features, and luminance features, are normalized using the min-max method to obtain the pixel anomaly feature vector. Finally, the pixel anomaly feature vector of each anomaly candidate region is output. This provides accurate and highly discriminative input data for the lightweight classifier in step S4.

[0074] Step S4: Input the pixel anomaly feature vector into the SE-ResNet classifier, output the anomaly type result, classify the pixel anomaly level based on the anomaly type result, and thus realize the pixel anomaly identification of the LCD panel.

[0075] The pixel anomaly feature vector of each anomaly candidate region output in step S3 is used as input to a lightweight classifier built on SF-ResNet18. This classifier strengthens the weights of key classification features through a compression-activation module while keeping the model lightweight to adapt to the needs of real-time industrial detection.

[0076] The SE-ResNet18 network structure consists of an input layer, four residual block groups (including SE modules), a global average pooling layer, and a fully connected classification layer. The input layer receives a 15-dimensional standardized feature vector. The feature dimension is mapped to 64 dimensions through 1×1 convolution (adapting to the residual block input); each residual block group contains several SE-ResNet basic modules. The core computation process of the SE module is divided into three steps: compression, activation, and recalibration. The compression operation compresses the feature map of each channel into a 1-dimensional feature vector through global average pooling. The calculation formula is:

[0077] in, The compression feature value of the c-th channel (c (corresponding to 64 channels) , Output the height and width of the feature map for the residual block. For the c-th channel feature map The pixel value of the location; The activation operation models the inter-channel correlation through two fully connected layers, and the calculation formula is as follows:

[0078] in, The incentive weight for the c-th channel is... and The learnable weight matrix for the fully connected layer (dimensions are 64×16 and 16×64 respectively, where 16 represents the number of intermediate channels; channel compression reduces computational cost). and For bias terms of fully connected layers, It is the ReLU activation function. This is the Sigmoid activation function.

[0079] The recalibration operation multiplies the excitation weights by the original channel features to obtain the enhanced feature map, calculated as follows: ,in, For the pixel value at position (i, j) of the c-th channel after recalibration, this process adaptively strengthens the key feature channels for distinguishing anomaly types (such as the chromaticity feature channel corresponding to color shift and the temporal feature channel corresponding to dynamic anomaly), and suppresses the interference of redundant feature channels.

[0080] The classifier training process is based on a dataset of 5000 LCD panel pixel anomaly samples (including defective pixel, color shift, dynamic anomaly, regional anomaly, and various subtype samples, with the sample ratio set to 3:3:2:2 according to the actual industrial defect distribution). The SGD optimizer is used to update network parameters, with a momentum of 0.9, weight decay of 0.0001, and an initial learning rate of 0.01. The learning rate is dynamically adjusted using a poly strategy (the learning rate decays linearly with the number of iterations). The batch size is 32, the training iterations are 200 epochs, and the cross-entropy loss function is used.

[0081] in, This represents the number of samples in the training batch. The number of classification categories (value 8, including 4 core anomalies and 4 subtypes). For the first The true label of the c-th class of a sample (one-hot encoded, 1 for the correct class, 0 for the rest), Let be the predicted probability of the k-th sample in class c. This represents the cross-entropy loss.

[0082] After training, the pixel anomaly feature vectors are input into the SE-ResNet classifier, and the output is the anomaly type result, specifically classified as follows: Ring-type dead pixels (including three subtypes: bright dead pixels, dark dead pixels, and fatal dead pixels) are classified by the amplitude of brightness difference. Distinguish between bright and dead pixels Dark defects correspond to and (That is, the average brightness of the abnormal area is less than 50% of the average brightness of the reference area), corresponding to fatal dead pixels. ); Color cast (including two subtypes: single-channel color cast and multi-channel mixed color cast, through...) , Distinguishing between positive and negative signs and their combinations, such as and It is reddish. and (It is a mixture of green and yellow). Dynamic anomaly type (including two subtypes: intermittent flickering and periodic bright / darkness, determined by the main frequency) Distinguish between periodic light and dark correspondences To define the value, intermittent flickering corresponds to (No obvious peak value); Regional anomaly class (including one subtype: continuous clusters of bad pixels and localized brightness unevenness, determined by the number of connected regions) or the standard deviation of the amplitude of local brightness difference distinguish).

[0083] After anomaly type classification is completed, pixel anomaly levels are determined based on the anomaly type results, as follows: Anomaly level assessment is performed based on the Spatial Distribution Index (SDI), Lightness Difference Index (TWBDI), Temporal Stability Index (TSI), and Color Deviation Index (CDI) extracted in step S3. First, the four indices are normalized using a min-max method to obtain the normalized index values. Subsequently, the weighting coefficients of each index are adjusted based on the results of the anomaly type. The weighting allocation formula is as follows:

[0084] in, The basic weights for each index are (0.3, 0.3, 0.2, and 0.2 for SDI, TWBDI, TSI, and CDI, respectively). The correlation coefficient between the anomaly type result and the index (e.g., the TWBDI for bad pixel type). =1.5, color cast corresponds to CDI. =1.5, the dynamic exception class corresponds to TSI's =1.5, the SDI corresponding to the regional anomaly class. =1.5, the rest of the indices =1.0), This represents the final weighting coefficient.

[0085] Based on the final weighting coefficient and the normalized exponent value The anomaly level score is obtained by performing a weighted summation. : ,in, Anomaly level scores are assigned (higher values ​​indicate more severe defects, reflecting the combined severity of spatial distribution complexity, brightness differences, temporal stability, and chromaticity deviation in the abnormal area). Based on these scores, pixel anomalies are categorized into three levels: minor anomalies correspond to... ≤0.3 (e.g., a single minor color cast, an isolated dark spot), moderate anomalies correspond to 0.3 < <0.6 (e.g., 3-5 consecutive clusters of bad pixels, moderate single-channel color cast), severe anomalies correspond to ≥0.6 (e.g., fatal bad pixels, large-area mixed color deviation, high-frequency flickering abnormalities).

[0086] Finally, the system outputs a combination of "abnormality type + pixel abnormality level" for each candidate region of anomaly, clearly indicating the specific category of the anomaly (such as "single-channel red cast - medium defect" and "fatal bright spot - severe defect") and the corresponding pixel coordinate range, providing accurate technical basis for the quality assessment and subsequent repair of LCD panels.

[0087] Furthermore, based on the above method embodiments, the present invention also provides a system including a memory, a processor, and a computer program stored in the memory, which is adapted to be loaded and executed by the processor to implement the above-described machine vision-based liquid crystal panel pixel anomaly recognition method.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, the phrase "comprising an element defined as..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for identifying pixel anomalies in a liquid crystal panel based on machine vision, characterized in that: Includes the following steps: Step S1: Acquire the visible light image and brightness signal of the LCD panel and perform preprocessing to obtain standardized image and brightness timing data; Step S2: Perform image segmentation on the standardized image using an improved image concatenated semantic segmentation algorithm to obtain candidate regions for pixel anomalies; Step S3: Based on the candidate regions of pixel anomalies, perform feature extraction on the standardized image and brightness time series data to obtain pixel anomaly feature vectors; Step S4: Input the pixel anomaly feature vector into the SE-ResNet classifier, output the anomaly type result, classify the pixel anomaly level based on the anomaly type result, and thus realize the pixel anomaly identification of the LCD panel.

2. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 1, characterized in that: The process of acquiring visible light images and brightness signals from the liquid crystal panel and preprocessing them to obtain standardized image and brightness time-series data includes the following specific steps: Visible light images of the LCD panel are captured by an industrial camera; the brightness signal of the LCD panel is collected by a photometer to obtain a brightness time sequence; and the trigger synchronization between the industrial camera and the photometer is achieved by a synchronous trigger signal generator. The acquired visible light image is filtered using a Butterworth low-pass filter to obtain a filtered image; the filtered image is then calibrated for color and brightness using an adaptive histogram matching algorithm to obtain a standardized image. The acquired luminance time series was denoised by Gaussian filtering to obtain luminance time series data.

3. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 2, characterized in that: The method of segmenting a standardized image using an improved image concatenated semantic segmentation algorithm to obtain candidate regions for pixel anomalies is as follows: The improved image cascaded semantic segmentation algorithm uses Res2Net50 as the backbone feature extraction network and adopts an artifact removal and dilated residual network. After Res2Net50 feature extraction, it obtains abnormal candidate regions through a bidirectional pyramid pooling module, an attention pyramid enhancement module, and a cascaded feature fusion module. The specific calculation of the artifact removal and holed residual network is as follows: ； in, The input feature map is the DDRN dilated convolution. Indicates the expansion rate The dilated convolution operation, is the dilated convolution kernel of DDRN, and p is the pixel coordinate in the output feature map. convolution kernel The set of pixels, where b is the convolution kernel. The pixel coordinates in the image. For the expansion rate of DDRN, This represents the pixel value at the dilated sampling position corresponding to the output pixel p in the input feature map.

4. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 3, characterized in that: The bidirectional pyramid pooling module is specifically as follows: The bidirectional pyramid pooling module uses bidirectional propagation, following a top-down path, with intermediate feature layers. The formula for calculation is: ； Where i is the feature layer scale index of the top-down path. Input weights for the i-th layer, Pass weights to the intermediate layer. For upsampling operation, The minimum value is 0.0001 to avoid the denominator being zero. This represents the intermediate feature map of the i-th layer in the top-down path. This is a 3×3 convolution operation. This is the original pooling feature map of the i-th layer output by the bidirectional pyramid pooling module. This is the intermediate feature map of the (i+1)th layer; Output feature layer along bottom-up path The formula for calculation is: ； Where j is the feature layer scale index of the bottom-up path. This is the output feature map of the j-th layer in the bottom-up path. This is the original pooling feature map of the j-th layer output by the bidirectional pyramid pooling module. Input weights for the j-th layer, For intermediate layer weights, This is the intermediate feature map of the j-th layer output by the top-down path. To pass weights to lower layers, For downsampling operation, This is the output feature map of the (j-1)th layer.

5. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 4, characterized in that: The attention pyramid enhancement module is as follows: The attention pyramid enhancement module employs three mechanisms: self-attention, downward attention, and upward attention. The downward attention mechanism calculates vector similarity independently for each attention head, and the attention graph is as follows: ； in, For the attention graph output function of the downward attention mechanism, For the nth learnable aggregation weight of the downward attention mechanism, Find the squared negative Euclidean distance between the query matrix and the key matrix for the nth feature block.

6. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 5, characterized in that: The step of extracting features from standardized images and brightness time-series data based on pixel anomaly candidate regions to obtain pixel anomaly feature vectors includes the following steps: Based on pixel anomaly candidate regions, local images of each pixel anomaly candidate region are cropped from normalized images and brightness temporal data. and local brightness time sequence ; Local image of each pixel anomaly candidate region and local brightness time sequence Feature extraction is performed, and the extracted features include spatial features, chromaticity features, temporal features, and luminance features, as well as spatial distribution index, luminance difference index, temporal stability index, and chromaticity deviation index. The extracted feature rows are normalized to obtain pixel anomaly feature vectors. .

7. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 6, characterized in that: The specific chromaticity features are as follows: Calculate the overall colorimetric deviation : ； in, This refers to the overall deviation of the chromaticity components. Due to color deviation, and For correction factors, , and These are the weighting coefficients. This is to account for overall color deviation.

8. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 7, characterized in that: The process of inputting pixel anomaly feature vectors into the SE-ResNet classifier and outputting anomaly type results includes the following specific steps: The SE-ResNet classifier uses the SE-ResNet18 network, whose structure consists of an input layer, four residual block groups, a global average pooling layer, and a fully connected classification layer. Each residual block group contains several SE-ResNet basic modules. The core computations of SE-ResNet include compression, excitation, and recalibration. The SE-ResNet classifier is trained. After training, the pixel anomaly feature vector is input into the SE-ResNet classifier, and the anomaly type result is output.

9. The method for identifying pixel anomalies in a liquid crystal panel based on machine vision according to claim 8, characterized in that: The method of classifying pixel anomaly levels based on anomaly type results is as follows: The spatial distribution index, luminance difference index, temporal stability index, and chromaticity deviation index are normalized to obtain normalized index values. ; The weighting coefficients of each index are adjusted based on the anomaly type results. The weighting allocation formula is as follows: ； in, The basic weights for each index, This represents the correlation coefficient between the anomaly type results and the index. These are the final weighting coefficients; Based on the final weighting coefficient and the normalized exponent value A weighted sum is performed to obtain the anomaly level score, and the pixel anomaly level is classified according to the anomaly level score.

10. A method and system for identifying pixel anomalies in a liquid crystal panel based on machine vision, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-9.

Images