Polarizer defect detection method and system based on deep learning

Abstract

The application provides a kind of based on deep learning's polaroid defect detection method and system, by obtaining the initial image data set corresponding to the polaroid to be detected, by image pre-processing and feature extraction operation from the initial image data set extraction defect candidate area feature in the polaroid surface image and the texture distribution feature of the polaroid surface;The defect candidate area feature and the texture distribution feature are input into the pre-trained defect recognition deep learning model to carry out joint defect recognition operation, generate the defect preliminary identification result of the polaroid surface image;Based on the defect preliminary identification result, determine the defect type of the defect existing in the polaroid to be detected and the spatial distribution feature information of the defect on the polaroid surface.The application can realize the effective convergence of defect detection result and production quality control process, improve the practicability of polaroid defect detection and the guiding value to production process.

Description

Technical Field

[0001] This invention relates to the field of deep learning, and more specifically, to a method and system for detecting defects in polarizers based on deep learning. Background Technology

[0002] With the development of display panel manufacturing technology, polarizers, as a key component, directly affect display performance due to their surface quality. Polarizer defect detection technology has emerged to address this, identifying various defects on the polarizer surface to ensure product quality—a crucial step in the display panel production process. Current technologies involve acquiring surface images of the polarizer, preprocessing the images to extract geometric or grayscale features of defects, and then using traditional machine learning classifiers to identify and judge the defects. However, existing technologies often fail to effectively capture some directional defects due to light reflection angles, making it difficult to comprehensively characterize the multi-dimensional characteristics of complex defects. Furthermore, the exploration of defect location information and distribution patterns is insufficient, resulting in limited guidance value for optimizing subsequent production processes. Summary of the Invention

[0003] This invention provides a method and system for detecting defects in polarizers based on deep learning.

[0004] In a first aspect, embodiments of the present invention provide a method for detecting defects in polarizers based on deep learning, the method comprising:

[0005] Acquire an initial image data set corresponding to the polarizer to be detected. The initial image data set contains multiple polarizer surface images under different acquisition angles.

[0006] Defect candidate region features and texture distribution features of the polarizer surface are extracted from the initial image dataset through image preprocessing and feature extraction operations.

[0007] The defect candidate region features and the texture distribution features are input into a pre-trained deep learning model for defect recognition to perform a joint defect recognition operation, thereby generating a preliminary defect recognition result for the polarizer surface image.

[0008] Based on the preliminary defect identification results, the defect type and spatial distribution characteristics of the defects on the surface of the polarizer to be tested are determined.

[0009] Secondly, embodiments of the present invention provide a defect detection system, comprising:

[0010] A memory, wherein a computer program is stored;

[0011] A processor is used to load the computer program to implement the deep learning-based polarizer defect detection method described above.

[0012] The present invention provides a deep learning-based method for detecting defects in polarizers. It acquires an initial image dataset containing multiple surface images of the polarizer at different acquisition angles. Through image preprocessing and feature extraction, it extracts candidate defect region features and texture distribution features from the polarizer surface images. These two features are then input into a pre-trained deep learning model for defect recognition, where a joint defect recognition operation is performed to generate a preliminary defect recognition result. Based on this result, it determines the defect type and spatial distribution characteristics of the defects on the polarizer surface. Finally, it generates a polarizer quality inspection report containing defect location coordinates based on the defect type and spatial distribution characteristics, and sends it to the target quality control system. By acquiring polarizer surface images from different acquisition angles, the optical performance of defects under different lighting conditions can be captured, avoiding the problem of missing directional defects. By simultaneously extracting defect candidate region features and texture distribution features, the local morphological features of defects and the overall texture variation features of the polarizer surface can be captured collaboratively, comprehensively reflecting the multi-dimensional performance of defects. By inputting the two features into a deep learning model for joint defect recognition, the nonlinear transformation and correlation modeling capabilities of the deep learning model for high-dimensional features can be utilized to explore the intrinsic correlation between defect candidate region features and texture distribution features, improving the accuracy of identifying complex defects in polarizers. By determining the spatial distribution feature information of defects, data support for the location and relative relationships of defects can be provided for the quality control of polarizer production. By generating a quality inspection report containing defect location coordinates and sending it to the quality control system, the defect detection results can be effectively linked with the production quality control process, improving the practicality of polarizer defect detection and its guiding value for the production process. Attached Figure Description

[0013] Figure 1 This is a flowchart of a polarizer defect detection method based on deep learning provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the composition of a defect detection system provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 1 , Figure 1 The flowchart illustrates a deep learning-based polarizer defect detection method provided in this embodiment of the invention. This deep learning-based polarizer defect detection method can be executed by a defect detection system and may include the following steps:

[0017] Step S100: Obtain the initial image data set corresponding to the polarizer to be detected. The initial image data set contains multiple polarizer surface images under different acquisition angles.

[0018] The initial image dataset serves as the foundational data source for subsequent polarizer defect detection, consisting of multiple polarizer surface images acquired from different acquisition angles. Setting different acquisition angles allows for comprehensive capture of information about the polarizer surface. For example, multiple cameras can be used to photograph the polarizer from different directions and angles. These cameras can be evenly distributed around the polarizer, with each camera's angle differing from the angle of its adjacent cameras by a set number of degrees, such as 30 degrees or 45 degrees.

[0019] Step S200: Extract the defect candidate region features and texture distribution features of the polarizer surface from the initial image dataset through image preprocessing and feature extraction operations.

[0020] Image preprocessing involves altering the original image to improve its quality and facilitate subsequent feature extraction. Feature extraction extracts features from the preprocessed image that represent candidate defect regions and texture distribution. Defect candidate region features refer to the characteristics of areas that may contain defects, such as area, perimeter, and shape. Texture distribution features describe the surface texture of the polarizer, such as energy, entropy, and contrast. By extracting these features, a more accurate basis can be provided for subsequent defect identification.

[0021] As one implementation method, step S200, which involves extracting defect candidate region features and texture distribution features of the polarizer surface image from the initial image dataset through image preprocessing and feature extraction operations, may specifically include the following steps S210~S270:

[0022] Step S210: Adjust the grayscale distribution range of each polarizer surface image in the initial image dataset using an adaptive histogram equalization algorithm to generate illumination equalized image data.

[0023] Adaptive histogram equalization (HQE) algorithms can adjust the grayscale distribution of an image based on local region information. In polarizer surface images, varying lighting conditions can lead to some areas being too bright and others too dark, affecting defect identification. HQE can adjust the grayscale distribution range, resulting in a more uniform overall brightness. Specifically, each polarizer surface image in the initial image dataset is divided into multiple small local regions. Histogram equalization is performed on each local region separately, and then the processed local regions are merged to obtain the equalized image data. This enhances local contrast without altering the overall image information, making defects more clearly visible.

[0024] Step S220: Based on the combined use of Gaussian filtering and median filtering, random noise and salt-and-pepper noise in the illumination equalization image data are eliminated to generate denoised image data.

[0025] Gaussian filtering achieves filtering by weighting each pixel in an image and its neighboring pixels, with the weights determined by a Gaussian function. Gaussian filtering effectively eliminates Gaussian noise, i.e., random noise, in images. Median filtering replaces the gray value of each pixel in an image with the median of the gray values ​​of its neighboring pixels. For salt-and-pepper noise, i.e., isolated bright or dark spots in an image, median filtering in this embodiment uses Gaussian filtering and median filtering in combination. First, Gaussian filtering is applied to the illumination-equalized image data to remove random noise, and then median filtering is applied to remove salt-and-pepper noise, ultimately generating denoised image data. This collaborative filtering approach can more effectively improve image quality and reduce the impact of noise on subsequent feature extraction and defect identification.

[0026] Step S230: Enhance the gray-level variation areas in the denoised image data using the Laplacian operator to highlight the edge contour information of the defective areas and generate edge-enhanced image data.

[0027] The Laplacian operator is a second-order differential operator that is highly sensitive to grayscale changes in images and can be used to detect edge information. In denoised image data, defect areas often exhibit significant grayscale variations. The Laplacian operator can enhance these grayscale variation areas, thereby highlighting the edge contour information of the defect areas. Specifically, the Laplacian operator is applied to the denoised image data, calculating the second derivative of each pixel to obtain a new image. Areas with large grayscale variations are enhanced, while areas with small grayscale variations are suppressed. This generates edge-enhanced image data, making the edges of defect areas clearer and facilitating subsequent defect candidate region extraction.

[0028] As one implementation method, step S230, which strengthens the gray-level variation areas in the denoised image data by using the Laplacian operator to highlight the edge contour information of the defective areas and generate edge-enhanced image data, may specifically include the following steps S231~S237:

[0029] Step S231: Convert the denoised image data into a single-channel grayscale image, retain the brightness information in the image, and generate grayscale image data.

[0030] Denoising the image data may result in a multi-channel color image. However, a single-channel grayscale image is easier to process during edge detection. Converting the denoised image data to a single-channel grayscale image involves weighted averaging of the color information of each pixel in the multi-channel image to obtain a single grayscale value, thus converting the image into a grayscale image containing only luminance information. This process preserves luminance information while removing color information, making the image simpler and facilitating subsequent edge detection operations. For example, if the denoised image data is an RGB three-channel color image, the formula: Grayscale value = 0.299 × R + 0.587 × G + 0.114 × B, can be used to convert the RGB value of each pixel into a single grayscale value, generating grayscale image data.

[0031] Step S232: Calculate the gradient values ​​of pixels in the grayscale image data in the horizontal and vertical directions based on the Sobel operator to obtain the horizontal gradient image and the vertical gradient image.

[0032] The Sobel operator is an edge detection operator that calculates the gradient values ​​of pixels in an image by performing a convolution operation. The gradient value reflects the degree of grayscale change in the image; a larger gradient value indicates a more significant grayscale change in the region where the pixel is located, potentially indicating an edge in the image. In this embodiment of the invention, both horizontal and vertical Sobel operators are used to perform convolution operations on the grayscale image data. The horizontal Sobel operator detects horizontal edges in the image, while the vertical Sobel operator detects vertical edges. Through these two convolution operations, horizontal and vertical gradient images are obtained, respectively reflecting the grayscale changes in the image data in the horizontal and vertical directions.

[0033] Step S233: Perform square root operations on the horizontal gradient image and the vertical gradient image, and merge them to obtain a gradient magnitude image representing the edge intensity.

[0034] After obtaining the horizontal and vertical gradient images, they are merged to comprehensively consider grayscale changes in both directions. For example, the gradient values ​​of each pixel in both images are squared and taken as a square root. Specifically, for pixels at the same location in both images, their gradient values ​​are squared separately, summed, and then the square root is taken to obtain the gradient magnitude of that pixel. By performing this operation on all pixels, a gradient magnitude image representing edge intensity is obtained. The grayscale value of each pixel in the gradient magnitude image represents the edge intensity at that location; a larger grayscale value indicates a more pronounced edge.

[0035] Step S234: Traverse each pixel in the gradient magnitude image, compare the gradient magnitude of the pixel with the gradient magnitude of its neighboring pixels in the gradient direction, retain the pixels with the local maximum value, and obtain the thinned edge image.

[0036] In gradient magnitude images, edges may be relatively wide. To obtain more accurate edge information, the edges need to be thinned. For example, each pixel in the gradient magnitude image is iterated over. For each pixel, its gradient magnitude is compared with that of its neighboring pixels along the gradient direction. If the gradient magnitude of the pixel is the largest among its neighboring pixels along the gradient direction, the pixel is retained; otherwise, its grayscale value is set to 0. Through this operation, only the pixels with local maximum values ​​on the edges are retained, and redundant pixels around the edges are removed, resulting in a thinned edge image. The edges in the thinned edge image are finer and more accurate, facilitating edge connection and defect region extraction.

[0037] Step S235: Set a high threshold and a low threshold to perform dual threshold processing on the thinned edge image. Mark pixels with gradient magnitude greater than the high threshold as strong edges, mark pixels with gradient magnitude between the low threshold and the high threshold and connected to strong edges as weak edges, and mark pixels with gradient magnitude less than the low threshold as non-edges.

[0038] Dual thresholding is a method for further filtering edge pixels. By setting a high threshold and a low threshold (the settings of which can be adjusted according to actual conditions, and this invention does not limit this), pixels in the refined edge image can be divided into three categories: strong edges, weak edges, and non-edges. Pixels with a gradient magnitude greater than the high threshold are considered true edge pixels and marked as strong edges. Pixels with gradient magnitudes between the low and high thresholds are considered edge pixels, but may also be noise points, requiring further judgment. If these pixels are connected to strong edges, they are marked as weak edges, as they may be extensions of strong edges. Pixels with gradient magnitudes less than the low threshold are marked as non-edges, as they may be noise points or regions that are not edges. Dual thresholding effectively removes noise points while preserving true edge information.

[0039] Step S236: Determine whether the pixels marked as weak edges have a connected path with the strong edges, retain the weak edge pixels connected to the strong edges, delete the isolated weak edge pixels, and obtain the complete edge image.

[0040] After double thresholding, pixels marked as weak edges are further evaluated to determine if they truly belong to an edge. For example, it's checked whether these weak edge pixels have a connected path to a strong edge. If a connected path exists, these weak edge pixels are considered part of a strong edge and are retained; if no connected path exists, these weak edge pixels are likely isolated noise points and are deleted. Through these operations, isolated weak edge pixels can be removed, resulting in a complete edge image. The edges in the complete edge image are more continuous and accurate.

[0041] Step S237: Overlay the complete edge image with the grayscale image data, adjust the grayscale value of the edge pixels to the preset highlight value, and generate edge enhancement image data that highlights the edge contour of the defect area.

[0042] To more clearly highlight the edge contours of the defective area, a complete edge image is overlaid with grayscale image data. During the overlay process, the grayscale values ​​of edge pixels in the complete edge image are adjusted to a preset highlight value, such as 255 (assuming the image's grayscale value range is 0-255). Thus, in the overlaid image, edge pixels appear brighter, while other non-edge pixels retain their original grayscale values ​​from the grayscale image data. In this way, edge-enhanced image data is generated that highlights the edge contours of the defective area, making the edges of the defective area more distinct.

[0043] Step S240: Starting with a preset seed point, merge adjacent pixels in the edge enhancement image data whose grayscale value difference is within a preset range into the same region to obtain multiple defect candidate region units.

[0044] Pre-defined seed points are predetermined starting points for region growing, typically located within areas that may contain defects. Region growing is an image segmentation method based on pixel grayscale similarity, merging adjacent pixels with similar grayscale values ​​into the same region. In this embodiment, starting with the pre-defined seed points, neighboring pixels in the edge-enhanced image data are searched, and the grayscale difference between these neighboring pixels and the seed points is calculated. If the grayscale difference is within a preset range, these neighboring pixels are included in the current growing region, and the process continues, starting with these newly included pixels, to find their neighboring pixels and repeating the above process until no new pixels meet the grayscale difference condition. Through this region growing process, the edge-enhanced image data is segmented into multiple regions, which are defect candidate region units. Each defect candidate region unit may contain one or more defects, or it may contain some normal regions, but they all have similar grayscale characteristics.

[0045] As one implementation method, step S240, starting from a preset seed point, merges adjacent pixels in the edge enhancement image data whose grayscale value differences are within a preset range into the same region to obtain multiple defect candidate region units. Specifically, this may include the following steps S241~S247:

[0046] Step S241: Set a grayscale threshold to convert the edge enhancement image data into a black and white binary image, where the pixel value of the edge contour region is 1 and the pixel value of the background region is 0, thus generating a binary edge image.

[0047] Binarization is the process of dividing the pixel values ​​in an image into two values. For example, pixels with values ​​greater than a certain threshold are assigned one value, and pixels with values ​​less than that threshold are assigned another value. In this embodiment of the invention, a grayscale threshold is set to classify the pixels in the edge enhancement image data. The pixel values ​​of pixels in the edge contour region are set to 1, and the pixel values ​​of pixels in the background region are set to 0, thereby converting the edge enhancement image data into a black and white binary image, i.e., a binarized edge image. Through binarization, the image representation can be simplified, facilitating morphological operations and contour extraction.

[0048] Step S242: Perform morphological closing operation of dilation and erosion on the binarized edge image based on the structuring element of the preset size, fill the small holes in the edge contour, connect the broken edge segments, and generate a closed edge image.

[0049] Morphological closing operations consist of two operations: dilation and erosion. Dilation expands the boundaries of objects in an image outward, while erosion contracts them inward. In this embodiment, a structuring element of a preset size is used to dilate the binarized edge image, expanding the edge contour region outward and filling small holes within the edge contour. Then, an erosion operation is performed on the dilated image, contracting the object boundaries inward and removing excess portions generated during dilation. Through these dilation and erosion operations, small holes within the edge contour are filled, broken edge segments are connected, and a closed edge image is generated. The edges in the closed edge image are more continuous and complete, facilitating contour tracking and region growing.

[0050] Step S243: Extract the set of pixel coordinates of each closed contour in the closed edge image using the contour tracking algorithm, determine the minimum bounding rectangle of each closed contour, and use the center pixel of the minimum bounding rectangle as the seed point for region growing.

[0051] Contour tracing algorithms are used to extract closed contours in an image. They trace pixels along the edges of the closed contours to obtain a set of pixel coordinates. After obtaining the closed edge image, the contour tracing algorithm extracts the pixel coordinate set for each closed contour. Then, for each closed contour, its minimum bounding rectangle is calculated—the smallest rectangle that completely contains the closed contour. The center pixel of the minimum bounding rectangle is located at the center of the closed contour and is used as the seed point for region growing. This ensures that region growing starts from the center of the closed contour, facilitating the merging of pixels within the entire closed contour into a single region.

[0052] Step S244: Sample the gray value of the pixel corresponding to each seed point in the denoised image data as the initial gray value reference for region growing.

[0053] After determining the seed points for region growing, an initial grayscale reference value is assigned to each seed point for region growing decisions. For example, the pixel corresponding to each seed point is found in the denoised image data, and its grayscale value is sampled and used as the initial grayscale reference value for region growing. Thus, during the region growing process, this initial grayscale reference value can be used to determine whether adjacent pixels should be included in the current growing region.

[0054] Step S245: Centered on the seed point, traverse the adjacent pixels using the eight-neighbor search method, calculate the absolute difference between the gray value of the adjacent pixel and the initial gray value reference value, and when the absolute difference is less than the preset gray value difference threshold, include the adjacent pixel in the current growth area and update the gray value reference value of the area to the average gray value of all pixels in the area.

[0055] The eight-neighbor search method refers to searching for eight neighboring pixels centered on a given pixel in an image. In this embodiment of the invention, the eight-neighbor search method is used to traverse the neighboring pixels centered on the seed point. For each neighboring pixel, the absolute difference between its grayscale value and the initial grayscale reference value is calculated. If this absolute difference is less than a preset grayscale difference threshold, it indicates that the neighboring pixel's grayscale value is similar to that of the current growth region, and it is included in the current growth region. After including a new pixel, the grayscale reference value of the updated region is the average grayscale value of all pixels within the region. Thus, as the region grows, the grayscale reference value is continuously updated to reflect the overall grayscale characteristics of the pixels within the region.

[0056] Step S246: Repeat the neighborhood search and region growth steps until no new pixels meet the grayscale difference condition, stop the growth process of the current region, and obtain a complete region unit.

[0057] After completing the neighborhood search and region growing for a single pixel, the process continues, centering on the newly included pixel, repeating the neighborhood search and region growing steps. Neighboring pixels are continuously searched, and their grayscale difference conditions are checked. Pixels that meet the conditions are included in the current growing region, and the grayscale reference value is updated. This process continues until no new pixels meet the grayscale difference conditions. At this point, the growing process for the current region stops, resulting in a complete region unit. Each region unit may contain one or more defects, or it may contain some normal regions, but they all share similar grayscale characteristics.

[0058] Step S247: Delete the region units with an area smaller than the preset minimum area threshold, and retain the region units with an area greater than or equal to the preset minimum area threshold as candidate defect region units.

[0059] After obtaining multiple region elements, these elements are filtered to remove those with excessively small areas. Region elements with excessively small areas may be noise points or unimportant regions, offering no practical significance for defect detection. For example, a preset minimum area threshold can be set, and the area of ​​each region element is calculated. If the area is less than the preset minimum area threshold, the region element is deleted; if the area is greater than or equal to the preset minimum area threshold, the region element is retained and considered a candidate defect region element. This filtering process reduces unnecessary computation and improves the efficiency of defect detection.

[0060] Step S250: Extract the area ratio, perimeter-to-area ratio, and aspect ratio of the circumscribed rectangle for each defect candidate region unit, and combine them to obtain the defect candidate region features.

[0061] The area ratio of a defect candidate region is the proportion of its area within the entire image area, reflecting the relationship between the size of the defect candidate region and the overall image size. The perimeter-to-area ratio is the ratio of the perimeter of the defect candidate region's outline to its area, reflecting the shape complexity of the defect candidate region. The aspect ratio of the bounding rectangle is the ratio of the length to the width of the smallest bounding rectangle of the defect candidate region, reflecting its shape characteristics. By extracting these features from each defect candidate region and combining them, the defect candidate region features can be obtained.

[0062] Step S260: Calculate the texture energy, entropy value and contrast parameters of the denoised image data under different directions and distances using the gray-level co-occurrence matrix, and combine them to obtain the texture distribution features.

[0063] The gray-level co-occurrence matrix (GLCM) is a matrix used to describe the texture features of an image, reflecting the spatial distribution of gray values ​​in the image. In this embodiment of the invention, the texture energy, entropy value, and contrast parameter of the denoised image data at different directions and distances are calculated using the GLCM. Texture energy reflects the uniformity of the texture in the image; the higher the energy value, the more uniform the texture. Entropy value reflects the complexity of the texture in the image; the higher the entropy value, the more complex the texture. The contrast parameter reflects the contrast of the texture in the image; the higher the contrast, the clearer the texture. By calculating these parameters and combining them, the texture distribution characteristics can be obtained.

[0064] Step S270: Adjust the dimensions of the defect candidate region features and texture distribution features to the same dimension through linear mapping to generate a joint feature set with a unified dimensional representation.

[0065] After obtaining the defect candidate region features and texture distribution features, since these two features may have different dimensions, their dimensions need to be adjusted to be the same for easier subsequent processing and analysis. For example, this can be achieved through linear mapping. Linear mapping maps vectors from one vector space to another. By adjusting the dimensions of the defect candidate region features and texture distribution features to the same dimension through linear mapping, a joint feature set with a unified dimensional representation is generated. This joint feature set contains information from both the defect candidate region features and the texture distribution features.

[0066] Step S300: Input the defect candidate region features and texture distribution features into the pre-trained defect recognition deep learning model to perform joint defect recognition operation and generate preliminary defect recognition results of the polarizer surface image.

[0067] A pre-trained deep learning model for defect recognition is a trained neural network model capable of identifying defects based on input feature information. In this embodiment of the invention, defect candidate region features and texture distribution features are input into the pre-trained deep learning model for defect recognition. The model analyzes and processes these features, and through joint defect recognition operations, generates preliminary defect recognition results for the polarizer surface image. The preliminary defect recognition results include information such as the type and location of possible defects in the polarizer surface image. By using a pre-trained deep learning model, the accuracy and efficiency of defect recognition can be improved.

[0068] As one implementation method, step S300 involves inputting the defect candidate region features and texture distribution features into a pre-trained deep learning model for defect recognition to perform a joint defect recognition operation, generating preliminary defect recognition results for the polarizer surface image. Specifically, this may include the following steps S310-S370:

[0069] Step S310: Concatenate the defect candidate region features and texture distribution features along the channel dimension to generate an initial fused feature vector.

[0070] Channel dimension refers to the different channels in a feature vector, each channel representing different feature information. In this embodiment of the invention, the defect candidate region features and texture distribution features are concatenated along the channel dimension, that is, these two feature vectors are connected in the channel direction to generate a new feature vector, namely the initial fused feature vector. The initial fused feature vector contains information from the defect candidate region features and texture distribution features, and these two features can be integrated through the concatenation operation.

[0071] Step S320: Adjust the mean and variance of each channel in the initial fusion feature vector to a preset range, eliminate the dimensional differences between different feature channels, and generate a normalized fusion feature vector.

[0072] Different feature channels may exhibit dimensional differences, meaning the range and scale of feature values ​​may differ between channels. These dimensional differences can impact the training and performance of deep learning models. In this embodiment of the invention, the mean and variance of each channel in the initial fused feature vector are adjusted to a preset range, and a normalization operation is used to eliminate the dimensional differences between different feature channels. For example, for each channel in the initial fused feature vector, its mean and variance are calculated. Then, the mean is subtracted from each feature value of that channel, and the result is divided by the standard deviation, converting it to a standard normal distribution with a mean of 0 and a variance of 1. This normalization operation generates a normalized fused feature vector, ensuring that different feature channels have the same scale and range.

[0073] Step S330: Input the normalized fused feature vector into the residual network module of the defect recognition deep learning model. Perform nonlinear transformation and dimensionality enhancement on the feature vector through multiple residual blocks. Each residual block contains two convolutional layers and one skip connection. The input features are directly superimposed on the output features of the convolutional layers through the skip connection to generate a deep feature representation vector.

[0074] The residual network module is a network structure in a deep learning model, composed of multiple residual blocks. Each residual block is the basic building block of the residual network, containing two convolutional layers and one skip connection. Convolutional layers are neural network layers used to extract features; they perform convolution operations on the input feature vector to extract its feature information. Skip connections directly superimpose the input features onto the output features of the convolutional layer, solving the gradient vanishing problem in deep learning models and allowing for the training of deeper networks. In this embodiment, a normalized fused feature vector is input into the residual network module of a defect recognition deep learning model. Multiple residual blocks perform nonlinear transformations and dimensionality enhancement on the feature vector. Each residual block performs a convolution operation on the input feature vector to extract its feature information and then directly superimposes the input features onto the output features of the convolutional layer through skip connections, generating a deep feature representation vector. This deep feature representation vector contains higher-level, more abstract feature information.

[0075] As one implementation method, step S330 involves inputting the normalized fused feature vector into the residual network module of the defect recognition deep learning model. Multiple residual blocks are used to perform nonlinear transformation and dimensionality enhancement on the feature vector. Each residual block contains two convolutional layers and one skip connection. The skip connection directly superimposes the input features onto the output features of the convolutional layers to generate a deep feature representation vector. Specifically, this may include the following steps S331-S337:

[0076] Step S331: Input the normalized fused feature vector into the first convolutional layer of the first residual block, perform convolution operation on the feature vector based on a preset number of convolution kernels, extract local feature information, and generate the first convolutional feature.

[0077] Convolutional layers extract local feature information by performing convolution operations between convolution kernels and the input feature vector. In this embodiment of the invention, the normalized and fused feature vector is input into the first convolutional layer of the first residual block, and a preset number of convolution kernels are used to perform convolution operations on the feature vector. Each convolution kernel can extract different feature information from the input feature vector. Through the convolution operation of multiple convolution kernels, multiple local feature information can be extracted. Through such convolution operations, a first convolutional feature is generated, which contains the local feature information in the normalized and fused feature vector.

[0078] Step S332: Process the first convolutional feature using the ReLU activation function to generate the first activated feature.

[0079] The ReLU activation function is a non-linear activation function, expressed as f(x) = max(0, x). In this embodiment of the invention, the ReLU activation function is applied to each element in the first convolutional feature, setting elements less than 0 to 0 and leaving elements greater than 0 unchanged. This process introduces a non-linear factor, allowing the model to learn more complex feature representations. The generated first activation feature contains local feature information after non-linear transformation, providing richer feature representations for the convolution operation.

[0080] Step S333: Input the first activation feature into the second convolutional layer of the first residual block, perform convolution operation on the feature based on the same number of convolutional kernels as the first convolutional layer, further extract deep local features, and generate the second convolutional feature.

[0081] After obtaining the first activation feature, it is input into the second convolutional layer of the first residual block. The same number of convolutional kernels as the first convolutional layer are used to perform convolution operations on the first activation feature to further extract deeper local features. Through the convolution operation of the second convolutional layer, the local feature information in the first activation feature is further extracted and abstracted, generating the second convolutional feature. The second convolutional feature contains higher-level, more abstract local feature information.

[0082] Step S334: Adjust the mean and variance of the second convolutional feature to generate the second normalized feature.

[0083] To eliminate the dimensional differences between different channels in the second convolutional feature and improve the training efficiency and performance of the model, the second convolutional feature needs to be normalized. For example, the mean and variance of each channel in the second convolutional feature can be adjusted to convert it into a standard normal distribution with a mean of 0 and a variance of 1. Through this normalization operation, a second normalized feature is generated, ensuring that different channels have the same scale and range.

[0084] Step S335: The normalized fused feature vector is superimposed onto the second normalized feature through a skip connection to achieve residual learning between the input feature and the convolutional processed feature, generating the first residual feature.

[0085] Skip connections allow input features to be directly superimposed onto the output features of a convolutional layer. In this embodiment, the normalized fused feature vector is superimposed onto a second normalized feature via a skip connection. Specifically, the elements at corresponding positions of the normalized fused feature vector and the second normalized feature are added to obtain the first residual feature. This superposition operation enables residual learning between the input features and the convolutionally processed features, allowing the model to learn the differences between them. This solves the gradient vanishing problem in deep learning models and improves the training efficiency and performance of the model.

[0086] Step S336: Input the first residual feature into the second residual block, and repeat the convolution, activation, batch normalization and skip connection operations in the first residual block to generate the second residual feature. The number of convolution kernels in the second residual block is twice that of the first residual block, thereby improving the feature dimension.

[0087] After obtaining the first residual feature, it is input into the second residual block. In the second residual block, the convolution, activation, batch normalization, and skip connection operations from the first residual block are repeated. Specifically, convolution is performed on the first residual feature to extract local feature information, followed by ReLU activation to introduce non-linearity, normalization to eliminate dimensional differences between channels, and finally, skip connections are used to superimpose the input features onto the output features of the convolutional layer. Unlike the first residual block, the second residual block has twice the number of convolution kernels. By increasing the number of kernels, the feature dimensionality is increased, allowing the second residual feature to contain richer feature information.

[0088] Step S337: Input the output features of each residual block into the next residual block in sequence. The number of convolution kernels in each subsequent residual block is twice that of the previous residual block. Through the cascading processing of multiple residual blocks, the dimensionality and abstraction of the features are gradually improved, and a deep feature representation vector with multi-scale feature information is generated.

[0089] After obtaining the second residual feature, it is input into the third residual block, and the convolution, activation, batch normalization, and skip connection operations are repeated. The output features of each residual block are sequentially input into the next residual block, with the number of convolutional kernels in each subsequent residual block being twice that of the previous one. Through the cascading processing of multiple residual blocks, the dimensionality and abstraction of the features are gradually increased. As the number of residual blocks increases, the dimensionality of the feature vector continuously increases, and the feature representation becomes increasingly abstract. Ultimately, a deep feature representation vector with multi-scale feature information is generated. This deep feature representation vector contains feature information at different scales from low to high levels, providing a more comprehensive feature representation for global feature extraction and defect identification.

[0090] Step S340: Calculate the average value of the deep feature representation vector in the spatial dimension to obtain the global feature vector with a global receptive field.

[0091] The global receptive field refers to the range of an input image that a neuron in a neural network can perceive. In this embodiment of the invention, the average value of the deep feature representation vector in the spatial dimension is calculated. That is, the feature values ​​of the deep feature representation vector at each position in the spatial direction are averaged to obtain a new feature vector, namely, a global feature vector with a global receptive field. The global feature vector contains global information from the deep feature representation vector. By calculating the average value, the local information in the spatial dimension is integrated to obtain a feature vector with global representativeness.

[0092] As one implementation method, step S340, calculating the average value of the deep feature representation vector in the spatial dimension to obtain a global feature vector with a global receptive field, may specifically include the following steps S341~S346:

[0093] Step S341: Divide the deep feature representation vector into shallow feature sub-vectors, mid-layer feature sub-vectors and deep feature sub-vectors according to the hierarchy of the residual network module.

[0094] Among them, the shallow feature vectors correspond to the output of the residual blocks in the first part, the middle feature vectors correspond to the output of the residual blocks in the middle part, and the deep feature vectors correspond to the output of the residual blocks in the last part. The sum of the proportions of the first part, the middle part, and the last part is equal to the total proportion of residual blocks in the residual network module.

[0095] After obtaining the deep feature representation vector, it is divided according to the hierarchy of the residual network modules to extract more comprehensive feature information. For example, the deep feature representation vector is divided into shallow feature sub-vectors, mid-level feature sub-vectors, and deep feature sub-vectors. The shallow feature sub-vectors correspond to the output of the first proportion of residual blocks, which extract relatively low-level feature information, such as edges and textures. The mid-level feature sub-vectors correspond to the output of the middle proportion of residual blocks, which extract mid-level feature information, such as the local shape of objects. The deep feature sub-vectors correspond to the output of the last proportion of residual blocks, which extract high-level feature information, such as the overall shape and category of objects. The sum of the proportions of the first, middle, and last parts equals the total proportion of residual blocks in the residual network module. Through this division, feature information at different levels can be processed and analyzed separately.

[0096] Step S342: Perform spatial dimension max pooling on the shallow feature vectors to retain the local detail information with the strongest response in the feature map and generate shallow key feature vectors.

[0097] Max pooling can reduce the spatial size of a feature map without changing the number of channels. In this embodiment of the invention, spatial max pooling is performed on the shallow feature vectors; that is, for each channel in the shallow feature vector, the maximum value within a local region is taken as the output of that region. Through max pooling, the strongest local detail information in the feature map is preserved, redundant information is removed, and a shallow key feature vector is generated. The shallow key feature vector contains important local detail information from the shallow feature vectors.

[0098] Step S343: Perform spatial dimension average pooling on the mid-layer feature sub-vectors to extract regional statistical information from the feature map and generate mid-layer regional feature vectors.

[0099] Average pooling can reduce the spatial size of a feature map without changing the number of channels. In this embodiment of the invention, spatial average pooling is performed on the mid-layer feature vectors; that is, for each channel in the mid-layer feature vector, the average value within a local region is taken as the output of that region. Through this average pooling process, regional statistical information, such as the average gray value of the region, is extracted from the feature map, generating a mid-layer region feature vector. The mid-layer region feature vector contains the regional statistical information from the mid-layer feature vectors.

[0100] Step S344: Perform global average pooling on the deep feature sub-vectors to obtain the overall distribution features of the feature map and generate deep global feature vectors.

[0101] Global average pooling performs average pooling on the entire feature map in the spatial dimension, obtaining a scalar value as the output of each channel. In this embodiment of the invention, global average pooling is performed on the deep feature sub-vectors in the spatial dimension. That is, for each channel in the deep feature sub-vectors, the feature values ​​of all pixels in the entire feature map are averaged in the spatial direction to obtain a scalar value. Through this global average pooling process, the overall distribution characteristics of the feature map are obtained, generating a deep global feature vector. The deep global feature vector contains global information from the deep feature sub-vectors.

[0102] Step S345: Concatenate the shallow key feature vector, the middle region feature vector, and the deep global feature vector along the channel dimension to generate a multi-scale fused feature vector.

[0103] After obtaining the shallow key feature vector, the mid-level regional feature vector, and the deep global feature vector, these three feature vectors are concatenated along the channel dimension, that is, they are connected in the channel direction to generate a new feature vector, namely the multi-scale fused feature vector. The multi-scale fused feature vector contains feature information from different scales from shallow to deep layers. Through the concatenation operation, this feature information of different scales is integrated, providing a more comprehensive input for dimensionality reduction processing.

[0104] Step S346: The multi-scale fused feature vector is reduced in dimensionality by a fully connected layer, compressing the feature dimension to a preset target dimension, and generating a global feature vector with a global receptive field. The target dimension matches the dimension of the defect category space.

[0105] The fully connected layer connects each input neuron to each output neuron. In this embodiment, the multi-scale fused feature vector is dimensionality reduced using the fully connected layer. The weight matrix of the fully connected layer compresses the feature dimension of the multi-scale fused feature vector, reducing it from a high dimension to a preset target dimension. The target dimension matches the dimension of the defect category space, allowing the global feature vector to be better used for defect category recognition. Through this dimensionality reduction, a global feature vector with a global receptive field is generated. This global feature vector contains global information from the multi-scale fused feature vector, and its dimension matches the dimension of the defect category space.

[0106] Step S350: Map the global feature vector to a preset defect category space through a linear transformation to generate a defect probability distribution vector containing the probability values ​​of each category.

[0107] The preset defect category space is a predefined space containing all possible defect categories. In this embodiment, the global feature vector is mapped to the preset defect category space through a linear transformation. The linear transformation is achieved using a weight matrix and a bias vector. The global feature vector is multiplied by the weight matrix, and then the bias vector is added to obtain a new vector. Each element in this new vector represents the score of the global feature vector belonging to a certain defect category. Then, a softmax function is used to convert these scores into probability values, generating a defect probability distribution vector containing the probability values ​​of each category. Each element in the defect probability distribution vector represents the probability that the global feature vector belongs to a certain defect category, and the sum of the probability values ​​of all elements is 1.

[0108] Step S360: Perform softmax normalization on the defect probability distribution vector to convert the probability values ​​into a probability distribution with a sum of 1. Extract the category with the largest probability value as the primary defect category, and record other categories with probability values ​​greater than a preset secondary threshold as secondary defect categories.

[0109] Softmax normalization is used to convert a set of real numbers into a probability distribution. It transforms each element of the input vector into a probability value, with the sum of all probabilities being 1. In this embodiment, the defect probability distribution vector is subjected to softmax normalization to ensure that the probability values ​​are converted into a probability distribution that sums to 1. Then, the category with the highest probability value is extracted from the normalized defect probability distribution vector as the primary defect category, representing the defect category to which the global feature vector most likely belongs. Simultaneously, a preset secondary threshold is set, and other categories with probability values ​​greater than this threshold are recorded as secondary defect categories, representing other defect categories to which the global feature vector may belong.

[0110] Step S370: Combine the primary defect category, secondary defect category and corresponding probability value to obtain the preliminary defect identification result. The preliminary defect identification result also includes the confidence score corresponding to each defect category.

[0111] After obtaining the primary defect category, secondary defect category, and their corresponding probability values, these are combined to obtain the preliminary defect identification results. Simultaneously, a confidence score is calculated for each defect category. The confidence score is determined based on the probability value of that defect category; the higher the probability value, the higher the confidence score. The preliminary defect identification results include the primary defect category, secondary defect category, corresponding probability values, and the confidence score for each defect category.

[0112] Step S400: Based on the preliminary defect identification results, determine the defect type and spatial distribution characteristics of the defects on the surface of the polarizer to be tested.

[0113] The preliminary defect identification results provide information on the categories and probabilities of defects that may exist on the surface of the polarizer. In this embodiment of the invention, based on this information, the defect type of the defects present in the polarizer to be detected is determined, that is, it is determined which specific defect type each defect belongs to. At the same time, the spatial distribution characteristics of the defects on the surface of the polarizer are determined, including the location, size, and distance between the defects.

[0114] As one implementation method, step S400, based on the preliminary defect identification results, determines the defect type and spatial distribution characteristics of the defects on the polarizer surface, which may specifically include the following steps S410~S470:

[0115] Step S410: Analyze the primary and secondary defect categories in the preliminary defect identification results to determine all defect types present in the polarizer to be tested. Each defect type corresponds to a unique defect category identifier.

[0116] After obtaining the initial defect identification results, the primary and secondary defect categories are analyzed. By matching these categories with predefined defect types, all defect types present in the polarizer under inspection are determined. Each defect type corresponds to a unique defect category identifier, which can be used for subsequent defect information recording and management. Through analysis, the specific defect types present on the polarizer surface are clarified, providing a basis for defect prioritization and spatial distribution analysis.

[0117] Step S420: Extract the confidence score corresponding to each defect type in the preliminary defect identification results, sort the confidence scores in descending order, and generate a defect type priority sequence.

[0118] After identifying all defect types in the polarizer to be inspected, a confidence score is extracted for each defect type. The confidence score reflects the reliability and accuracy of the defect type identification. The confidence scores are then sorted from highest to lowest to generate a defect type priority sequence.

[0119] Step S430: Number the defect candidate area units and establish an association relationship between each defect candidate area unit and the defect type. The primary defect category is associated with the defect candidate area unit with the largest area, and the secondary defect categories are associated with the remaining defect candidate area units in order of priority.

[0120] After obtaining the defect type priority sequence, candidate defect regions are numbered, with each region assigned a unique number. Then, an association is established between each candidate defect region and the defect type. For example, the primary defect category is associated with the candidate defect region with the largest area, as the largest candidate defect region is more likely to contain a primary defect. For secondary defect categories, the remaining candidate defect regions are associated sequentially according to their priority sequence. Through this association, the defect types that each candidate defect region may contain are clearly defined.

[0121] Step S440: Obtain the set of pixel coordinates of each defect candidate region unit associated with a defect type in the polarizer surface image, and calculate the geometric center coordinates of the set of pixel coordinates as the center positioning coordinates of the defect.

[0122] After establishing the association between candidate defect regions and defect types, for each candidate defect region associated with a defect type, its pixel coordinate set in the polarizer surface image is obtained. This pixel coordinate set contains the coordinate information of all pixels within that candidate defect region. Then, the geometric center coordinates of this pixel coordinate set are calculated by summing the x and y coordinates of all pixels in the set and dividing by the number of pixels. These geometric center coordinates are used as the center location coordinates of the defect, which can be used to determine the defect's position on the polarizer surface.

[0123] Step S450: Determine the minimum bounding rectangle of the defect candidate region unit based on the pixel coordinate set, and calculate the width and height parameters of the minimum bounding rectangle as the size feature parameters of the defect.

[0124] The minimum bounding rectangle is the smallest rectangle that can completely contain the candidate defect region unit. In this embodiment of the invention, the minimum bounding rectangle of the candidate defect region unit is determined based on the set of pixel coordinates. For example, all pixels in the set of pixel coordinates are traversed to find the minimum and maximum values ​​of the horizontal coordinate, as well as the minimum and maximum values ​​of the vertical coordinate. Using these minimum and maximum values ​​as boundaries, the position and size of the minimum bounding rectangle are determined. Then, the width and height parameters of the minimum bounding rectangle are calculated. The width parameter is the difference between the coordinates of the right and left boundaries of the minimum bounding rectangle, and the height parameter is the difference between the coordinates of the upper and lower boundaries of the minimum bounding rectangle. These width and height parameters are used as the size characteristic parameters of the defect, which can be used to describe the size of the defect on the polarizer surface.

[0125] In one implementation, step S450, determining the minimum bounding rectangle of the defect candidate region unit based on the pixel coordinate set, and calculating the width and height parameters of the minimum bounding rectangle as the size feature parameters of the defect, may specifically include the following steps S451~S456:

[0126] Step S451: Traverse all pixels in the pixel coordinate set, extract the x-coordinate and y-coordinate of each pixel, and obtain the x-coordinate set and y-coordinate set.

[0127] After obtaining the pixel coordinate set, iterate through all pixels within it. For each pixel, extract its x-coordinate and y-coordinate, and store these coordinates in a x-coordinate set and a y-coordinate set, respectively. The x-coordinate set and y-coordinate set contain the x-coordinate and y-coordinate information of all pixels in the pixel coordinate set.

[0128] Step S452: Find the minimum and maximum values ​​in the set of horizontal coordinates, and use them as the left and right boundary coordinates of the minimum bounding rectangle, respectively.

[0129] After obtaining the set of horizontal coordinates, find the minimum and maximum values. The minimum value is used as the left boundary coordinate of the minimum bounding rectangle, and the maximum value is used as the right boundary coordinate. Through this search operation, the horizontal boundary position of the minimum bounding rectangle is determined.

[0130] Step S453: Find the minimum and maximum values ​​in the set of ordinates, and use them as the lower and upper boundary coordinates of the minimum bounding rectangle, respectively.

[0131] After obtaining the set of ordinates, find the minimum and maximum values. The minimum value is used as the lower boundary coordinate of the minimum bounding rectangle, and the maximum value is used as the upper boundary coordinate. Through this search operation, the vertical boundary position of the minimum bounding rectangle is determined.

[0132] Step S454: Calculate the difference between the coordinates of the right boundary and the left boundary to obtain the width parameter of the minimum bounding rectangle; calculate the difference between the coordinates of the upper boundary and the lower boundary to obtain the height parameter of the minimum bounding rectangle.

[0133] After determining the coordinates of the left and right boundaries of the minimum bounding rectangle, the difference between the right and left boundary coordinates is calculated to obtain the width parameter of the minimum bounding rectangle. The width parameter represents the size of the minimum bounding rectangle in the horizontal direction. After determining the coordinates of the top and bottom boundaries of the minimum bounding rectangle, the difference between the top and bottom boundary coordinates is calculated to obtain the height parameter of the minimum bounding rectangle. The height parameter represents the size of the minimum bounding rectangle in the vertical direction.

[0134] Step S455: Convert the width and height parameters into physical size parameters based on the actual size of the polarizer, where the physical size parameter is equal to the pixel size parameter multiplied by the pixel resolution conversion coefficient of the image acquisition system.

[0135] After obtaining the width and height parameters of the minimum bounding rectangle, these pixel size parameters are converted into physical size parameters based on the actual size of the polarizer. The pixel resolution conversion coefficient of the image acquisition system is a predetermined coefficient representing the size of each pixel in the actual physical dimensions. Multiplying the width and height parameters by the pixel resolution conversion coefficient of the image acquisition system yields the corresponding physical width and height parameters. Through this conversion, the pixel size parameters are transformed into physical size parameters, allowing the size of the defect to be described using actual physical dimensions.

[0136] Step S456: Combine the standardized width and height parameters to obtain the size characteristic parameters of the defect. The size characteristic parameters are used to describe the actual size of the defect on the polarizer surface.

[0137] After obtaining the physical width and physical height parameters, they are combined to obtain the defect's dimensional characteristic parameters. These dimensional characteristic parameters describe the actual size of the defect on the polarizer surface, providing a clear understanding of its size.

[0138] Step S460: Calculate the Euclidean distance between the coordinates of any two defect centers and generate the distance matrix between defects.

[0139] In this embodiment of the invention, the Euclidean distance between the center coordinates of any two defects is calculated. For each pair of defect center coordinates, the sum of the squares of the differences between their X-axis and Y-axis coordinates is calculated, and then the square root of this sum is taken to obtain the Euclidean distance between the two defects. All Euclidean distance values ​​between defects are stored in a matrix to generate a defect distance matrix. The defect distance matrix can be used to analyze the spatial relationships between defects and determine which defects are close together and may be related.

[0140] As one implementation method, step S460, calculating the Euclidean distance between the location coordinates of any two defect centers to generate a distance matrix between defects, may specifically include the following steps S461~467:

[0141] Step S461: Obtain the set of center positioning coordinates for all defects. The set of center positioning coordinates includes the X-axis coordinate and Y-axis coordinate of each defect.

[0142] After obtaining the center coordinates of each defect, the center coordinates of all defects are collected to form a center coordinate set. The center coordinate set contains the X-axis and Y-axis coordinates of each defect.

[0143] Step S462: Construct a two-dimensional coordinate system, which is used to describe the position of the defect center location coordinates in the polarizer surface image.

[0144] To facilitate the calculation of the distance between the coordinates of the defect center, a two-dimensional coordinate system is constructed. The origin of the two-dimensional coordinate system can be set at the upper left corner of the polarizer surface image, with the X-axis and Y-axis representing the horizontal and vertical directions, respectively. Using this two-dimensional coordinate system, the center coordinates of each defect can be represented as a point on a two-dimensional plane.

[0145] Step S463: Traverse each center positioning coordinate in the center positioning coordinate set, take it as the current defect coordinate, and pair it with all other defect coordinates in the set in turn.

[0146] After constructing a two-dimensional coordinate system, each center positioning coordinate in the set of center positioning coordinates is traversed. The currently traversed center positioning coordinate is taken as the current defect coordinate, and then it is paired with all other defect coordinates in the set in turn.

[0147] Step S464: For each pair of defect coordinates, calculate the sum of the square of the difference between the X-axis coordinates and the square of the difference between the Y-axis coordinates, and then perform a square root operation on the sum to obtain the Euclidean distance between the two defects.

[0148] Step S465: Using the defect number as the row and column index, fill the calculated Euclidean distance value into the corresponding matrix position to construct an initial distance matrix, where the diagonal elements of the matrix are 0, representing the distance of the defect itself.

[0149] After obtaining the Euclidean distance values ​​between all defect pairs, a matrix is ​​constructed using the defect numbers as row and column indices. The calculated Euclidean distance values ​​are then filled into the corresponding matrix positions. For example, if the Euclidean distance value between defect i and defect j is d... ij Then d ij Fill in the i-th row and j-th column of the matrix. The diagonal elements of the matrix are 0, representing the distances between defects themselves. By constructing the matrix in this way, we obtain the initial distance matrix, which stores the Euclidean distance information between all defects.

[0150] Step S466: Normalize the initial distance matrix by dividing all Euclidean distance values ​​by the diagonal length of the polarizer surface image to obtain the normalized distance matrix.

[0151] To eliminate the influence of different polarizer surface image sizes, the initial distance matrix is ​​normalized. For example, all Euclidean distance values ​​in the initial distance matrix are divided by the diagonal length of the polarizer surface image. The diagonal length of the polarizer surface image can be calculated from the image's width and height, i.e., diagonal length = By normalizing the data, a normalized distance matrix was obtained. The elements in the normalized distance matrix represent the relative distances between defects, which facilitates the comparison and analysis between different images.

[0152] Step S467: Mark the positions of elements in the normalized distance matrix that are less than a preset distance threshold, identify the defect pairs corresponding to these positions as potential associated defect groups, and generate a distance matrix between defects containing the association relationship markers.

[0153] After obtaining the normalized distance matrix, a preset distance threshold is set. Elements in the normalized distance matrix with distances less than the preset threshold are located, and the defect pairs corresponding to these locations are identified as potentially related defect groups. Potentially related defect groups indicate that these defects are close to each other and may be related in some way, such as being caused by the same reason. The locations of these potentially related defect groups are marked in the normalized distance matrix, generating an inter-defect distance matrix containing association markers. This inter-defect distance matrix with association markers helps users quickly understand which defects may be related.

[0154] Step S470: Combine the defect type, center positioning coordinates, size feature parameters and the distance matrix between defects to obtain the spatial distribution feature information of defects on the polarizer surface. The spatial distribution feature information also includes the statistical results of the number of candidate region units corresponding to each defect type.

[0155] After obtaining the defect type, center location coordinates, dimensional characteristic parameters, and distance matrix between defects, this information is combined to obtain the spatial distribution characteristic information of defects on the polarizer surface. This spatial distribution characteristic information describes the location, size, distance between defects, and defect type on the polarizer surface. Simultaneously, the number of candidate defect region units corresponding to each defect type is counted, and this statistical result is also included in the spatial distribution characteristic information. This spatial distribution characteristic information helps users comprehensively understand the defect distribution on the polarizer surface.

[0156] As one implementation method, the method provided in this embodiment of the invention may further include the following steps S500~S900:

[0157] Step S500: Extract the quality level threshold and defect allowable range parameters associated with each defect type from the preset quality standard library.

[0158] The preset quality standard library is a database that pre-stores quality grade thresholds and defect allowable range parameters corresponding to various defect types. In this embodiment of the invention, the quality grade threshold and defect allowable range parameters associated with each defect type are extracted from the preset quality standard library. The quality grade threshold represents the upper limit of the allowed defect size under different quality grades, and the defect allowable range parameter represents the allowable distribution range of defects on the polarizer surface. By extracting these parameters, standards can be provided for subsequent defect acceptability determination.

[0159] Step S600: Compare the size feature parameters in the spatial distribution feature information with the corresponding quality level threshold. If the size feature parameters are less than the quality level threshold, the defect is determined to be an acceptable defect; otherwise, it is determined to be an unacceptable defect.

[0160] After obtaining the quality grade threshold associated with each defect type, the dimensional feature parameters in the spatial distribution feature information are compared with the corresponding quality grade threshold. The dimensional feature parameters include the width and height parameters of the defect, and the quality grade threshold represents the upper limit of the allowable defect size under different quality grades. When the dimensional feature parameter is less than the quality grade threshold, it means that the size of the defect is within the allowable range, and the defect is determined to be an acceptable defect; when the dimensional feature parameter is greater than or equal to the quality grade threshold, it means that the size of the defect exceeds the allowable range, and the defect is determined to be an unacceptable defect.

[0161] In one implementation, step S600 compares the size characteristic parameter in the spatial distribution feature information with the corresponding quality level threshold. If the size characteristic parameter is less than the quality level threshold, the defect is determined to be an acceptable defect; otherwise, it is determined to be an unacceptable defect. Specifically, this may include the following steps S610~S670:

[0162] Step S610: Read the quality level threshold table corresponding to the current defect type from the preset quality standard library. The quality level threshold table contains the upper limit value of defect size under different quality levels.

[0163] When determining defect acceptability, the system first retrieves the quality level threshold table corresponding to the current defect type from a pre-defined quality standard library. This table contains the upper limits of allowed defect sizes for different quality levels, such as the upper limit for defect size corresponding to quality level A, quality level B, etc. By retrieving the quality level threshold table, the dimensional standards for the current defect type at different quality levels can be obtained, providing a basis for subsequently determining the threshold based on the specific quality level. For example, for scratch defects, the quality level threshold table might list the upper limits for scratch length and width under the first-level quality standard, as well as the corresponding upper limits under second- and third-level quality standards.

[0164] Step S620: Query the corresponding target quality level according to the product model of the polarizer to be tested, and extract the upper limit value of the defect size corresponding to the target quality level from the quality level threshold table as the quality level threshold.

[0165] The product model of the polarizer to be inspected determines the quality level requirements it should meet. Different product models may correspond to different target quality levels due to factors such as application scenarios and customer needs. By consulting a pre-established correspondence table between product models and quality levels, the target quality level of the polarizer to be inspected is determined. Then, the upper limit value of the defect size corresponding to the target quality level is extracted from the quality level threshold table read in step S610, and this value is used as the quality level threshold for this defect acceptability determination. For example, if the product model of the polarizer to be inspected corresponds to the first-level quality standard, then the upper limit value of the defect type under the first-level quality standard is extracted from the quality level threshold table.

[0166] Step S630: Obtain the size feature parameters corresponding to the current defect type in the spatial distribution feature information. The size feature parameters include width and height parameters.

[0167] After obtaining the quality level threshold, it is necessary to acquire the actual size characteristic parameters of the current defect type. These parameters are stored in the previously generated spatial distribution characteristic information. The size characteristic parameters include the width and height parameters of the defect, which are obtained in step S450 and its sub-steps by calculating the minimum bounding rectangle of the defect candidate region unit and converted into physical size parameters based on the actual size of the polarizer. For example, for a circular defect, the width and height parameters may be approximately equal, while for a rectangular defect, the width and height parameters represent the lengths of its short and long sides, respectively.

[0168] Step S640: Compare the width parameter with the quality grade threshold. If the width parameter is less than the quality grade threshold, the width comparison result is passed; otherwise, it is failed.

[0169] The width parameter of the current defect type is compared with the quality grade threshold. This is a simple numerical comparison process. If the width parameter is less than the quality grade threshold, it means that the defect's dimensions in the width direction meet the quality requirements, and the width comparison result is "pass." Conversely, if the width parameter is greater than or equal to the quality grade threshold, it means that the defect's dimensions in the width direction exceed the allowable range, and the width comparison result is "fail." For example, if the quality grade threshold specifies that the scratch width cannot exceed 0.5 mm, and the current defect's width parameter is 0.3 mm, then the width comparison result is "pass"; if the width parameter is 0.6 mm, then the width comparison result is "fail."

[0170] Step S650: Compare the height parameter with the quality grade threshold. If the height parameter is less than the quality grade threshold, the height comparison result is passed; otherwise, it is failed.

[0171] Similarly, the height parameter is compared. The height parameter of the current defect type is compared with the quality grade threshold. If the height parameter is less than the quality grade threshold, it means that the size of the defect in the height direction is within the allowable range, and the height comparison result is passed. If the height parameter is greater than or equal to the quality grade threshold, it means that the size of the defect in the height direction exceeds the quality requirements, and the height comparison result is failed.

[0172] Step S660: When both the width comparison result and the height comparison result are passed, the defect is determined to be an acceptable defect; when at least one of the width comparison result or the height comparison result is failed, the defect is determined to be an unacceptable defect.

[0173] The acceptability of a defect is determined by combining the results of both width and height comparisons. If both width and height comparisons pass, the defect is considered acceptable, meaning its dimensions meet quality requirements in all directions. If either the width or height comparison fails, the defect is considered unacceptable, indicating its dimensions exceed the allowable range in some direction. For example, for an elliptical defect, if both the width and height comparisons pass, the defect is acceptable; however, if the width comparison fails, even if the height comparison passes, the defect is unacceptable.

[0174] Step S670: Record the acceptability judgment result of each defect to the judgment result list, and associate it with the corresponding defect type and size characteristic parameters for subsequent generation of polarizer quality inspection report.

[0175] After determining the acceptability of each defect, the results are recorded in a result list. Simultaneously, to facilitate subsequent viewing and analysis, each result is associated with its corresponding defect type and dimensional characteristic parameters. This way, when generating a polarizer quality inspection report, the specific details of each defect and whether it meets quality requirements can be clearly displayed. For example, the result list might be presented in tabular form, with each row recording information for one defect, including defect type, width parameters, height parameters, and acceptability determination result.

[0176] Step S700: Statistically analyze the number of defect types in the spatial distribution characteristic information, calculate the proportion of unacceptable defects to the total number of defects, and use this as an evaluation indicator for the defect pass rate.

[0177] First, the number of defect types is statistically analyzed from spatial distribution feature information, recording the frequency of each defect type. Then, the number of unacceptable defects is counted, i.e., defects deemed unacceptable are selected from the judgment result list and counted. Simultaneously, the total number of defects is calculated. Finally, the proportion of unacceptable defects to the total number of defects is calculated and used as the defect pass rate evaluation index. This index directly reflects the overall quality of the polarizer; a lower proportion indicates better polarizer quality.

[0178] Step S800: Based on the center positioning coordinates and size feature parameters in the spatial distribution feature information, generate a positioning coordinate description of each defect on the polarizer surface. The positioning coordinate description includes the X-axis coordinates and Y-axis coordinates of the defect center and the coordinates of the diagonal vertices of the defect's circumscribed rectangle.

[0179] The location coordinates of each defect are generated using the center coordinates and size feature parameters from the previously generated spatial distribution feature information. The center coordinates, calculated in step S440, represent the pixel coordinates of the defect's center position in the polarizer surface image. The size feature parameters include the defect's width and height, which allow the calculation of the diagonal vertices of the defect's circumscribed rectangle. Specifically, assuming the defect's center coordinates are (Xc, Yc), the width parameter is W, and the height parameter is H, then the coordinates of the top-left vertex of the defect's circumscribed rectangle are (Xc-W / 2, Yc-H / 2), and the coordinates of the bottom-right vertex are (Xc+W / 2, Yc+H / 2). Combining the X-axis and Y-axis coordinates of the defect's center with the coordinates of the diagonal vertices of the circumscribed rectangle forms the location coordinate description, used to accurately locate the position of each defect on the polarizer surface.

[0180] Step S900: Format the defect type, defect pass rate evaluation index, location coordinate description, and acceptability judgment result according to the preset report template to generate a polarizer quality inspection report containing text description and table data.

[0181] The preset report template is a pre-designed format used to standardize the content and presentation of polarizer quality inspection reports. The previously obtained defect types, defect pass rate assessment indicators, location coordinate descriptions, and acceptability judgment results are formatted according to the report template. The report may include a text description section for an overall explanation and analysis of the inspection results, such as the distribution of defect types and the analysis of major defect types; it may also include a tabular data section, presenting detailed information for each defect in tabular form, such as defect type, center location coordinates, dimensional characteristic parameters, and acceptability judgment results. Through this formatting process, a complete, clear, and easy-to-understand polarizer quality inspection report is generated, providing a strong basis for polarizer quality assessment and subsequent processing.

[0182] It is understood that the various algorithms involved in the above descriptions of the embodiments of the present invention, such as adaptive histogram equalization algorithm, contour tracking algorithm, Euclidean algorithm, etc., can all be obtained from relevant content in the prior art. To save space, they will not be elaborated on in the embodiments of the present invention. In addition, those skilled in the art can supplement the details based on common knowledge in the art when implementing the solution of the present invention. For example, they can use normalization to eliminate dimensional conflicts before feature fusion, use interpolation to eliminate dimensional differences, reasonably set the threshold based on historical data, experience or business scenario requirements, train the model based on a general model training method, set the number of layers in the model structure based on actual needs, select the activation function, etc. The present invention will not provide redundant descriptions of overly detailed implementation processes here.

[0183] Please see Figure 2 , Figure 2This is a schematic diagram of a defect detection system provided in an embodiment of the present invention. The defect detection system can be a computer system installed on a polarizer production line, comprising at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 can be connected via a bus or other means. The processor 101 (or Central Processing Unit, CPU) is the computational and control core of the defect detection system, capable of parsing various instructions and processing various data within the system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 101; the communication interface 102 can also be used for data transmission and interaction within the defect detection system. The memory 103 is a storage device in the defect detection system used to store programs and data. It is understood that the memory 103 here can include the built-in memory of the defect detection system, or it can include extended memory supported by the defect detection system. The memory 103 provides storage space for storing the operating system of the defect detection system, which is not limited in this invention.

[0184] In one embodiment, the processor 101 executes the deep learning-based polarizer defect detection method provided above in the embodiments of the present invention by running a computer program in the memory 103.

Claims

1. A method for detecting defects in polarizers based on deep learning, characterized in that, The method comprises: acquiring an initial image data set corresponding to a polarizer to be detected, the initial image data set containing a plurality of polarizer surface images under different acquisition angles; extracting defect candidate region features in the polarizer surface images and texture distribution features of the polarizer surface from the initial image data set through image preprocessing and feature extraction operations; inputting the defect candidate region features and the texture distribution features into a pre-trained defect recognition deep learning model to perform joint defect recognition operations, and generating a defect preliminary recognition result of the polarizer surface image; specifically including: splicing the defect candidate region features and the texture distribution features along the channel dimension to generate an initial fusion feature vector; adjusting the mean and variance of each channel in the initial fusion feature vector to a preset range to eliminate the dimensional difference between different feature channels, and generating a normalized fusion feature vector; inputting the normalized fusion feature vector into a residual network module of the defect recognition deep learning model, performing nonlinear transformation and dimensionality lifting on the feature vector through a plurality of residual blocks, each residual block containing two convolution layers and a skip connection, directly superimposing the input features onto the convolution layer output features through the skip connection to generate a deep feature representation vector; calculating the average value of the deep feature representation vector in the spatial dimension to obtain a global feature vector with a global receptive field; mapping the global feature vector to a preset defect category space through linear transformation to generate a defect probability distribution vector containing probability values of each category; performing softmax normalization processing on the defect probability distribution vector to convert the probability values into a probability distribution with a sum of 1, and extracting the category with the maximum probability value as the main defect category while recording other categories with probability values greater than a preset secondary threshold as secondary defect categories; combining the main defect category, the secondary defect category and the corresponding probability values to obtain the defect preliminary recognition result, the defect preliminary recognition result further containing a confidence score corresponding to each defect category; determining the defect type and the spatial distribution feature information of the defect in the polarizer to be detected based on the defect preliminary recognition result.

2. The method of claim 1, wherein, The method comprises: adjusting the gray scale distribution range of each polarizer surface image in the initial image data set through an adaptive histogram equalization algorithm to generate light equalization image data; eliminating random noise and salt and pepper noise in the light equalization image data based on the cooperation of Gaussian filtering and median filtering to generate denoised image data; enhancing the gray scale change area in the denoised image data through a Laplacian operator to highlight the edge contour information of the defect area, and generating edge enhanced image data; taking a preset seed point as the starting point, merging adjacent pixel points in the edge enhanced image data with a gray value difference within a preset range into the same region to obtain a plurality of defect candidate region units; extracting area area proportion, contour perimeter and area ratio and circumscribed rectangle length-width ratio of each of the defect candidate region units, and combining to obtain the defect candidate region features; calculating texture energy, entropy value and contrast parameter of the denoised image data in different directions and distances through a gray level co-occurrence matrix, and combining to obtain the texture distribution features; adjusting dimensions of the defect candidate region features and the texture distribution features to the same dimension through linear mapping, and generating a joint feature set with unified dimension representation.

3. The method of claim 2, wherein, the edge enhancement image data is generated by strengthening the gray scale change area in the denoised image data through the Laplace operator, and highlighting the edge contour information of the defect area, including: converting the denoised image data into a single-channel gray scale image, retaining the brightness information in the image, and generating gray scale image data; calculating gradient values of pixel points in the gray scale image data in horizontal and vertical directions respectively based on a Sobel operator, and obtaining a horizontal gradient image and a vertical gradient image; performing square and square root operations on the horizontal gradient image and the vertical gradient image, and merging to obtain a gradient amplitude image representing edge intensity; traversing each pixel point in the gradient amplitude image, comparing the gradient amplitude of the pixel point with that of adjacent pixel points in the gradient direction, retaining local maximum value pixel points, and obtaining a thinned edge image; setting a high threshold and a low threshold to perform double threshold processing on the thinned edge image, marking pixel points with gradient amplitude greater than the high threshold as strong edges, marking pixel points with gradient amplitude between the low threshold and the high threshold and connected with the strong edges as weak edges, and marking pixel points with gradient amplitude less than the low threshold as non-edges; judging whether the pixel points marked as weak edges have a connected path with the strong edges, retaining weak edge pixel points connected with the strong edges, and deleting isolated weak edge pixel points, to obtain a complete edge image; superimposing the complete edge image and the gray scale image data, adjusting the gray scale value of the edge pixel points to a preset highlight value, and generating edge enhancement image data highlighting the edge contour of the defect area.

4. The method of claim 2, wherein, the edge enhancement image data is merged into a same region with adjacent pixel points having a gray scale value difference within a preset range, to obtain a plurality of defect candidate region units, including: setting a gray scale threshold to convert the edge enhancement image data into a black and white binary image, wherein the pixel value of the edge contour region is 1 and the pixel value of the background region is 0, to generate a binary edge image; performing a morphological closing operation of dilation and then corrosion on the binary edge image based on a structure element with a preset size, filling small cavities in the edge contour, and connecting broken edge segments, to generate a closed edge image; extracting a pixel coordinate set of each closed contour in the closed edge image through a contour tracking algorithm, determining a minimum circumscribed rectangle of each closed contour, and taking the center pixel point of the minimum circumscribed rectangle as a seed point for region growing; sampling the pixel gray scale value corresponding to each seed point in the denoised image data as an initial gray scale reference value for region growing; The seed point is taken as a center, adjacent pixel points are traversed, an absolute difference value of the gray scale value of the adjacent pixel points and an initial gray scale reference value is calculated, when the absolute difference value is less than a preset gray scale difference threshold, the adjacent pixel points are included in a current growth region, and the gray scale reference value of the region is updated as an average gray scale value of all pixel points in the region; The steps of neighborhood search and region growth are repeatedly executed until there is no new pixel point satisfying the gray scale difference condition, the growth process of the current region is stopped, and a complete region unit is obtained; Region units with an area less than a preset minimum area threshold are deleted, and region units with an area greater than or equal to the preset minimum area threshold are reserved as defect candidate region units.

5. The method of claim 1, wherein, The residual network module of the defect recognition deep learning model is inputted with the normalized fusion feature vector, and the feature vector is subjected to non-linear transformation and dimension promotion through a plurality of residual blocks, each residual block comprising two convolution layers and a skip connection, the input feature is directly superimposed on the convolution layer output feature through the skip connection to generate a deep feature representation vector, comprising: The first convolution layer of the first residual block is inputted with the normalized fusion feature vector, a convolution operation is performed on the feature vector based on a preset number of convolution kernels to extract local feature information and generate first convolution features; The first convolution features are subjected to ReLU activation function processing to generate first activation features; The second convolution layer of the first residual block is inputted with the first activation features, a convolution operation is performed on the features based on the same number of convolution kernels as the first convolution layer to further extract deep local features and generate second convolution features; The mean and variance of the second convolution features are adjusted to generate second normalized features; The normalized fusion feature vector is superimposed on the second normalized features through the skip connection to generate first residual features; The first residual features are inputted into the second residual block to repeatedly perform the convolution, activation, batch normalization and skip connection operations in the first residual block to generate second residual features; The output features of each residual block are sequentially inputted into the next residual block, the number of convolution kernels of each subsequent residual block is twice that of the previous residual block, the dimension and abstraction degree of the features are gradually improved through the cascade processing of the plurality of residual blocks to generate a deep feature representation vector with multi-scale feature information.

6. The method of claim 1, wherein, The defect type and the spatial distribution feature information of the defect in the polaroid are determined based on the preliminary defect recognition result, comprising: The main defect category and the secondary defect category in the preliminary defect recognition result are analyzed to determine all defect types existing in the polaroid to be detected, each defect type corresponds to a unique defect category identifier; The confidence score corresponding to each defect type in the preliminary defect recognition result is extracted, and the confidence scores are sorted in descending order to generate a defect type priority sequence; The defect candidate region units are numbered, and each defect candidate region unit is associated with the defect type, wherein the main defect category is associated with the defect candidate region unit with the largest area, and the secondary defect category is sequentially associated with the remaining defect candidate region units according to the priority sequence. Obtaining a pixel coordinate set of each defect candidate region unit associated with a defect type in the polarizing sheet surface image, calculating a geometric center coordinate of the pixel coordinate set as a center positioning coordinate of the defect; Determining a minimum circumscribed rectangle of the defect candidate region unit according to the pixel coordinate set, and calculating a width and height parameter of the minimum circumscribed rectangle as a size feature parameter of the defect; Calculating the Euclidean distance between any two defect center positioning coordinates to generate an inter-defect distance matrix; Combining the defect type, the center positioning coordinate, the size feature parameter, and the inter-defect distance matrix to obtain the spatial distribution feature information of the defect on the polarizing sheet surface, and the spatial distribution feature information further includes a quantity statistical result of the defect candidate region unit corresponding to each defect type.

7. The method of claim 6, wherein, The method further includes: Extracting a quality level threshold and a defect allowable range parameter associated with each defect type from a preset quality standard library; Comparing the size feature parameter in the spatial distribution feature information with the corresponding quality level threshold, determining that the defect is an acceptable defect when the size feature parameter is less than the quality level threshold, otherwise determining that the defect is an unacceptable defect; Statistically analyzing the quantity statistical result of the defect type in the spatial distribution feature information, calculating a proportion of the number of unacceptable defects in the total number of defects as a defect qualification rate evaluation index; Generating a positioning coordinate description of each defect on the polarizing sheet surface based on the center positioning coordinate and the size feature parameter in the spatial distribution feature information, and the positioning coordinate description includes an X-axis coordinate and a Y-axis coordinate of the defect center and a diagonal vertex coordinate of the defect circumscribed rectangle; Formatting the defect type, the defect qualification rate evaluation index, the positioning coordinate description, and the acceptability determination result according to a preset report template to generate a polarizing sheet quality detection report including text description and table data. The method further includes: A memory in which a computer program is stored.

8. The method of claim 1, wherein, ​ ​ ​ ​ ​ ​ 9. A defect detection system, characterized by, ​ ​ A processor is configured to load the computer program to implement the method for detecting defects of a polarizer based on deep learning according to any one of claims 1-8.

Images