A method and system for automatic defect lighting detection and classification of LCDs

Abstract

The application discloses an LCD defect automatic lighting detection and classification method and system, and belongs to the technical field of automatic optical detection. The method comprises the following steps: acquiring an initial image of an LCD screen collected under a standard illumination mode, detecting the initial image, and identifying a suspected defect area; extracting image features of the suspected defect area to dynamically generate a light source excitation sequence, illuminating the suspected defect area, collecting a multi-modal image sequence, extracting physical features, processing the physical features by using a logic judgment tree containing optical physical rules, and generating a defect filtering result; and according to the defect filtering result, performing defect type identification on the multi-modal image sequence to generate a defect type classification result. The dynamic generation of the light source excitation sequence for multi-modal image collection, combined with physical rule filtering and deep learning classification, can effectively improve the detection accuracy and classification precision for complex defects.

Description

Technical Field

[0001] This invention relates to the field of automated optical inspection technology, and in particular to a method and system for automatic illumination detection and classification of LCD defects. Background Technology

[0002] As a core display component of modern electronic devices, the manufacturing quality of LCDs directly affects the performance of end products and the user experience. During the LCD manufacturing process, various minor defects, such as blemishes, scratches, foreign objects, bright spots, and dark spots, are inevitably generated due to factors such as materials, equipment, and process environment. Therefore, efficient and accurate defect detection of LCD screens is a key step in ensuring product yield and controlling production costs. Automated optical inspection technology is currently the mainstream inspection method.

[0003] Existing LCD defect detection methods typically employ a fixed illumination scheme, such as acquiring LCD screen images under uniform bright or dark lighting conditions. Subsequently, traditional image processing algorithms, such as template matching, thresholding, or background subtraction, are used to identify abnormal regions in the image. Some methods have also incorporated machine learning or shallow neural networks to classify features extracted under a single, fixed illumination, attempting to distinguish between different defect types.

[0004] However, using a fixed illumination pattern to treat all types of defects the same results in poor adaptability to defects with different optical properties. Many defects may not be visible or have indistinct features under specific single illumination, leading to missed detections. Secondly, traditional image processing algorithms are sensitive to noise and environmental changes, and struggle to effectively distinguish between genuine structural defects and pseudo-defects such as surface dust and dirt, resulting in a high false alarm rate. Finally, relying solely on image information from a single modality limits the ability to accurately classify complex defects that are similar in appearance but have different physical causes. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an automatic illumination detection and classification method and system for LCD defects. It employs dynamically generated light source excitation sequences for multimodal image acquisition and combines physical rule filtering and deep learning classification techniques to effectively improve the detection accuracy and classification precision of complex defects.

[0006] The above objectives can be achieved through the following approach:

[0007] An automatic illumination detection and classification method for LCD defects includes: acquiring an initial image of an LCD screen under standard illumination mode, detecting the initial image to identify suspected defect areas; extracting image features of the suspected defect areas and dynamically generating a light source excitation sequence based on the image features; illuminating the suspected defect areas according to the light source excitation sequence to acquire a multimodal image sequence; extracting physical features from the multimodal image sequence and processing the physical features using a preset logical judgment tree containing optical physical rules to generate a defect filtering result; and identifying the defect type of the multimodal image sequence according to the defect filtering result to generate a defect type classification result.

[0008] Optionally, identifying suspected defective regions includes: acquiring an initial image of the LCD screen under standard lighting conditions, and performing grayscale processing on the initial image to generate a grayscale image; performing filtering and noise reduction processing on the grayscale image to generate a smooth image; acquiring a preset defect-free template image, and performing difference calculation between the smooth image and the defect-free template image to generate a difference image; performing binarization processing on the difference image to generate a binary image; and performing connected component analysis on the binary image to identify suspected defective regions.

[0009] Optionally, the generation of the light source excitation sequence includes: extracting the texture and brightness features of the suspected defective region to generate a region feature vector; processing the region feature vector based on a preset feature light source mode mapping relationship to generate a light source mode recommendation list; and combining and sorting multiple light source modes according to the light source mode recommendation list to generate a light source excitation sequence.

[0010] Optionally, the acquisition of the multimodal image sequence includes: adjusting the angle and intensity parameters of the light source according to the light source excitation sequence to generate optimized lighting conditions; irradiating the suspected defect area multiple times based on the optimized lighting conditions and simultaneously triggering image acquisition to obtain a test image set; and performing image alignment and radiometric calibration processing on the test image set to generate a multimodal image sequence.

[0011] Optionally, extracting physical features from the multimodal image sequence includes: calculating grayscale statistical features and geometric morphological features for each image in the multimodal image sequence to generate a statistical feature set; performing time-series analysis on the multimodal image sequence to extract the variation features of defects under different lighting conditions to generate a dynamic feature set; and merging the statistical feature set and the dynamic feature set to generate physical features.

[0012] Optionally, generating the defect filtering result includes: inputting the physical features into a preset logical judgment tree containing optical physical rules for rule matching to generate a preliminary judgment result; evaluating the confidence of the preliminary judgment result to generate a confidence score; and comparing the confidence score with a preset classification threshold to generate the defect filtering result.

[0013] Optionally, generating the defect type classification result includes: performing a convolution operation on the multimodal image sequence based on the defect filtering result to generate a high-level feature map; performing pooling processing on the high-level feature map to generate a pooled feature vector; and mapping the pooled feature vector to generate the defect type classification result.

[0014] Optionally, the method further includes: collecting the defect type classification results and the corresponding manual verification results to generate a training dataset; and using the training dataset to update the feature light source pattern mapping relationship.

[0015] Optionally, updating the feature light source pattern mapping relationship using the training dataset includes: extracting multimodal image sequences and corresponding defect labels from the training dataset to generate sample pairs; performing reinforcement learning on the updated feature light source pattern mapping relationship based on the classification correctness of the sample pairs to obtain the updated feature light source pattern mapping relationship.

[0016] Based on the same inventive concept, this invention also provides an automatic illumination detection and classification system for LCD defects. The system includes: a suspected defect identification module, used to acquire an initial image of the LCD screen under standard illumination mode, and to detect and identify suspected defect areas in the initial image; a light source sequence generation module, used to extract image features from the suspected defect areas and dynamically generate a light source excitation sequence based on the image features; a multimodal image acquisition module, used to illuminate the suspected defect areas according to the light source excitation sequence and acquire a multimodal image sequence; a physical feature processing module, used to extract physical features from the multimodal image sequence and process the physical features using a preset logical judgment tree containing optical physical rules to generate a defect filtering result; and a defect type classification module, used to identify the defect type of the multimodal image sequence according to the defect filtering result and generate a defect type classification result.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] 1. This invention generates a dynamic light source excitation sequence and generates an optimized multimodal illumination scheme for the initial image features of each suspected defect. It can actively select the light source angle and intensity combination most likely to highlight the essence of the defect, thereby collecting image data with richer information, improving the detection capability and recognition of complex, blurry or invisible defects under single illumination, and enhancing the adaptability and accuracy of detection.

[0019] 2. This invention introduces a logical judgment tree based on optical physics rules to filter the extracted physical features. This method does not simply rely on a data-driven model for black-box judgment, but combines the optical physics principles of defects for logical reasoning. It can effectively distinguish real structural defects from non-real defect artifacts such as dust, stains or image noise, effectively reducing the false alarm rate of the system and improving the reliability and credibility of the detection results.

[0020] 3. This invention constructs a closed-loop optimization system based on feedback learning of verification results; by collecting the differences between machine classification results and verification results, and using reinforcement learning algorithms to continuously update the mapping relationship of feature light source patterns, the system can continuously learn and evolve from actual production cases; this self-improvement mechanism enables the system to gradually improve the intelligence level of its strategy selection and continuously enhance its ability to detect and classify new types or rare defects.

[0021] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating an automatic LCD defect detection and classification method according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of optimized lighting conditions according to an embodiment of the present invention.

[0025] Figure 3 This is a schematic diagram of the test image set according to an embodiment of the present invention.

[0026] Figure 4This is a schematic diagram of the final physical features of an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of the structure of an automatic LCD defect detection and classification system according to an embodiment of the present invention.

[0028] Figure 6 This is a schematic diagram of the data from the experimental group and the control group in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Reference Figure 1 One embodiment of the present invention proposes an automatic illumination detection and classification method for LCD defects. It adopts a dynamically generated light source excitation sequence for multimodal image acquisition, and combines physical rule filtering and deep learning classification to effectively improve the detection accuracy and classification precision of complex defects.

[0031] The method described in this embodiment specifically includes:

[0032] Acquire an initial image of the LCD screen under standard lighting conditions, and inspect the initial image to identify suspected defect areas;

[0033] Extract image features from the suspected defect area, and dynamically generate a light source excitation sequence based on the image features;

[0034] The suspected defect area is illuminated according to the light source excitation sequence, and a multimodal image sequence is acquired.

[0035] Physical features are extracted from the multimodal image sequence, and the physical features are processed using a preset logical judgment tree containing optical physical rules to generate defect filtering results.

[0036] Based on the defect filtering results, defect type identification is performed on the multimodal image sequence to generate defect type classification results.

[0037] The dynamic generation strategy of light source excitation sequence in this invention greatly improves the visibility and identification of defects. It can proactively customize optimal observation conditions for each suspected defect, thereby effectively revealing defect features that are difficult to detect or easily confused under single, fixed lighting conditions. Secondly, the introduction of a logical judgment tree based on optical physics rules for defect filtering endows the detection process with interpretable physical reasoning capabilities, effectively enhancing the system's ability to distinguish non-real defect artifacts such as noise and dirt, effectively reducing the false alarm rate, and improving the reliability of detection results. Finally, this method integrates adaptive lighting, multimodal information fusion, and physical rule judgment, enabling more accurate and comprehensive automated detection and classification of defects in LCD screens, especially demonstrating good technical performance in handling complex and subtle defects.

[0038] Optionally, identifying suspected defective areas includes:

[0039] Acquire an initial image of the LCD screen under standard lighting mode, and perform grayscale processing on the initial image to generate a grayscale image;

[0040] The grayscale image is filtered and denoised to generate a smooth image;

[0041] Obtain a preset defect-free template image, and perform a difference calculation between the smoothed image and the defect-free template image to generate a difference image;

[0042] The difference image is binarized to generate a binary image;

[0043] Connectivity analysis was performed on the binary image to identify suspected defective regions.

[0044] Specifically, the initial image of the LCD screen under standard lighting conditions is first acquired using an image acquisition device. This initial image is typically a three-channel color image. To facilitate subsequent calculations and analysis, this initial image needs to be converted to grayscale, transforming it into a single-channel grayscale image. This process can be achieved using a weighted average method, calculated as follows:

[0045] ,

[0046] in, This represents the grayscale value of a specific pixel in the generated grayscale image. , , These represent the intensity values ​​of the red, green, and blue channels of the corresponding pixel in the initial image, respectively. , , The weighting coefficients are preset and sum to 1, typically set based on the human eye's sensitivity to different colors. After grayscale processing, to eliminate random noise introduced during image acquisition, such as Gaussian noise or salt-and-pepper noise, the grayscale image needs to be filtered and denoised to generate a smooth image. Algorithms such as Gaussian filtering or median filtering can be used, effectively suppressing noise and preserving edge details by performing a weighted average or median operation on each pixel and its neighboring pixels. Next, a preset defect-free template image acquired under the same standard lighting mode is obtained; this image represents an ideal LCD screen. The smooth image generated in the previous step is then compared with this defect-free template image by performing a difference calculation, i.e., calculating the absolute value of the difference between their grayscale values ​​pixel by pixel, thus generating a difference image. The difference image effectively highlights areas where the smooth image differs from the template image; these areas represent potential defects. To accurately segment these areas from the difference image, it needs to be binarized. By setting an appropriate threshold, pixels in the difference image with values ​​greater than the threshold are designated as foreground values ​​(e.g., 255), while pixels with values ​​less than or equal to the threshold are designated as background values ​​(e.g., 0), thus generating a binary image containing only foreground and background. Finally, connected component analysis is performed on this binary image. This analysis algorithm organizes spatially adjacent or connected foreground pixels into a single connected region. By traversing all connected regions and extracting their position, size, and other information, each identified connected region is ultimately identified as a potential defect region, providing precise targets for subsequent dynamic lighting and classification.

[0047] Optionally, the generated light source excitation sequence includes:

[0048] Extract the texture and brightness features of the suspected defective region to generate a region feature vector;

[0049] Based on the preset feature light source mode mapping relationship, the feature vector of the region is processed to generate a light source mode recommendation list.

[0050] Based on the recommended list of light source modes, multiple light source modes are combined and sorted to generate a light source excitation sequence.

[0051] Specifically, the texture and brightness features of the suspected defective region are first extracted. Texture features describe the spatial distribution and arrangement of pixel grayscale values ​​within the region and can be calculated using algorithms such as Local Binary Pattern (LBP) or Gabor filters. These algorithms can quantify the roughness, directionality, and periodicity of the region. Brightness features describe the overall brightness and contrast of the region and can be obtained by calculating statistical measures such as the average grayscale value, standard deviation, skewness, and kurtosis of the pixels within the region. The calculated texture and brightness feature values ​​are then combined and arranged in a predetermined order to form a multidimensional region feature vector, which can be represented as:

[0052] ,

[0053] in, It is the generated region feature vector. to Representing n distinct texture feature components, to This represents m different brightness feature components. These feature components are scalar values ​​directly calculated from the image data of the suspected defect area. Subsequently, this region feature vector is input into a pre-defined feature light source pattern mapping relationship for processing. This mapping relationship is a model incorporating expert knowledge or trained through machine learning; essentially, it is a function that maps the image feature space to the light source pattern space. Based on the input region feature vector, it determines which types of defects the suspected defect is most likely to belong to, and accordingly recommends a set of light source patterns that best highlight the characteristics of these potential defects, generating a light source pattern recommendation list. Each light source pattern defines a specific lighting configuration, such as high-angle bright-field illumination, low-angle dark-field illumination, or monochromatic illumination of a specific wavelength. Finally, based on this light source pattern recommendation list, multiple light source patterns in the list are combined and sorted to generate the final light source excitation sequence. The sorting can be based on the recommendation confidence score or a pre-defined diagnostic logic, such as transitioning from a general lighting pattern to a more targeted lighting pattern to ensure the comprehensiveness and efficiency of information collection.

[0054] Optionally, the acquired multimodal image sequence includes:

[0055] Based on the light source excitation sequence, the angle and intensity parameters of the light source are adjusted to generate optimized lighting conditions;

[0056] The suspected defect area is illuminated multiple times based on the optimized lighting conditions, and image acquisition is triggered simultaneously to obtain a test image set.

[0057] The test image set is subjected to image alignment and radiometric calibration to generate a multimodal image sequence.

[0058] Specifically, firstly, based on each light source mode in the light source excitation sequence, the control system drives the programmable light source to precisely adjust its physical parameters, such as angle and intensity, generating a series of optimized lighting conditions. For example, when one mode in the light source excitation sequence is "high-angle bright-field illumination," the system controls the light source to move to an incident angle smaller than the LCD screen normal and sets a higher brightness; when the mode is "low-angle dark-field illumination," the light source is moved to a very large incident angle and adjusted to a suitable intensity to highlight surface scattering. This set of physical lighting parameters, tailored to each mode in the sequence, constitutes the optimized lighting conditions required for subsequent data acquisition. Next, as... Figure 2 As shown, based on these optimized lighting conditions, the suspected defect area is illuminated one by one, and the image acquisition device is triggered simultaneously to obtain a test image set. Specifically, the first optimized lighting condition is applied first, and after the light source stabilizes, the camera is immediately triggered to photograph the suspected defect area, acquiring the first test image. Subsequently, the system switches to the second optimized lighting condition, stabilizes the light source again, and triggers the camera to acquire the second test image. This process is repeated until all modes in the light source excitation sequence have been executed, thus obtaining a raw image set containing the optical responses of the suspected defect area under various lighting conditions, i.e., the test image set, as shown below. Figure 3 As shown, Image 1 is a high-angle bright-field image, Image 2 is a low-angle dark-field image, and Image 3 is an obliquely illuminated image. Finally, due to mechanical positioning errors or environmental vibrations that may cause slight displacements between images in the test image set, and the potential for nonlinear changes in the response of the light source or camera, post-processing of the test image set is necessary to generate a standardized multimodal image sequence. This processing includes two steps: image alignment and radiometric calibration. Image alignment aims to eliminate spatial displacement between images. Typically, one image from the test image set is selected as a reference. Algorithms based on feature point matching or phase correlation are used to calculate the geometric transformation relationship of other images relative to the reference image, and these images are resampled to ensure that the defect regions in all images correspond precisely at the pixel level. Radiometric calibration aims to correct brightness inconsistencies. A calibration model is established to correct pixel value deviations caused by light source fluctuations or nonlinear camera responses, ensuring that changes in pixel values ​​between images truly reflect the physical interaction between the defect and light, rather than equipment errors. After image alignment and radiometric calibration, the resulting standardized image sequence is the final multimodal image sequence, where each frame represents the pure optical characteristics of the defect under a specific illumination mode.

[0059] Optionally, extracting physical features from the multimodal image sequence includes:

[0060] For each image in the multimodal image sequence, calculate grayscale statistical features and geometric morphological features to generate a statistical feature set;

[0061] Temporal analysis is performed on the multimodal image sequence to extract the variation features of defects under different lighting conditions and generate a dynamic feature set;

[0062] The statistical feature set and the dynamic feature set are merged to generate physical features.

[0063] Specifically, the first step is to generate a statistical feature set, a process performed on each individual image in the multimodal image sequence. For any image in the sequence, the system first locates the defect region and then calculates its gray-level statistical features and geometric features. Gray-level statistical features include the average gray value of the pixels within the region, used to quantify its overall brightness; the gray-level standard deviation, used to quantify its internal contrast or non-uniformity; and the entropy of the gray-level histogram, used to measure the complexity of the texture. Geometric features describe the shape information of the defect region, such as the area of ​​the region (i.e., the total number of pixels); the perimeter; and the roundness calculated from the area and perimeter, used to determine whether its shape is close to a circle; as well as the principal axis length and eccentricity describing its direction and extent of extension. The above calculations are performed on each image in the sequence, and the resulting series of feature values ​​are combined into a feature vector. The feature vectors of all images together constitute the statistical feature set. The second step is to perform temporal analysis on the multimodal image sequence to extract the variation characteristics of the defect under different lighting conditions, generating a dynamic feature set. This time-series analysis no longer views individual images in isolation, but treats the entire sequence as a whole, focusing on the evolution of specific features under different lighting modes. For example, the system tracks and records the change trajectory of the average gray value of the defect area throughout the image sequence, forming a brightness response curve. Dynamic features are then extracted from this curve, such as the difference between the maximum and minimum values ​​of the brightness variation range, and the variance of the brightness variation. These features reflect the sensitivity of the defect to changes in illumination. Similarly, the system analyzes the stability of geometric features such as the area or roundness of the defect in the sequence, calculating their coefficient of variation to determine whether the appearance of the defect changes significantly with the lighting angle. These features, which quantify the optical response behavior of the defect, together constitute the dynamic feature set. The final step merges the statistical feature set and the dynamic feature set obtained in the first two steps to generate the final physical features. The merging operation typically involves concatenating the individual feature values ​​in the dynamic feature set with all the feature values ​​generated from multiple images in the statistical feature set, forming a single feature vector with higher dimension and more comprehensive information, such as... Figure 4 As shown, this final physical feature vector not only contains static snapshot information of the defect under various single illuminations, but more importantly, it incorporates its dynamic behavioral fingerprint that varies with illumination conditions, thus fully characterizing the physical and optical nature of the defect.

[0064] Optionally, the generated defect filtering results include:

[0065] The physical features are input into a preset logic judgment tree containing optical physics rules for rule matching to generate a preliminary judgment result;

[0066] The confidence level of the preliminary judgment is assessed to generate a confidence score;

[0067] The confidence score is compared with a preset classification threshold to generate a defect filtering result.

[0068] Specifically, the physical feature vector generated in the previous step for the suspected defect area is first input into a pre-defined logical decision tree containing optical physics rules. This logical decision tree is a hierarchical decision model, where each node represents a test of a specific component of the physical feature vector, such as "whether the average brightness of the defect area under dark lighting is greater than a certain set value," or "whether the rate of change of the defect area under different lighting angles is less than a certain threshold." These rules are pre-defined based on optical physics principles and defect formation mechanisms. After the physical feature vector is input, the system starts from the root node of the tree and performs rule matching based on the values ​​of the corresponding feature components in the vector, traversing downwards along branches that meet the conditions until a leaf node is reached. Each leaf node corresponds to a preliminary judgment result, such as "suspected scratch," "suspected foreign object," or "non-defect artifact." Simultaneously with obtaining the preliminary judgment result, the system evaluates its confidence level, generating a quantified confidence score. This confidence score is usually bound to the leaf nodes of the logical decision tree and is pre-defined based on historical data statistics or expert experience; it represents the probability or credibility of the judgment path reaching a correct conclusion. For example, a leaf node reached by a path validated by multiple strong features will have a higher confidence score. The final step is to generate the final defect filtering result based on this confidence score. The system compares the confidence score generated in the previous step with a preset classification threshold. This classification threshold is a key control parameter used to balance recall and precision. If the confidence score is greater than or equal to the classification threshold, the system determines the suspected defect to be a real defect, and the defect filtering result is "retain" or "pass". Conversely, if the confidence score is lower than the threshold, the system considers the detection result unreliable, possibly due to noise, image artifacts, or variations that do not meet the criteria of interest, and determines it to be a non-real defect, with the defect filtering result being "filtered" or "rejected". This final binary result will determine whether the suspected defect enters the subsequent fine classification stage.

[0069] Optionally, the generated defect type classification results include:

[0070] Based on the defect filtering results, a convolution operation is performed on the multimodal image sequence to generate a high-level feature map;

[0071] The high-level feature map is subjected to pooling processing to generate a pooled feature vector;

[0072] The pooled feature vectors are mapped to generate defect type classification results.

[0073] Specifically, using the defect filtering results generated in the previous step as instructions, processing is performed only on defects judged as "retained" and their corresponding multimodal image sequences. First, this multimodal image sequence is fed into a pre-trained convolutional neural network. This sequence can be viewed as a multi-channel tensor, where each channel represents a defect image under a specific lighting pattern. The network performs convolution operations on this input, sliding computations across the image sequence using a series of learnable convolutional kernels, automatically extracting features from low to high levels layer by layer from the original pixel data. This process generates multiple high-level feature maps. These feature maps are no longer simple sets of pixels, but rather abstract and conceptual representations of the texture, shape, brightness response, and other characteristics of defects under different lighting conditions. Subsequently, to reduce the spatial dimensionality of the feature maps, enhance their robustness to small displacements and deformations, and reduce subsequent computation, the system performs pooling processing on these high-level feature maps. Max pooling or average pooling is typically used. This operation summarizes local regions of the feature maps using their maximum or average values, thereby generating smaller but more information-concentrated feature representations. After multiple rounds of convolution and pooling operations, the resulting compact feature representation is flattened to form a one-dimensional pooled feature vector. This vector highly condenses the core information needed to distinguish different defect types. Finally, this pooled feature vector is mapped non-linearly from the high-dimensional feature space to a class score space through one or more fully connected layers. The last layer of this mapping is typically a Softmax function, which transforms the score vector into a probability distribution, calculated as follows:

[0074] ,

[0075] in, This represents the probability that the defect belongs to the i-th predefined defect type. It is the raw score of the i-th type output by the fully connected layer. This is the raw score for the j-th type among all possible defect types. The system ultimately selects the type with the highest probability value as the output defect type classification result, such as "blemishes", "scratches", or "foreign objects".

[0076] Optionally, the method further includes:

[0077] Collect the defect type classification results and corresponding manual verification results to generate a training dataset;

[0078] The feature light source pattern mapping relationship is updated using the training dataset.

[0079] Specifically, the system systematically collects the defect type classification results generated by the aforementioned steps, as well as the parallel manual verification results of the same defect sample by professional quality inspectors. For each detected suspected defect, the system records its automatically generated classification label, and simultaneously acquires and stores an authoritative label given by a human expert, i.e., the "gold standard." By continuously accumulating a large number of such data pairs, a training dataset containing both machine judgment and human ground truth is generated. Each entry in this dataset constitutes a complete case, containing not only the final classification and verification results, but also the complete information chain leading to that classification result, especially the region feature vector initially used to generate the light source excitation sequence. Next, the system uses this newly generated training dataset to update the feature light source pattern mapping relationship.

[0080] Optionally, updating the feature light source pattern mapping relationship using the training dataset includes:

[0081] Multimodal image sequences and corresponding defect labels are extracted from the training dataset to generate sample pairs;

[0082] Based on the classification correctness of the sample pairs, reinforcement learning is performed to update the feature light source pattern mapping relationship, resulting in the updated feature light source pattern mapping relationship.

[0083] Specifically, the method of updating the feature light source pattern mapping relationship using the training dataset is a process of optimizing the system's decision-making strategy using a reinforcement learning framework. First, key information is extracted from the training dataset to construct the environmental interaction samples required for reinforcement learning. For each case in the dataset, the system extracts its initial region feature vector as the state S in reinforcement learning, and the light source excitation sequence dynamically generated and actually executed by the system for this feature vector is considered as the action A taken by the system in state S. Then, by comparing the machine classification result with the human verification result for this case, the system generates a quantified reward signal R. If the classification is correct, the reward R is a positive value; if the classification is incorrect, the reward R is a negative value or zero. Thus, the training dataset is transformed into a series of (S, A, R) triples. Next, based on these triples, the system performs reinforcement learning to update the feature light source pattern mapping relationship, which is its core decision-making strategy. This mapping relationship can be modeled as a policy function or a value function, the goal of which is to select an action, i.e., a light source excitation sequence, that maximizes the expected reward for any given state, i.e., the region feature vector. The update process follows the value iteration or policy gradient principle of reinforcement learning. For example, a value update formula can be used:

[0084] ,

[0085] in, This represents the value estimate of taking action A in state S, i.e., the score related to the mapping relationship of the characteristic light source pattern. It is the value before the update. R is the updated value. R is the actual reward obtained for this action, which comes directly from the classification correctness. The learning rate is a constant between 0 and 1 that controls the impact of a single learning experience on the overall policy. The core idea of ​​this formula is that if an action A in state S achieves better results than expected... A higher reward R increases the value of the (state, action) pair; conversely, a lower reward decreases it. By iteratively updating all samples in the training dataset, the value of each possible mapping in the entire feature light source pattern mapping is adjusted, strengthening mappings that historically led to correct classification and weakening those that resulted in misclassification. Ultimately, the system obtains a data-driven optimized and updated feature light source pattern mapping.

[0086] Based on the same inventive concept, such as Figure 5 As shown, the present invention also provides an automatic LCD defect illumination detection and classification system, the system comprising:

[0087] The suspected defect identification module is used to acquire an initial image of the LCD screen under standard lighting conditions, and to detect the initial image to identify suspected defect areas.

[0088] A light source sequence generation module is used to extract image features of the suspected defect area and dynamically generate a light source excitation sequence based on the image features;

[0089] A multimodal image acquisition module is used to illuminate the suspected defect area according to the light source excitation sequence and acquire a multimodal image sequence;

[0090] The physical feature processing module is used to extract physical features from the multimodal image sequence and process the physical features using a preset logical judgment tree containing optical physical rules to generate defect filtering results.

[0091] The defect type classification module is used to identify the defect type of the multimodal image sequence based on the defect filtering results and generate a defect type classification result.

[0092] To verify the feasibility of this invention in practice, it was applied to an LCD screen production line. During defect detection on this line, the traditional fixed-lighting machine vision system had a high rate of missed detections and false alarms for complex defects such as blemishes and minor scratches, affecting production efficiency and product quality. The method and system of this invention were deployed in the final quality inspection stage before shipment. To verify the effectiveness of this invention, a three-month comparative test was conducted. The experimental group used the method and system proposed in this invention, while the control group used the production line's existing detection system based on fixed lighting and traditional image processing algorithms. During the test, the system inspected thousands of LCD screens and recorded detailed inspection, classification, and manual verification data, such as... Figure 6 As shown.

[0093] In this embodiment, the system of the present invention first performs a preliminary screening of the LCD screen. For example, at 10:30 a.m. on a certain day, the system inspects a screen with batch number PD-S100-240515. The suspected defect identification module acquires an initial image under standard lighting mode, and after grayscale conversion, Gaussian filtering, difference calculation with a defect-free template image, binarization, and connected component analysis, identifies a suspected defect area of ​​approximately 50x30 pixels near the screen coordinates (1024, 768).

[0094] Subsequently, the light source sequence generation module extracts the image features of the region and calculates its texture features, namely the LBP histogram distribution and brightness features (average gray value of 110, standard deviation of 45). Based on the high brightness standard deviation, the preset feature light source mode mapping relationship determines that the defect may be a scattering or absorption type defect, and dynamically generates a light source excitation sequence containing three modes: [1. High-angle bright field illumination; 2. Low-angle dark field illumination; 3. 45-degree oblique illumination].

[0095] Based on this sequence, the multimodal image acquisition module precisely controls the programmable light source to illuminate the suspected defect area three times, simultaneously acquiring images. After image alignment and radiometric calibration, a multimodal image sequence containing three frames is generated. The physical feature processing module extracts physical features from this sequence. Analysis shows that the geometry of the defect, i.e., its area and roundness, remains relatively stable under different lighting conditions, but its grayscale statistical characteristics change significantly: the average brightness drops to 45 under high-angle bright field conditions, while it soars to 210 under low-angle dark field conditions. These static and dynamic features are merged into the final physical feature vector.

[0096] The physical feature vector was input into a logical decision tree containing optical physics rules. The system matched a rule: "If the brightness of a defect is significantly enhanced under dark lighting (greater than the threshold of 200) and significantly reduced under bright lighting (less than the threshold of 50), it is a high-confidence physical defect." This suspected defect received a confidence score of 0.95, higher than the preset classification threshold of 0.7, so its defect filtering result was "retained," and it proceeded to subsequent classification. During this process, another artifact caused by tiny dust particles disappeared under specific angle lighting. Its physical feature change did not conform to any preset rules, resulting in a confidence score of 0.2, which was successfully filtered out, effectively reducing the false alarm rate.

[0097] Finally, the defect type classification module inputs the filtered defects and their multimodal image sequences into a pre-trained convolutional neural network. The network extracts high-level features through convolution and pooling operations, and finally outputs classification probabilities through a Softmax layer: {Scratch: 0.91, Foreign Object: 0.08, Blemish: 0.01}. Based on this, the system generates a defect type classification result of "Scratch". This result is manually verified and confirmed to be correct by production line quality inspectors.

[0098] This invention also achieves self-optimization through a closed-loop learning mechanism. In the initial testing phase, the system misclassified an atypical, weak mottled defect as "normal background" because its initial features were not significant, resulting in a generated light source excitation sequence that failed to effectively highlight the defect. Manual review corrected the result to "mottling." The system collected this misclassification case, using its initial region feature vector, the generated light source excitation sequence, and the misclassification result (i.e., the negative reward) as sample pairs, and updated the feature light source pattern mapping relationship using a reinforcement learning algorithm. After this update, when the system encounters suspected defects with similar initial features in subsequent tests, it tends to generate sequences containing monochromatic light illumination of a specific wavelength, thus more effectively stimulating the mottled defect's response.

[0099] This invention's system, while identifying more suspected defects, effectively reduces false alarms and missed detections through a logic judgment tree based on physical rules. The defect filtering mechanism of this invention can effectively distinguish between real defects and noise artifacts, significantly improving the reliability of initial detection. As testing progresses and training data accumulates, the system's overall classification accuracy improves. Through reinforcement learning, it continuously optimizes its feature light source pattern mapping relationship, enabling its defect detection strategy to possess self-evolution and adaptive capabilities.

[0100] It should be noted that the electrical connections between the various units described above do not necessarily represent direct or indirect connections. Any method of indirect connection is applicable to the embodiments of the present invention as long as it achieves the purpose of the present invention. The above descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the present invention.

[0101] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A method for automatic detection and classification of LCD defects, characterized in that, The method includes: Acquire an initial image of the LCD screen under standard lighting conditions, and inspect the initial image to identify suspected defect areas; Extracting image features from the suspected defective region and dynamically generating a light source excitation sequence based on the image features; including: extracting texture and brightness features from the suspected defective region to generate a region feature vector; processing the region feature vector based on a preset feature light source mode mapping relationship to generate a light source mode recommendation list; and combining and sorting multiple light source modes according to the light source mode recommendation list to generate a light source excitation sequence. According to the light source excitation sequence, the suspected defect area is illuminated to acquire a multimodal image sequence; this includes: adjusting the angle and intensity parameters of the light source according to the light source excitation sequence to generate optimized lighting conditions; illuminating the suspected defect area multiple times based on the optimized lighting conditions and simultaneously triggering image acquisition to obtain a test image set; and performing image alignment and radiometric calibration processing on the test image set to generate a multimodal image sequence. Physical features are extracted from the multimodal image sequence, and processed using a preset logical judgment tree containing optical physical rules to generate defect filtering results. This includes: calculating grayscale statistical features and geometric morphological features for each image in the multimodal image sequence to generate a statistical feature set; performing time-series analysis on the multimodal image sequence to extract the variation features of defects under different lighting conditions to generate a dynamic feature set; merging the statistical feature set and the dynamic feature set to generate physical features; inputting the physical features into a preset logical judgment tree containing optical physical rules for rule matching to generate preliminary judgment results; evaluating the confidence level of the preliminary judgment results to generate a confidence score; and comparing the confidence score with a preset classification threshold to generate defect filtering results. Based on the defect filtering results, defect type identification is performed on the multimodal image sequence to generate defect type classification results; this includes: performing a convolution operation on the multimodal image sequence based on the defect filtering results to generate a high-level feature map; performing pooling processing on the high-level feature map to generate a pooled feature vector; and mapping the pooled feature vector to generate defect type classification results.

2. The method for automatic illumination detection and classification of LCD defects according to claim 1, characterized in that, The identified suspected defect areas include: Acquire an initial image of the LCD screen under standard lighting mode, and perform grayscale processing on the initial image to generate a grayscale image; The grayscale image is filtered and denoised to generate a smooth image; Obtain a preset defect-free template image, and perform a difference calculation between the smoothed image and the defect-free template image to generate a difference image; The difference image is binarized to generate a binary image; Connectivity analysis was performed on the binary image to identify suspected defective regions.

3. The method for automatic illumination detection and classification of LCD defects according to claim 1, characterized in that, The method further includes: Collect the defect type classification results and corresponding manual verification results to generate a training dataset; The feature light source pattern mapping relationship is updated using the training dataset.

4. The method for automatic illumination detection and classification of LCD defects according to claim 3, characterized in that, The step of updating the feature light source pattern mapping relationship using the training dataset includes: Multimodal image sequences and corresponding defect labels are extracted from the training dataset to generate sample pairs; Based on the classification correctness of the sample pairs, reinforcement learning is performed to update the feature light source pattern mapping relationship, resulting in the updated feature light source pattern mapping relationship.

5. An automatic LCD defect detection and classification system, characterized in that, The system includes: The suspected defect identification module is used to acquire an initial image of the LCD screen under standard lighting conditions, and to detect the initial image to identify suspected defect areas. A light source sequence generation module is used to extract image features of the suspected defective region and dynamically generate a light source excitation sequence based on the image features; including: extracting texture and brightness features of the suspected defective region to generate a region feature vector; processing the region feature vector based on a preset feature light source mode mapping relationship to generate a light source mode recommendation list; and combining and sorting multiple light source modes according to the light source mode recommendation list to generate a light source excitation sequence. A multimodal image acquisition module is used to illuminate the suspected defect area according to the light source excitation sequence and acquire a multimodal image sequence. This includes: adjusting the angle and intensity parameters of the light source according to the light source excitation sequence to generate optimized lighting conditions; illuminating the suspected defect area multiple times based on the optimized lighting conditions and simultaneously triggering image acquisition to obtain a test image set; and performing image alignment and radiometric calibration processing on the test image set to generate a multimodal image sequence. A physical feature processing module is used to extract physical features from the multimodal image sequence and process the physical features using a preset logical judgment tree containing optical physical rules to generate defect filtering results. This includes: calculating grayscale statistical features and geometric morphological features for each image in the multimodal image sequence to generate a statistical feature set; performing time-series analysis on the multimodal image sequence to extract the variation features of defects under different lighting conditions to generate a dynamic feature set; merging the statistical feature set and the dynamic feature set to generate physical features; inputting the physical features into a preset logical judgment tree containing optical physical rules for rule matching to generate a preliminary judgment result; evaluating the confidence level of the preliminary judgment result to generate a confidence score; and comparing the confidence score with a preset classification threshold to generate defect filtering results. The defect type classification module is used to identify the defect type of the multimodal image sequence based on the defect filtering result and generate a defect type classification result; including: performing a convolution operation on the multimodal image sequence based on the defect filtering result to generate a high-level feature map; performing pooling processing on the high-level feature map to generate a pooled feature vector; and mapping the pooled feature vector to generate a defect type classification result.

Images