A polarization-multispectral fusion OLED defect detection method

Abstract

The present invention discloses a polarization multi-spectral fusion OLED screen defect detection method, which introduces multi-bit information such as polarization and multi-spectral information and intensity information to highlight the characteristics of subtle defects to the greatest extent, uses a method based on multi-scale decomposition to fuse the intensity image, polarization image and multi-spectral image to generate a multi-spectral polarization image, and uses the YoloV8 detection model to perform multi-classification training and detection on the generated image. In response to the problem that the OLED defect data set is missing and the model is difficult to train, the present invention improves the detection capability of the model in small sample scenarios by introducing a training strategy of transfer learning of multiple data enhancements and transfer learning, and introduces an attention mechanism into the YoloV8 model for improvement and enhancement. The present invention improves the detection capability of the model in small sample scenarios. The present invention improves the detection capability of subtle defects on the surface of the screen, improves the detection rate of multiple types of defects on the surface of the screen, and improves the detection capability of the OLED screen detection model under small samples.

Description

Technical Field

[0001] The present invention relates to the technical field of screen appearance defect detection, and specifically, to an OLED screen defect detection method using polarization multi-spectral fusion. Background Art

[0002] The manufacturing process of OLED screens is complex and involves numerous steps. Various defects are inevitably introduced at each stage of manufacturing. Due to the unique material properties and high pixel density, even subtle defects can cause the materials to oxidize and deteriorate at a rapid rate, leading to major display issues. Traditional manual inspection solutions are labor-intensive and easily influenced by subjective factors. Current automated optical inspection solutions based on machine learning face the following challenges:

[0003] 1. Low detection rate and accuracy of subtle defects: Since the scene information reflected by the intensity image is easily affected by factors such as object texture characteristics and complex lighting, the system has poor resolution for subtle defects, limiting the detection rate and accuracy.

[0004] 2. Incomplete detection types: Due to the rich variety of OLED defects and their varying scales, it is difficult to fully detect defects of varying shapes and sizes in a single intensity image. This results in a low defect recall rate. Furthermore, most detection methods only detect the presence of defects without further quantitative classification, making it difficult to conduct a review and analysis of the manufacturing process based on the detection results.

[0005] 3. Small sample size and difficult to train: Since there are currently very few public datasets on OLED screen defects and it is difficult to collect a large number of defect samples in industrial production lines, it is difficult to use more robust deep learning models for training. Summary of the Invention

[0006] The present invention improves upon the shortcomings of the prior art and designs an OLED screen defect detection method based on polarization and multi-spectral fusion.

[0007] To address the limited accuracy of traditional image-based detection methods, multi-bit information and intensity information such as polarization and multispectral are introduced to highlight the characteristics of subtle defects to the greatest extent.

[0008] To address the problem of incomplete detection of categories in a single intensity image, a method based on multi-scale decomposition is used to fuse the intensity image, polarization image and multispectral image to generate a multispectral polarization image, and the YoloV8 detection model is used to perform multi-classification training and detection on the generated image.

[0009] To address the problems of missing OLED defect datasets and difficulty in model training, we introduced a variety of data augmentation and transfer learning training strategies, and introduced an attention mechanism in the YoloV8 model to improve and enhance the model's detection capabilities in small sample scenarios.

[0010] The present invention is achieved through the following technical solutions:

[0011] The present invention discloses a polarization multi-spectral fusion OLED defect detection method, which is implemented by the following necessary devices:

[0012] Polarization camera: used to capture images of the OLED screen in four different polarization directions;

[0013] Multispectral camera: used to collect spectral images in multiple spectral bands;

[0014] Illumination device: including a polarized light source capable of emitting uniform linearly polarized light, and a multi-spectral light source capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film attached to the luminous surface of the coaxial light source;

[0015] Sample transport device: including a conveyor belt for automatically loading and transporting OLED samples;

[0016] Industrial computer: used to synchronously control the acquisition speed of the image acquisition device and perform image preprocessing, image fusion and defect detection algorithms;

[0017] Light source controller: used to control the switching and brightness of the polarized light source and multispectral light source, and to control the multispectral ring light emission frequency emitted by the multispectral light source to be consistent with the multispectral camera acquisition frequency;

[0018] The light source controller, conveyor belt, polarization camera, and multispectral camera are all connected to the industrial computer. The polarized light source and multispectral light source are both connected to the light source controller. The polarization camera and polarized light source constitute the first detection point where the OLED sample passes, and the multispectral light source and multispectral camera constitute the second detection point where the OLED sample passes. The detection method includes the following steps:

[0019] 1) At the first detection point, the polarization camera is used to collect polarization images of the OLED screen at four different polarization directions (0°, 45°, 90°, 135°), which are respectively recorded as I0, I 45 , I 90 , I 135 ;

[0020] 2) Based on the collected polarization images in the four directions, the Stokes parameter S is calculated, and the degree of linear polarization DoLP and angle of polarization AoP are solved according to the Stokes parameter. The Stokes parameter, DoLP, and AoP are then mapped to the HSV color space to generate a polarization fusion image.

[0021] 3) At the second detection point, a multispectral camera is used to capture images of the OLED screen in multiple spectral bands, and a multispectral fusion image is generated using a weighted average method;

[0022] 4) Register the polarization fusion image and the multispectral fusion image based on the SIFT algorithm to ensure the consistency of the two images in spatial position and resolution;

[0023] 5) Using wavelet transform to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image to obtain sub-images of different scales, and fuse them separately. Finally, the polarization multispectral fusion image is generated through inverse transformation.

[0024] 6) Based on the polarization multispectral fusion image, the improved YoloV8 algorithm is used for defect detection, and the OLED screen is judged whether it is qualified according to the detection results, and the corresponding judgment results are output.

[0025] As a further improvement, the calculation formula of the Stokes parameter described in the present invention is:

[0026] As a further improvement, the calculation formulas for the degree of linear polarization (DoLP) and angle of polarization (AoP) described in the present invention are respectively

[0027] As a further improvement, the spectral bands collected by the multispectral camera described in the present invention include 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm spectral bands.

[0028] As a further improvement, the present invention describes the registration of polarization fusion images and multispectral fusion images based on the SIFT algorithm, which specifically detects key points, generates feature vectors, matches feature point pairs, and uses the RANSAC algorithm to eliminate erroneous matching point pairs.

[0029] As a further improvement, the present invention describes a method for defect detection based on polarization multispectral fusion images, using the YoloV8 algorithm, and judging whether the OLED screen is qualified based on the detection results, and outputting the corresponding judgment results. Specifically, the YoloV8 algorithm is optimized based on the original model, including the introduction of the CABM (Convolutional Block Attention Module) attention mechanism, and fusing the output of CABM with the original features to enhance the generalization ability of the model.

[0030] The beneficial effects achieved by the present invention are as follows:

[0031] 1) Improved detection of subtle surface defects: This invention effectively overcomes the limitations of intensity images, which are often affected by complex background interference, by fusing intensity information with polarization multispectral information. By leveraging the polarization and spectral characteristics of surface reflections, the system highlights previously difficult-to-identify small defects, significantly improving the accuracy of OLED surface defect detection. This also further enhances the system's detection capabilities and improves the reliability of OLED product quality testing.

[0032] 2) Improve the detection rate of multiple types of defects on the screen surface: This invention introduces a two-stage fusion strategy of polarization fusion and multispectral fusion, combined with multi-scale decomposition and image registration technology, to ensure the precise fusion of polarization and multispectral information. The image registration technology based on the SIFT algorithm ensures the precise alignment of the image before fusion in space and resolution, providing a high-quality data foundation for subsequent fusion processing. The use of wavelet transform for multi-scale decomposition not only improves the flexibility of information processing, but also helps to capture defect features at different scales. Finally, the YoloV8 detection model is used to perform multi-classification training and prediction on the generated fusion image, which improves the system's detection rate for multi-scale and multi-type defects.

[0033] 3) Improving the detection capability of the OLED screen inspection model under small sample conditions: To address the problem that OLED defect samples are difficult to miss and difficult to train, the model is pre-trained using multiple data augmentation strategies and transfer learning training strategies. The CABM attention mechanism is introduced into the model architecture, and the output of CABM is fused with the original features to enhance the generalization ability of the model, greatly improving the model's detection capability in small sample scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural diagram of the device required to implement this method;

[0035] Figure 2 It is the polarization multi-spectral fusion detection process.

[0036] In the figure, 1 is a polarization camera, 2 is a coaxial light source, 3 is a polarization film, 4 is a sample stage, 5 is a conveyor belt, 7 is a multispectral light source, 8 is a multispectral camera, 9 is a light source controller, and 10 is an industrial computer. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0038] Figure 1 It is a structural diagram of the device required to implement this method; it mainly includes a polarization camera 1, a multispectral camera 8, an illumination device, a sample transmission device, an industrial computer 10 and a light source controller 9.

[0039] Sample transport device: including a conveyor belt 5, which uses a Zhenyu automated multi-station guide rail line group for automatic loading, placement and transport of OLED samples;

[0040] Polarization camera 1 and multispectral camera 8: Polarization camera 1 uses a Hikvision MV-CH050-10UP industrial camera and MVL-MF2528M-8MP industrial camera lens set to capture four-axis polarization images; multispectral camera 8 uses a Keyence CA-H500MX industrial camera and CA-LH25 industrial camera lens set to capture multispectral images.

[0041] Illumination device: including a polarized light source capable of emitting uniform linearly polarized light, and a multi-spectral light source 7 capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film 3 attached to the luminous surface of the coaxial light source 2; specifically

[0042] It includes a Dongguan COAX-200X coaxial light source 2 and a Keyence CA-DRM10X multispectral light source 7. The coaxial light source 2 is used to emit coaxial light onto the OLED sample and generate reflected light into the field of view of the polarization camera 1; the multispectral light source 7 is used to emit multispectral ring light onto the OLED sample and generate reflected light into the field of view of the multispectral camera 8.

[0043] Light source controller 9: A KEYENCE CA-DRM10X light source controller 9 is used to control the switching and brightness of the coaxial light source 2 and the multispectral light source 7, and to control the emission frequency of the multispectral ring light emitted by the multispectral light source 7 to be consistent with the acquisition frequency of the multispectral camera 8;

[0044] Industrial computer 10: A Dell Inspiron 3020 office computer is used to synchronously control the acquisition speed of the image acquisition device, generate polarization fusion images and multispectral fusion images respectively, execute the defect detection algorithm, and post-fuse the two-stage detection results to determine whether the sample is qualified.

[0045] Polarizing film 3: A new material 200mm*200mm linear polarizing film 3 is grown to match the coaxial light emitting area and is attached to the coaxial light emitting surface to obtain polarized light.

[0046] The light source controller 9, conveyor belt 5, polarization camera 1, and multispectral camera 8 are all connected to an industrial computer 10. The polarized light source and multispectral light source 7 are both connected to the light source controller 9. The polarization camera 1 and polarized light source constitute the first inspection point for the OLED sample, while the multispectral light source 7 and multispectral camera 8 constitute the second inspection point for the OLED sample. The sample is placed on a sample stage 4 on the conveyor belt 5. Inspection point 1 is used to detect cracks, scratches, foreign matter, and broken fragments on the OLED screen surface; inspection point 2 is used to detect defects such as bubbles and dirt on the OLED screen.

[0047] The two detection points also include a fixing device for fixing the cameras and light sources at the two points on the main device, wherein the fixing device is connected to the polarization camera 1 and coaxial light source 2 at point one, and the fixing device is connected to the multispectral camera 8 and multispectral ring light at point two, to ensure the stability and accuracy of each point during the detection process.

[0048] Figure 2 The polarization multispectral fusion detection process is mainly divided into two stages of fusion. The first stage is the polarization fusion based on four-way polarization images and the multispectral fusion based on multi-spectral images. The second stage is the polarization and multispectral image fusion based on multi-scale decomposition. The specific detection steps are as follows:

[0049] 1. Polarization image acquisition and processing:

[0050] (1) When detecting the first point, polarization camera 1 is used to collect polarization images of the OLED screen at four directions of 0°, 45°, 90° and 135°, which are recorded as I0, I 45 , I 90 , I 135 ;

[0051] (2) According to the calculation formula of Stokes parameter, the Stokes parameter S is solved using the polarization images collected in four directions, where

[0052] (3) The degree of linear polarization (DoLP) and angle of polarization (AoP) are calculated from the Stokes parameters. The calculation formulas are:

[0053] (4) Map S0, DoLP, and AoP to the HSV color space to obtain a polarization fusion image.

[0054] 2. Multispectral image acquisition and processing:

[0055] (5) When detecting the second point, use the multispectral camera 8 to collect images of the OLED screen under seven spectral bands of 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm;

[0056] (6) Perform weighted averaging on the collected multispectral images to generate a multispectral fusion image. The weighted average calculation formula is: where w i is the weight of each spectral image, P i (x,y) is the pixel value of each spectral image at position (x,y).

[0057] 3. Image Registration: Due to differences in acquisition equipment and shooting conditions, before fusing the polarization-fused image and the multispectral-fused image, it is necessary to ensure that the two images are aligned in terms of spatial position and resolution. Therefore, image registration based on the SIFT algorithm is required.

[0058] (7) Feature point detection: SIFT algorithm is applied to polarization fusion image and multispectral fusion image respectively to detect key points (i.e., feature points). These key points are scale invariant and rotation invariant.

[0059] (8) Feature point description: For each detected key point, the gradient direction histogram of the surrounding area is calculated and a feature vector describing the key point is generated.

[0060] (9) Feature point matching: Using Euclidean distance as a measure of similarity between feature vectors, matching feature point pairs are found between the polarization fusion image and the multispectral fusion image.

[0061] (10) Transformation model estimation: Based on the matched feature point pairs, the multispectral fusion image is mapped to the spatial position and resolution of the polarization fusion image using affine transformation.

[0062] (11) Image resampling and interpolation: The multispectral fusion image is resampled and interpolated to generate a new image that is aligned with the polarization fusion image in terms of space and resolution.

[0063] (12) Registration optimization and verification: Use the RANSAC (Random Sample Consensus) algorithm to eliminate incorrect matching point pairs and optimize the transformation model.

[0064] 4. Polarization and multispectral image fusion: Use wavelet transform to perform multi-scale decomposition of polarization fusion and multispectral fusion images to obtain sub-images of different scales, which are fused separately and then inversely transformed to generate polarization multispectral fusion images.

[0065] (13) Image decomposition: Two-dimensional discrete wavelet transform (2D-DWT) is used to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image, respectively, and decompose them into low-frequency coefficient I, high-frequency coefficient I, low-frequency coefficient II, and high-frequency coefficient III;

[0066] (14) Fusion operation: For the approximate component (low-frequency information), since it contains the main structure and contour information of the image, weighted averaging can be used for fusion to obtain the fused low-frequency coefficients; for the detail component (high-frequency information), since it contains the texture and edge information of the image, local contrast-based fusion is used to obtain the fused high-frequency coefficients;

[0067] (15) Inverse transformation and reconstruction: The fused approximate component and detail component are reconstructed through inverse two-dimensional discrete wavelet transform (2D-IDWT) to generate the final polarization multispectral fusion image.

[0068] 5. Defect detection and qualification determination:

[0069] (16) Data enhancement: Based on the generated polarization multispectral fusion image, five data enhancement strategies including translation, rotation, brightness adjustment, random noise addition, and random cropping are used to expand the collected OLED defect dataset fivefold;

[0070] (17) Introduction of model attention mechanism: In order to improve the model's feature extraction and detection and classification capabilities in small sample scenarios, the original YoloV8 model was optimized by adding the CABM attention mechanism and fusing the CABM output with the original features to enhance the model's generalization ability;

[0071] (18) Model pre-training: Using the transfer learning training strategy, the improved YoloV8 model is pre-trained on the NEU-DET public dataset to obtain a pre-train model, and then formally trained on the enhanced OLED dataset to obtain a trained detection model.

[0072] (19) Model prediction and result output: The OLED fusion image to be tested is input into the model for defect detection. Based on the test results, the OLED screen is judged to be OK or NG, and the corresponding judgment result is output.

[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A polarization multi-spectral fusion OLED screen defect detection method, characterized in that: The method is implemented by the following means: Polarization camera: used to capture images of the OLED screen in four different polarization directions; Multispectral camera: used to collect spectral images in multiple spectral bands; The lighting device includes a polarized light source capable of emitting uniform linearly polarized light and a multi-spectral light source capable of emitting multi-spectral light. The polarized light source is formed by attaching a polarizing film to the luminous surface of the coaxial light source. Sample transport device: including a conveyor belt for automatically loading and transporting OLED samples; Industrial computer: used to synchronously control the acquisition speed of the image acquisition device and perform image preprocessing, image fusion and defect detection algorithms; Light source controller: used to control the switching and brightness of the polarized light source and multispectral light source, and to control the multispectral ring light emission frequency emitted by the multispectral light source to be consistent with the multispectral camera acquisition frequency; The light source controller, conveyor belt, polarization camera, and multispectral camera are all connected to an industrial computer. The polarized light source and multispectral light source are both connected to the light source controller. The polarization camera and polarized light source constitute the first detection point where the OLED sample passes, and the multispectral light source and multispectral camera constitute the second detection point where the OLED sample passes. The detection method comprises the following steps: 1) At the first detection point, the polarization camera is used to collect polarization images of the OLED screen at four different polarization directions of 0°, 45°, 90°, and 135°, which are recorded as , , , ; 2) Based on the collected polarization images in four directions, the Stokes parameter S is calculated, and the degree of linear polarization (DoLP) and angle of polarization (AoP) are solved based on the Stokes parameter. The Stokes parameter, DoLP, and AoP are then mapped to the HSV color space to generate a polarization fused image. 3) At the second detection point, a multispectral camera is used to capture images of the OLED screen in multiple spectral bands, and a multispectral fusion image is generated using a weighted average method; 4) Register the polarization fusion image and the multispectral fusion image based on the SIFT algorithm to ensure the consistency of the two images in spatial position and resolution; 5) Use wavelet transform to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image to obtain sub-images of different scales, which are fused separately. Finally, the polarization multispectral fusion image is generated through inverse transformation. 6) Based on the polarization multispectral fusion image, the improved YoloV8 algorithm is used for defect detection, and the OLED screen is judged to be qualified according to the detection results, and the corresponding judgment results are output.

2. The OLED screen defect detection method based on polarization multi-spectral fusion according to claim 1 is characterized in that: The calculation formula of the Stokes parameter is: .

3. The OLED screen defect detection method based on polarization multi-spectral fusion according to claim 1 or 2, characterized in that: The calculation formulas for the degree of linear polarization DoLP and the angle of polarization AoP are respectively , .

4. The OLED screen defect detection method based on polarization multi-spectral fusion according to claim 3 is characterized in that: The spectrum segments collected by the multispectral camera include 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm spectrum segments.

5. The OLED screen defect detection method based on polarization multi-spectral fusion according to claim 1, 2 or 4, characterized in that: The registration of the polarization fusion image and the multispectral fusion image based on the SIFT algorithm specifically includes detecting key points, generating feature vectors, matching feature point pairs, and using the RANSAC algorithm to eliminate erroneous matching point pairs.

6. The method for detecting defects in OLED screens using polarization and multi-spectral fusion according to claim 5, wherein: The polarization-based multispectral fusion image uses the YoloV8 algorithm for defect detection, and judges whether the OLED screen is qualified based on the detection results, and outputs the corresponding judgment results. Specifically, the YoloV8 algorithm is optimized based on the original model, including the introduction of the CABM attention mechanism, and the fusion of the CABM output with the original features to enhance the generalization ability of the model.

Images