Polarization multispectral fusion OLED defect detection method

Through polarization and multi-spectral fusion technology combined with YoloV8 detection model, the problems of low detection accuracy and incomplete defect types in OLED screen detection are solved, and the detection accuracy and reliability are improved, especially in small sample scenarios.

CN118982500AActive Publication Date: 2024-11-19ZHEJIANG UNIV

Patent Information

Application Number
CN202410976725.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-21
Publication Date
2024-11-19
Estimated Expiration
2044-07-21

AI Technical Summary

Technical Problem

The existing OLED screen defect detection technology has problems such as low detection rate and detection accuracy of subtle defects, incomplete defect types, and small sample size, making the model difficult to train, resulting in poor detection results.

Method used

Using polarization and multi-spectral fusion detection methods, multi-angle and multi-spectral images are collected through polarization cameras and multi-spectral cameras, combined with YoloV8 detection model and attention mechanism, multi-classification training and detection are carried out, and wavelet transformation and SIFT algorithm are used for Image registration and multi-scale decomposition to generate polarized multi-spectral fusion images for defect detection.

Benefits of technology

It significantly improves the detection accuracy of subtle defects and the detection rate of multiple types of defects, enhances the detection ability of the model in small sample scenarios, and improves the reliability of OLED product quality inspection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982500A_ABST
    Figure CN118982500A_ABST
Patent Text Reader

Abstract

The invention discloses a polarization multispectral fusion OLED (Organic Light Emitting Diode) screen defect detection method, which introduces polarization, multispectral and other multi-bit information and intensity information to highlight the characteristics of fine defects to the greatest extent, and fuses an intensity image, a polarization image and a multispectral image to generate a multispectral polarization image by using a method based on multi-scale decomposition. And carrying out multi-classification training and detection on the generated image by using a YoV8 detection model. Aiming at the problems that OLED defect data sets are missing and models are difficult to train, various data enhancement and transfer learning training strategies of transfer learning are introduced, and an attention mechanism is introduced into a YoV8 model for improvement and enhancement, so that the detection capability of the model in a small sample scene is improved. According to the invention, the detection capability of fine defects on the surface of the screen body is improved, the detection rate of various defects on the surface of the screen body is improved, and the detection capability of an OLED screen body detection model under a small sample is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of screen appearance defect detection, and in particular to an OLED screen defect detection method using polarization multi-spectrum fusion. Background Art

[0002] The manufacturing process of OLED screens is complex and involves many processes. Various defects are inevitably introduced at each manufacturing stage. Due to its special material properties and high pixel density, even slight defects may cause the materials to oxidize and be damaged at a faster rate, thus leading to major display problems. Traditional manual inspection solutions have high labor costs, and the inspection results are easily affected by subjective factors. The current automatic optical inspection solutions based on machine learning have the following problems:

[0003] 1. Low detection rate and accuracy of subtle defects: Since the scene information reflected by the intensity image is easily disturbed by factors such as object texture features and complex lighting, the system has poor resolution for subtle defects, and the detection rate and accuracy are limited;

[0004] 2. Incomplete detection types: Due to the rich variety of OLED defects and the varying scales, it is difficult to completely detect defects of different shapes and sizes in one intensity image. The defect recall rate is low, and most of them only detect the presence or absence of defects without further quantitative classification. It is difficult to conduct a review and analysis of the manufacturing process based on the detection results.

[0005] 3. Small sample size and difficult to train: Since there are currently very few public data sets of OLED screen defects and it is difficult to collect a large number of defect samples in industrial production lines, it is difficult to use more robust deep learning models for training. Summary of the invention

[0006] The present invention improves upon the deficiencies in the prior art and designs an OLED screen defect detection method based on polarization and multi-spectral fusion.

[0007] In order to address the limited accuracy of traditional image-based detection methods, multi-bit information such as polarization and multi-spectrum and intensity information are introduced to highlight the characteristics of subtle defects to the greatest extent.

[0008] To address the problem of incomplete detection of categories in a single intensity image, a method based on multi-scale decomposition is used to fuse the intensity image, polarization image and multispectral image to generate a multispectral polarization image, and the YoloV8 detection model is used to perform multi-classification training and detection on the generated image.

[0009] In order to solve the problems of missing OLED defect datasets and difficulty in model training, we introduced a variety of data augmentation and transfer learning training strategies, and introduced the attention mechanism in the YoloV8 model to improve and enhance the model's detection capability in small sample scenarios.

[0010] The present invention is achieved through the following technical solutions:

[0011] The present invention discloses a polarization multi-spectral fusion OLED defect detection method, which is implemented by the following necessary devices:

[0012] Polarization camera: used to collect images of the OLED screen in four different polarization directions;

[0013] Multispectral camera: used to collect spectral images in multiple spectral bands;

[0014] Illumination device: including a polarized light source capable of emitting uniform linearly polarized light, and a multi-spectral light source capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film attached to the luminous surface of the coaxial light source;

[0015] Sample transport device: including a conveyor belt for automatically loading and transporting OLED samples;

[0016] Industrial computer: used to synchronously control the acquisition speed of the image acquisition device and perform image preprocessing, image fusion and defect detection algorithms;

[0017] Light source controller: used to control the switch and brightness of the polarized light source and the multi-spectral light source, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source to be consistent with the acquisition frequency of the multi-spectral camera;

[0018] The light source controller, the conveyor belt, the polarization camera and the multispectral camera are all connected to the industrial computer, the polarization light source and the multispectral light source are all connected to the light source controller, the polarization camera and the polarization light source constitute the first detection point where the OLED sample passes, and the multispectral light source and the multispectral camera constitute the second detection point where the OLED sample passes; the detection method comprises the following steps:

[0019] 1) At the first detection point, the polarization camera is used to collect polarization images of the OLED screen in four different polarization directions (0°, 45°, 90°, and 135°), which are respectively denoted as I 0 , I 45 , I 90 , I 135 ;

[0020] 2) Based on the collected polarization images in four directions, the Stokes parameter S is calculated, and the degree of linear polarization DoLP and the angle of polarization AoP are solved according to the Stokes parameter. Then, the Stokes parameter, DoLP and AoP are mapped to the HSV color space to generate a polarization fusion image;

[0021] 3) When detecting the second point, the multi-spectral camera is used to collect images of the OLED screen in multiple spectral bands, and a multi-spectral fusion image is generated by a weighted average method;

[0022] 4) Based on the SIFT algorithm, the polarization fusion image and the multispectral fusion image are registered to ensure the consistency of the two images in spatial position and resolution;

[0023] 5) Using wavelet transform to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image to obtain sub-images of different scales, and perform fusion processing respectively, and finally generate a polarization multispectral fusion image through inverse transformation;

[0024] 6) Based on the polarization multi-spectral fusion image, the improved YoloV8 algorithm is used for defect detection, and the OLED screen is judged whether it is qualified according to the detection results, and the corresponding judgment results are output.

[0025] As a further improvement, the calculation formula of the Stokes parameter described in the present invention is:

[0026] As a further improvement, the calculation formulas for the degree of linear polarization (DoLP) and the angle of polarization (AoP) described in the present invention are respectively

[0027] As a further improvement, the spectral bands collected by the multi-spectral camera described in the present invention include 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm spectral bands.

[0028] As a further improvement, the registration of polarization fusion images and multispectral fusion images based on SIFT algorithm described in the present invention specifically includes detecting key points, generating feature vectors, matching feature point pairs through SIFT algorithm, and eliminating erroneous matching point pairs using RANSAC algorithm.

[0029] As a further improvement, the present invention describes a polarization-based multi-spectral fusion image, uses the YoloV8 algorithm for defect detection, and judges whether the OLED screen is qualified based on the detection results, and outputs the corresponding judgment results: The YoloV8 algorithm is optimized based on the original model, including the introduction of the CABM (Convolutional Block Attention Module) attention mechanism, and the output of the CABM is fused with the original features to enhance the generalization ability of the model.

[0030] The beneficial effects achieved by the present invention are as follows:

[0031] 1) Improve the ability to detect subtle defects on the screen surface: This invention effectively overcomes the limitation of intensity images being interfered by complex backgrounds by fusing intensity information with polarization multispectral information. By using the polarization and spectral characteristics reflected from the surface of an object, the characteristics of small defects that were originally difficult to identify are highlighted, significantly improving the accuracy of OLED surface defect detection. At the same time, the detection capability of the system is further enhanced, and the reliability of OLED product quality detection is improved.

[0032] 2) Improve the detection rate of multiple types of defects on the screen surface: The present invention introduces a two-stage fusion strategy of polarization fusion and multi-spectral fusion, combined with multi-scale decomposition and image registration technology, to ensure the precise fusion of polarization and multi-spectral information. The image registration technology based on the SIFT algorithm ensures the precise alignment of the image before fusion in space and resolution, providing a high-quality data foundation for subsequent fusion processing. The use of wavelet transform for multi-scale decomposition not only improves the flexibility of information processing, but also helps to capture defect features at different scales. Finally, the YoloV8 detection model is used to perform multi-classification training and prediction on the generated fused image, which improves the system's detection rate for multi-scale and multi-type defects.

[0033] 3) Improve the detection capability of the OLED screen inspection model under small sample conditions: In view of the fact that OLED defect samples are difficult to miss and difficult to train, the model is pre-trained using a variety of data augmentation strategies and transfer learning training strategies. The CABM attention mechanism is introduced into the model architecture, and the output of CABM is fused with the original features to enhance the generalization ability of the model, greatly improving the detection capability of the model in small sample scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the structure of the device required to implement the method;

[0035] Figure 2 It is the polarization multi-spectral fusion detection process.

[0036] In the figure, 1 is a polarization camera, 2 is a coaxial light source, 3 is a polarization film, 4 is a sample stage, 5 is a conveyor belt, 7 is a multispectral light source, 8 is a multispectral camera, 9 is a light source controller, and 10 is an industrial computer. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is further described below by means of specific embodiments in conjunction with the accompanying drawings:

[0038] Figure 1 It is a schematic diagram of the structure of the device required to implement the method; it mainly includes a polarization camera 1, a multi-spectral camera 8, an illumination device, a sample transmission device, an industrial computer 10 and a light source controller 9.

[0039] Sample transport device: including a conveyor belt 5, using a Zhenyu automated multi-station guide rail line group, for automatically loading, placing and transporting OLED samples;

[0040] Polarization camera 1 and multispectral camera 8: Polarization camera 1 uses Hikvision MV-CH050-10UP industrial camera and MVL-MF2528M-8MP industrial camera lens group to collect four-way polarization images; multispectral camera 8 uses Keyence CA-H500MX industrial camera and CA-LH25 industrial camera lens group to collect multi-spectral images.

[0041] The lighting device includes a polarized light source capable of emitting uniform linearly polarized light and a multi-spectral light source 7 capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film 3 attached to the luminous surface of the coaxial light source 2; specifically,

[0042] It includes a Dongguan COAX-200X coaxial light source 2 and a Keyence CA-DRM10X multi-spectral light source 7. The coaxial light source 2 is used to emit coaxial light to the OLED sample and generate reflected light to the field of view of the polarization camera 1; the multi-spectral light source 7 is used to emit multi-spectral ring light to the OLED sample and generate reflected light to the field of view of the multi-spectral camera 8.

[0043] Light source controller 9: A KEYENCE CA-DRM10X light source controller 9 is used to control the switch and brightness of the coaxial light source 2 and the multi-spectral light source 7, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source 7 to be consistent with the acquisition frequency of the multi-spectral camera 8;

[0044] Industrial computer 10: A Dell Inspiron 3020 office unit is used to synchronously control the acquisition speed of the image acquisition device, generate polarization fusion images and multi-spectral fusion images respectively, execute the defect detection algorithm, and post-fuse the two-stage detection results to determine whether the sample is qualified.

[0045] Polarizing film 3: A new material 200mm*200mm linear polarizing film 3 is grown, which matches the coaxial light emitting area and is attached to the coaxial light emitting surface to obtain polarized light.

[0046] The light source controller 9, the conveyor belt 5, the polarization camera 1 and the multi-spectral camera 8 are all connected to the industrial computer 10, the polarization light source and the multi-spectral light source 7 are all connected to the light source controller 9, the polarization camera 1 and the polarization light source constitute the first detection point through which the OLED sample passes, and the multi-spectral light source 7 and the multi-spectral camera 8 constitute the second detection point through which the OLED sample passes. The sample is located on the sample stage 4 placed on the conveyor belt 5. The detection point 1 is used to detect cracks, scratches, foreign matter and broken pieces on the surface of the OLED screen; the detection point is used to detect defects such as bubbles and dirt on the OLED screen.

[0047] The two detection points also include a fixing device for fixing the cameras and light sources in the two points on the main device, wherein the fixing device is connected to the polarization camera 1 and the coaxial light source 2 in point one, and the fixing device is connected to the multispectral camera 8 and the multispectral ring light in point two, to ensure the stability and accuracy of each point during the detection process.

[0048] Figure 2 It is a polarization multispectral fusion detection process, which is mainly divided into two stages of fusion. The first stage is polarization fusion based on four-way polarization images and multispectral fusion based on multi-spectral images. The second stage is polarization and multispectral image fusion based on multi-scale decomposition. The specific detection steps are as follows:

[0049] 1. Polarization image acquisition and processing:

[0050] (1) When detecting the first point, use polarization camera 1 to collect polarization images of the OLED screen at four directions of 0°, 45°, 90° and 135°, denoted as I 0 , I 45 , I 90 , I 135 ;

[0051] (2) According to the calculation formula of Stokes parameters, the Stokes parameters S are solved using the polarization images collected in four directions, where

[0052] (3) The degree of linear polarization (DoLP) and angle of polarization (AoP) are calculated from the Stokes parameters. The calculation formulas are:

[0053] (4) S 0, DoLP and AoP are mapped to the HSV color space to obtain the polarization fused image.

[0054] 2. Multispectral image acquisition and processing:

[0055] (5) When detecting the second point, use the multi-spectral camera 8 to collect images of the OLED screen under seven spectral bands of 405 nm, 457 nm, 527 nm, 600 nm, 660 nm, 730 nm, and 860 nm;

[0056] (6) Perform weighted averaging on the collected multispectral images to generate a multispectral fusion image. The weighted average calculation formula is: where w i is the weight of each spectral image, P i (x, y) is the pixel value of each spectral image at position (x, y).

[0057] 3. Image registration: Due to the differences in acquisition equipment and shooting conditions, before the polarization fusion image and the multispectral fusion image are fused, it is necessary to ensure that the two images are aligned in terms of spatial position and resolution. Therefore, image registration based on the SIFT algorithm is required.

[0058] (7) Feature point detection: SIFT algorithm is applied to polarization fusion image and multispectral fusion image to detect key points (i.e., feature points). These key points are scale invariant and rotation invariant.

[0059] (8) Feature point description: For each detected key point, the gradient direction histogram of the surrounding area is calculated and a feature vector describing the key point is generated.

[0060] (9) Feature point matching: Use Euclidean distance as a measure of similarity between feature vectors to find matching feature point pairs between the polarization fusion image and the multispectral fusion image.

[0061] (10) Transformation model estimation: Based on the matched feature point pairs, the multispectral fusion image is mapped to the spatial position and resolution of the polarization fusion image using affine transformation.

[0062] (11) Image resampling and interpolation: The multispectral fusion image is resampled and interpolated to generate a new image that is aligned with the polarization fusion image in terms of space and resolution.

[0063] (12) Registration optimization and verification: Use the RANSAC (Random Sample Consensus) algorithm to eliminate incorrect matching point pairs and optimize the transformation model.

[0064] 4. Polarization and multispectral image fusion: Use wavelet transform to perform multi-scale decomposition of polarization fusion and multispectral fusion images to obtain sub-images of different scales, which are fused separately and finally inversely transformed to generate polarization multispectral fusion images.

[0065] (13) Image decomposition: Use two-dimensional discrete wavelet transform (2D-DWT) to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image, respectively, and decompose them into low-frequency coefficient I, high-frequency coefficient I, low-frequency coefficient II, and high-frequency coefficient III;

[0066] (14) Fusion operation: For the approximate component (low-frequency information), since it contains the main structure and contour information of the image, weighted averaging can be used for fusion to obtain the fused low-frequency coefficients; for the detail component (high-frequency information), since it contains the texture and edge information of the image, local contrast-based fusion is used to obtain the fused high-frequency coefficients;

[0067] (15) Inverse transformation and reconstruction: The fused approximate component and detail component are reconstructed through inverse two-dimensional discrete wavelet transform (2D-IDWT) to generate the final polarization multispectral fusion image.

[0068] 5. Defect detection and qualification determination:

[0069] (16) Data enhancement: Based on the generated polarization multispectral fusion image, five data enhancement strategies such as translation, rotation, brightness adjustment, random noise addition, and random cropping are used to expand the collected OLED defect dataset by five times;

[0070] (17) Introduction of model attention mechanism: In order to improve the model's feature extraction and detection and classification capabilities in small sample scenarios, the original YoloV8 model was optimized by adding the CABM attention mechanism, and the output of CABM was fused with the original features to enhance the generalization ability of the model;

[0071] (18) Model pre-training: Using the transfer learning training strategy, the improved YoloV8 model is pre-trained on the NEU-DET public dataset to obtain the pre-train model, and then formally trained on the enhanced OLED dataset to obtain the trained detection model.

[0072] (19) Model prediction and result output: The OLED fusion image to be tested is input into the model for defect detection. Based on the test results, it is determined whether the OLED screen is OK or NG, and the corresponding judgment result is output.

[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A polarization multi-spectral fusion OLED defect detection method, characterized in that: The method is implemented by the following means: Polarization camera: used to collect images of the OLED screen in four different polarization directions; Multispectral camera: used to collect spectral images in multiple spectral bands; The lighting device includes a polarized light source capable of emitting uniform linearly polarized light and a multi-spectral light source capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film attached to the light-emitting surface of the coaxial light source; Sample transport device: including a conveyor belt for automatically loading and transporting OLED samples; Industrial computer: used to synchronously control the acquisition speed of the image acquisition device and perform image preprocessing, image fusion and defect detection algorithms; Light source controller: used to control the switch and brightness of the polarized light source and the multi-spectral light source, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source to be consistent with the acquisition frequency of the multi-spectral camera; The light source controller, conveyor belt, polarization camera and multi-spectral camera are all connected to the industrial computer, the polarization light source and multi-spectral light source are all connected to the light source controller, the polarization camera and polarization light source constitute the first detection point through which the OLED sample passes, and the multi-spectral light source and multi-spectral camera constitute the second detection point through which the OLED sample passes; the detection method comprises the following steps: 1) At the first detection point, the polarization camera is used to collect polarization images of the OLED screen in four different polarization directions (0°, 45°, 90°, and 135°), which are respectively recorded as I0, I 45 , I 90 , I 135 ; 2) Based on the collected polarization images in four directions, the Stokes parameter S is calculated, and the degree of linear polarization DoLP and the angle of polarization AoP are solved according to the Stokes parameter. Then, the Stokes parameter, DoLP and AoP are mapped to the HSV color space to generate a polarization fusion image; 3) When detecting the second point, the multi-spectral camera is used to collect images of the OLED screen in multiple spectral bands, and a multi-spectral fusion image is generated by a weighted average method; 4) Based on the SIFT algorithm, the polarization fusion image and the multispectral fusion image are registered to ensure the consistency of the two images in spatial position and resolution; 5) Using wavelet transform to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image to obtain sub-images of different scales, and perform fusion processing respectively, and finally generate a polarization multispectral fusion image through inverse transformation; 6) Based on the polarization multi-spectral fusion image, the improved YoloV8 algorithm is used for defect detection, and the OLED screen is judged whether it is qualified according to the detection results, and the corresponding judgment results are output.

2. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 1, characterized in that: The calculation formula of the Stokes parameter is:

3. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 1 or 2, characterized in that: The calculation formulas for the degree of linear polarization (DoLP) and the angle of polarization (AoP) are respectively 4. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 3 is characterized in that: The spectral bands collected by the multi-spectral camera include 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm spectral bands.

5. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 1, 2 or 4, characterized in that: The registration of the polarization fusion image and the multispectral fusion image based on the SIFT algorithm specifically includes detecting key points, generating feature vectors, matching feature point pairs through the SIFT algorithm, and eliminating erroneous matching point pairs using the RANSAC algorithm.

6. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 5, characterized in that: The polarization-based multispectral fusion image uses the YoloV8 algorithm for defect detection, and judges whether the OLED screen is qualified based on the detection results, and outputs the corresponding judgment results: The YoloV8 algorithm is optimized based on the original model, including the introduction of the CABM (Convolutional BlockAttention Module) attention mechanism, and the output of CABM is fused with the original features to enhance the generalization ability of the model.

Citation Information

Patent Citations

  • Camouflage object identification method and system based on polarized light acquisition and wave band operation

    CN115359357A

  • Target detection method based on snapshot spectrum polarization camera

    CN115731456A

  • Systems and methods for augmentation of sensor systems and imaging systems with polarization

    US20210356572A1

Cited By

  • Digital image target detection method and system based on medical microbiological examination

    CN119693945A

  • A digital image target detection method and system based on medical microbiological inspection

    CN119693945B

  • Liquid crystal display screen defect detection method

    CN119827522A

  • Screen risk assessment method and device based on self-built data set, equipment and medium

    CN120318623A

  • Qualitative detection method and device for tiny linear defects on surface of magnetic shoe and medium

    CN120635097A