Polarization multispectral fusion OLED defect detection method
Through polarization and multi-spectral fusion technology combined with YoloV8 detection model, the problems of low detection accuracy and incomplete defect types in OLED screen detection are solved, and the detection accuracy and reliability are improved, especially in small sample scenarios.
Patent Information
- Application Number
- CN202410976725.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-21
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-21
AI Technical Summary
The existing OLED screen defect detection technology has problems such as low detection rate and detection accuracy of subtle defects, incomplete defect types, and small sample size, making the model difficult to train, resulting in poor detection results.
Using polarization and multi-spectral fusion detection methods, multi-angle and multi-spectral images are collected through polarization cameras and multi-spectral cameras, combined with YoloV8 detection model and attention mechanism, multi-classification training and detection are carried out, and wavelet transformation and SIFT algorithm are used for Image registration and multi-scale decomposition to generate polarized multi-spectral fusion images for defect detection.
It significantly improves the detection accuracy of subtle defects and the detection rate of multiple types of defects, enhances the detection ability of the model in small sample scenarios, and improves the reliability of OLED product quality inspection.
Smart Images

Figure CN118982500A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of screen appearance defect detection, and in particular to an OLED screen defect detection method using polarization multi-spectrum fusion. Background Art
[0002] The manufacturing process of OLED screens is complex and involves many processes. Various defects are inevitably introduced at each manufacturing stage. Due to its special material properties and high pixel density, even slight defects may cause the materials to oxidize and be damaged at a faster rate, thus leading to major display problems. Traditional manual inspection solutions have high labor costs, and the inspection results are easily affected by subjective factors. The current automatic optical inspection solutions based on machine learning have the following problems:
[0003] 1. Low detection rate and accuracy of subtle defects: Since the scene information reflected by the intensity image is easily disturbed by factors such as object texture features and complex lighting, the system has poor resolution for subtle defects, and the detection rate and accuracy are limited;
[0004] 2. Incomplete detection types: Due to the rich variety of OLED defects and the varying scales, it is difficult to completely detect defects of different shapes and sizes in one intensity image. The defect recall rate is low, and most of them only detect the presence or absence of defects without further quantitative classification. It is difficult to conduct a review and analysis of the manufacturing process based on the detection results.
[0005] 3. Small sample size and difficult to train: Since there are currently very few public data sets of OLED screen defects and it is difficult to collect a large number of defect samples in industrial production lines, it is difficult to use more robust deep learning models for training. Summary of the invention
[0006] The present invention improves upon the deficiencies in the prior art and designs an OLED screen defect detection method based on polarization and multi-spectral fusion.
[0007] In order to address the limited accuracy of traditional image-based detection methods, multi-bit information such as polarization and multi-spectrum and intensity information are introduced to highlight the characteristics of subtle defects to the greatest extent.
[0008] To address the problem of incomplete detection of categories in a single intensity image, a method based on multi-scale decomposition is used to fuse the intensity image, polarization image and multispectral image to generate a multispectral polarization image, and the YoloV8 detection model is used to perform multi-classification training and detection on the generated image.
[0009] In order to solve the problems of missing OLED defect datasets and difficulty in model training, we introduced a variety of data augmentation and transfer learning training strategies, and introduced the attention mechanism in the YoloV8 model to improve and enhance the model's detection capability in small sample scenarios.
[0010] The present invention is achieved through the following technical solutions:
[0011] The present invention discloses a polarization multi-spectral fusion OLED defect detection method, which is implemented by the following necessary devices:
[0012] Polarization camera: used to collect images of the OLED screen in four different polarization directions;
[0013] Multispectral camera: used to collect spectral images in multiple spectral bands;
[0014] Illumination device: including a polarized light source capable of emitting uniform linearly polarized light, and a multi-spectral light source capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film attached to the luminous surface of the coaxial light source;
[0015] Sample transport device: including a conveyor belt for automatically loading and transporting OLED samples;
[0016] Industrial computer: used to synchronously control the acquisition speed of the image acquisition device and perform image preprocessing, image fusion and defect detection algorithms;
[0017] Light source controller: used to control the switch and brightness of the polarized light source and the multi-spectral light source, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source to be consistent with the acquisition frequency of the multi-spectral camera;
[0018] The light source controller, the conveyor belt, the polarization camera and the multispectral camera are all connected to the industrial computer, the polarization light source and the multispectral light source are all connected to the light source controller, the polarization camera and the polarization light source constitute the first detection point where the OLED sample passes, and the multispectral light source and the multispectral camera constitute the second detection point where the OLED sample passes; the detection method comprises the following steps:
[0019] 1) At the first detection point, the polarization camera is used to collect polarization images of the OLED screen in four different polarization directions (0°, 45°, 90°, and 135°), which are respectively denoted as I 0 , I 45 , I 90 , I 135 ;
[0020] 2) Based on the collected polarization images in four directions, the Stokes parameter S is calculated, and the degree of linear polarization DoLP and the angle of polarization AoP are solved according to the Stokes parameter. Then, the Stokes parameter, DoLP and AoP are mapped to the HSV color space to generate a polarization fusion image;
[0021] 3) When detecting the second point, the multi-spectral camera is used to collect images of the OLED screen in multiple spectral bands, and a multi-spectral fusion image is generated by a weighted average method;
[0022] 4) Based on the SIFT algorithm, the polarization fusion image and the multispectral fusion image are registered to ensure the consistency of the two images in spatial position and resolution;
[0023] 5) Using wavelet transform to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image to obtain sub-images of different scales, and perform fusion processing respectively, and finally generate a polarization multispectral fusion image through inverse transformation;
[0024] 6) Based on the polarization multi-spectral fusion image, the improved YoloV8 algorithm is used for defect detection, and the OLED screen is judged whether it is qualified according to the detection results, and the corresponding judgment results are output.
[0025] As a further improvement, the calculation formula of the Stokes parameter described in the present invention is:
[0026] As a further improvement, the calculation formulas for the degree of linear polarization (DoLP) and the angle of polarization (AoP) described in the present invention are respectively
[0027] As a further improvement, the spectral bands collected by the multi-spectral camera described in the present invention include 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm spectral bands.
[0028] As a further improvement, the registration of polarization fusion images and multispectral fusion images based on SIFT algorithm described in the present invention specifically includes detecting key points, generating feature vectors, matching feature point pairs through SIFT algorithm, and eliminating erroneous matching point pairs using RANSAC algorithm.
[0029] As a further improvement, the present invention describes a polarization-based multi-spectral fusion image, uses the YoloV8 algorithm for defect detection, and judges whether the OLED screen is qualified based on the detection results, and outputs the corresponding judgment results: The YoloV8 algorithm is optimized based on the original model, including the introduction of the CABM (Convolutional Block Attention Module) attention mechanism, and the output of the CABM is fused with the original features to enhance the generalization ability of the model.
[0030] The beneficial effects achieved by the present invention are as follows:
[0031] 1) Improve the ability to detect subtle defects on the screen surface: This invention effectively overcomes the limitation of intensity images being interfered by complex backgrounds by fusing intensity information with polarization multispectral information. By using the polarization and spectral characteristics reflected from the surface of an object, the characteristics of small defects that were originally difficult to identify are highlighted, significantly improving the accuracy of OLED surface defect detection. At the same time, the detection capability of the system is further enhanced, and the reliability of OLED product quality detection is improved.
[0032] 2) Improve the detection rate of multiple types of defects on the screen surface: The present invention introduces a two-stage fusion strategy of polarization fusion and multi-spectral fusion, combined with multi-scale decomposition and image registration technology, to ensure the precise fusion of polarization and multi-spectral information. The image registration technology based on the SIFT algorithm ensures the precise alignment of the image before fusion in space and resolution, providing a high-quality data foundation for subsequent fusion processing. The use of wavelet transform for multi-scale decomposition not only improves the flexibility of information processing, but also helps to capture defect features at different scales. Finally, the YoloV8 detection model is used to perform multi-classification training and prediction on the generated fused image, which improves the system's detection rate for multi-scale and multi-type defects.
[0033] 3) Improve the detection capability of the OLED screen inspection model under small sample conditions: In view of the fact that OLED defect samples are difficult to miss and difficult to train, the model is pre-trained using a variety of data augmentation strategies and transfer learning training strategies. The CABM attention mechanism is introduced into the model architecture, and the output of CABM is fused with the original features to enhance the generalization ability of the model, greatly improving the detection capability of the model in small sample scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a schematic diagram of the structure of the device required to implement the method;
[0035] Figure 2 It is the polarization multi-spectral fusion detection process.
[0036] In the figure, 1 is a polarization camera, 2 is a coaxial light source, 3 is a polarization film, 4 is a sample stage, 5 is a conveyor belt, 7 is a multispectral light source, 8 is a multispectral camera, 9 is a light source controller, and 10 is an industrial computer. DETAILED DESCRIPTION
[0037] The technical solution of the present invention is further described below by means of specific embodiments in conjunction with the accompanying drawings:
[0038] Figure 1 It is a schematic diagram of the structure of the device required to implement the method; it mainly includes a polarization camera 1, a multi-spectral camera 8, an illumination device, a sample transmission device, an industrial computer 10 and a light source controller 9.
[0039] Sample transport device: including a conveyor belt 5, using a Zhenyu automated multi-station guide rail line group, for automatically loading, placing and transporting OLED samples;
[0040] Polarization camera 1 and multispectral camera 8: Polarization camera 1 uses Hikvision MV-CH050-10UP industrial camera and MVL-MF2528M-8MP industrial camera lens group to collect four-way polarization images; multispectral camera 8 uses Keyence CA-H500MX industrial camera and CA-LH25 industrial camera lens group to collect multi-spectral images.
[0041] The lighting device includes a polarized light source capable of emitting uniform linearly polarized light and a multi-spectral light source 7 capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film 3 attached to the luminous surface of the coaxial light source 2; specifically,
[0042] It includes a Dongguan COAX-200X coaxial light source 2 and a Keyence CA-DRM10X multi-spectral light source 7. The coaxial light source 2 is used to emit coaxial light to the OLED sample and generate reflected light to the field of view of the polarization camera 1; the multi-spectral light source 7 is used to emit multi-spectral ring light to the OLED sample and generate reflected light to the field of view of the multi-spectral camera 8.
[0043] Light source controller 9: A KEYENCE CA-DRM10X light source controller 9 is used to control the switch and brightness of the coaxial light source 2 and the multi-spectral light source 7, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source 7 to be consistent with the acquisition frequency of the multi-spectral camera 8;
[0044] Industrial computer 10: A Dell Inspiron 3020 office unit is used to synchronously control the acquisition speed of the image acquisition device, generate polarization fusion images and multi-spectral fusion images respectively, execute the defect detection algorithm, and post-fuse the two-stage detection results to determine whether the sample is qualified.
[0045] Polarizing film 3: A new material 200mm*200mm linear polarizing film 3 is grown, which matches the coaxial light emitting area and is attached to the coaxial light emitting surface to obtain polarized light.
[0046] The light source controller 9, the conveyor belt 5, the polarization camera 1 and the multi-spectral camera 8 are all connected to the industrial computer 10, the polarization light source and the multi-spectral light source 7 are all connected to the light source controller 9, the polarization camera 1 and the polarization light source constitute the first detection point through which the OLED sample passes, and the multi-spectral light source 7 and the multi-spectral camera 8 constitute the second detection point through which the OLED sample passes. The sample is located on the sample stage 4 placed on the conveyor belt 5. The detection point 1 is used to detect cracks, scratches, foreign matter and broken pieces on the surface of the OLED screen; the detection point is used to detect defects such as bubbles and dirt on the OLED screen.
[0047] The two detection points also include a fixing device for fixing the cameras and light sources in the two points on the main device, wherein the fixing device is connected to the polarization camera 1 and the coaxial light source 2 in point one, and the fixing device is connected to the multispectral camera 8 and the multispectral ring light in point two, to ensure the stability and accuracy of each point during the detection process.
[0048] Figure 2 It is a polarization multispectral fusion detection process, which is mainly divided into two stages of fusion. The first stage is polarization fusion based on four-way polarization images and multispectral fusion based on multi-spectral images. The second stage is polarization and multispectral image fusion based on multi-scale decomposition. The specific detection steps are as follows:
[0049] 1. Polarization image acquisition and processing:
[0050] (1) When detecting the first point, use polarization camera 1 to collect polarization images of the OLED screen at four directions of 0°, 45°, 90° and 135°, denoted as I 0 , I 45 , I 90 , I 135 ;
[0051] (2) According to the calculation formula of Stokes parameters, the Stokes parameters S are solved using the polarization images collected in four directions, where
[0052] (3) The degree of linear polarization (DoLP) and angle of polarization (AoP) are calculated from the Stokes parameters. The calculation formulas are:
[0053] (4) S 0, DoLP and AoP are mapped to the HSV color space to obtain the polarization fused image.
[0054] 2. Multispectral image acquisition and processing:
[0055] (5) When detecting the second point, use the multi-spectral camera 8 to collect images of the OLED screen under seven spectral bands of 405 nm, 457 nm, 527 nm, 600 nm, 660 nm, 730 nm, and 860 nm;
[0056] (6) Perform weighted averaging on the collected multispectral images to generate a multispectral fusion image. The weighted average calculation formula is: where w i is the weight of each spectral image, P i (x, y) is the pixel value of each spectral image at position (x, y).
[0057] 3. Image registration: Due to the differences in acquisition equipment and shooting conditions, before the polarization fusion image and the multispectral fusion image are fused, it is necessary to ensure that the two images are aligned in terms of spatial position and resolution. Therefore, image registration based on the SIFT algorithm is required.
[0058] (7) Feature point detection: SIFT algorithm is applied to polarization fusion image and multispectral fusion image to detect key points (i.e., feature points). These key points are scale invariant and rotation invariant.
[0059] (8) Feature point description: For each detected key point, the gradient direction histogram of the surrounding area is calculated and a feature vector describing the key point is generated.
[0060] (9) Feature point matching: Use Euclidean distance as a measure of similarity between feature vectors to find matching feature point pairs between the polarization fusion image and the multispectral fusion image.
[0061] (10) Transformation model estimation: Based on the matched feature point pairs, the multispectral fusion image is mapped to the spatial position and resolution of the polarization fusion image using affine transformation.
[0062] (11) Image resampling and interpolation: The multispectral fusion image is resampled and interpolated to generate a new image that is aligned with the polarization fusion image in terms of space and resolution.
[0063] (12) Registration optimization and verification: Use the RANSAC (Random Sample Consensus) algorithm to eliminate incorrect matching point pairs and optimize the transformation model.
[0064] 4. Polarization and multispectral image fusion: Use wavelet transform to perform multi-scale decomposition of polarization fusion and multispectral fusion images to obtain sub-images of different scales, which are fused separately and finally inversely transformed to generate polarization multispectral fusion images.
[0065] (13) Image decomposition: Use two-dimensional discrete wavelet transform (2D-DWT) to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image, respectively, and decompose them into low-frequency coefficient I, high-frequency coefficient I, low-frequency coefficient II, and high-frequency coefficient III;
[0066] (14) Fusion operation: For the approximate component (low-frequency information), since it contains the main structure and contour information of the image, weighted averaging can be used for fusion to obtain the fused low-frequency coefficients; for the detail component (high-frequency information), since it contains the texture and edge information of the image, local contrast-based fusion is used to obtain the fused high-frequency coefficients;
[0067] (15) Inverse transformation and reconstruction: The fused approximate component and detail component are reconstructed through inverse two-dimensional discrete wavelet transform (2D-IDWT) to generate the final polarization multispectral fusion image.
[0068] 5. Defect detection and qualification determination:
[0069] (16) Data enhancement: Based on the generated polarization multispectral fusion image, five data enhancement strategies such as translation, rotation, brightness adjustment, random noise addition, and random cropping are used to expand the collected OLED defect dataset by five times;
[0070] (17) Introduction of model attention mechanism: In order to improve the model's feature extraction and detection and classification capabilities in small sample scenarios, the original YoloV8 model was optimized by adding the CABM attention mechanism, and the output of CABM was fused with the original features to enhance the generalization ability of the model;
[0071] (18) Model pre-training: Using the transfer learning training strategy, the improved YoloV8 model is pre-trained on the NEU-DET public dataset to obtain the pre-train model, and then formally trained on the enhanced OLED dataset to obtain the trained detection model.
[0072] (19) Model prediction and result output: The OLED fusion image to be tested is input into the model for defect detection. Based on the test results, it is determined whether the OLED screen is OK or NG, and the corresponding judgment result is output.
[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A polarization multi-spectral fusion OLED defect detection method, characterized in that: The method is implemented by the following means: Polarization camera: used to collect images of the OLED screen in four different polarization directions; Multispectral camera: used to collect spectral images in multiple spectral bands; The lighting device includes a polarized light source capable of emitting uniform linearly polarized light and a multi-spectral light source capable of emitting multi-spectral light; the polarized light source is composed of a polarizing film attached to the light-emitting surface of the coaxial light source; Sample transport device: including a conveyor belt for automatically loading and transporting OLED samples; Industrial computer: used to synchronously control the acquisition speed of the image acquisition device and perform image preprocessing, image fusion and defect detection algorithms; Light source controller: used to control the switch and brightness of the polarized light source and the multi-spectral light source, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source to be consistent with the acquisition frequency of the multi-spectral camera; The light source controller, conveyor belt, polarization camera and multi-spectral camera are all connected to the industrial computer, the polarization light source and multi-spectral light source are all connected to the light source controller, the polarization camera and polarization light source constitute the first detection point through which the OLED sample passes, and the multi-spectral light source and multi-spectral camera constitute the second detection point through which the OLED sample passes; the detection method comprises the following steps: 1) At the first detection point, the polarization camera is used to collect polarization images of the OLED screen in four different polarization directions (0°, 45°, 90°, and 135°), which are respectively recorded as I0, I 45 , I 90 , I 135 ; 2) Based on the collected polarization images in four directions, the Stokes parameter S is calculated, and the degree of linear polarization DoLP and the angle of polarization AoP are solved according to the Stokes parameter. Then, the Stokes parameter, DoLP and AoP are mapped to the HSV color space to generate a polarization fusion image; 3) When detecting the second point, the multi-spectral camera is used to collect images of the OLED screen in multiple spectral bands, and a multi-spectral fusion image is generated by a weighted average method; 4) Based on the SIFT algorithm, the polarization fusion image and the multispectral fusion image are registered to ensure the consistency of the two images in spatial position and resolution; 5) Using wavelet transform to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image to obtain sub-images of different scales, and perform fusion processing respectively, and finally generate a polarization multispectral fusion image through inverse transformation; 6) Based on the polarization multi-spectral fusion image, the improved YoloV8 algorithm is used for defect detection, and the OLED screen is judged whether it is qualified according to the detection results, and the corresponding judgment results are output.
2. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 1, characterized in that: The calculation formula of the Stokes parameter is:
3. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 1 or 2, characterized in that: The calculation formulas for the degree of linear polarization (DoLP) and the angle of polarization (AoP) are respectively 4. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 3 is characterized in that: The spectral bands collected by the multi-spectral camera include 405nm, 457nm, 527nm, 600nm, 660nm, 730nm, and 860nm spectral bands.
5. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 1, 2 or 4, characterized in that: The registration of the polarization fusion image and the multispectral fusion image based on the SIFT algorithm specifically includes detecting key points, generating feature vectors, matching feature point pairs through the SIFT algorithm, and eliminating erroneous matching point pairs using the RANSAC algorithm.
6. The OLED defect detection method based on polarization multi-spectrum fusion according to claim 5, characterized in that: The polarization-based multispectral fusion image uses the YoloV8 algorithm for defect detection, and judges whether the OLED screen is qualified based on the detection results, and outputs the corresponding judgment results: The YoloV8 algorithm is optimized based on the original model, including the introduction of the CABM (Convolutional BlockAttention Module) attention mechanism, and the output of CABM is fused with the original features to enhance the generalization ability of the model.
Citation Information
Patent Citations
Camouflage object identification method and system based on polarized light acquisition and wave band operation
CN115359357A
Target detection method based on snapshot spectrum polarization camera
CN115731456A
Systems and methods for augmentation of sensor systems and imaging systems with polarization
US20210356572A1
Cited By
Digital image target detection method and system based on medical microbiological examination
CN119693945A
A digital image target detection method and system based on medical microbiological inspection
CN119693945B
Liquid crystal display screen defect detection method
CN119827522A
Screen risk assessment method and device based on self-built data set, equipment and medium
CN120318623A
Qualitative detection method and device for tiny linear defects on surface of magnetic shoe and medium
CN120635097A
