Multistage multi-mode fused OLED defect detection device
By adopting a two-point detection system combining polarization and multi-spectral information in OLED defect detection, the problem of specular reflection interference and detection accuracy in traditional detection technology is solved, and efficient, comprehensive and reliable defect detection is achieved, which improves the Detection rate of subtle defects.
Patent Information
- Application Number
- CN202410976724.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-21
- Publication Date
- 2024-11-08
AI Technical Summary
Traditional OLED defect detection technology has problems such as specular reflection interference, limited detection accuracy and incomplete detection types, resulting in inaccurate and incomplete detection results.
A multi-stage multi-modal fusion OLED defect detection device is designed, using a dual-point detection system combining polarization and multi-spectral information. Polarization fusion and multi-spectral fusion detection are carried out through coaxial light sources and multi-spectral light sources respectively to improve detection. Accuracy and comprehensiveness.
It realizes high-quality image acquisition under a large vertical field of view, improves the detection ability of subtle defects, enhances the defect detection rate, and provides an efficient, comprehensive and reliable OLED defect detection solution.
Smart Images

Figure CN118914221A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of workpiece manufacturing and processing, and in particular to a multi-level multi-modal fusion OLED defect detection device. Background Art
[0002] Defect detection of OLED flexible screens is a crucial link in the manufacturing process and is of great significance to the quality of finished products and the analysis of the manufacturing process. Traditional visual inspection solutions have high labor costs, low efficiency, and are easily affected by subjective factors. The currently widely used automatic optical inspection technology (Automatic Optic Inspection, AOI) mostly uses machine vision solutions based on intensity images. Most of these inspection solutions have the following problems:
[0003] 1. Mirror reflection interferes with the measurement results: Since the surface of the OLED screen is smooth and it is a mirror-reflecting object, there is often a camera reflection in the high-resolution images taken vertically, which in turn causes great interference to the detection results. If the camera angle is adjusted to shoot at an angle, a certain full-field resolution is usually sacrificed, resulting in low measurement accuracy.
[0004] 2. Limited detection accuracy: Due to the material characteristics of OLED, slight defects may cause the screen to oxidize and be damaged at a faster rate. However, the slight defects in the intensity images collected by ordinary industrial cameras often have low contrast and are difficult to distinguish, which cannot meet the needs of actual industrial inspection.
[0005] 3. Incomplete detection types: There are many types of defects in OLED screens, and their shapes and characteristics are complex and varied. A single detection device is often unable to detect all types of defects. Summary of the invention
[0006] The present invention improves upon the deficiencies of the prior art and designs a multi-level multi-modal fusion device based on polarization and multi-spectrum for six types of defects of OLED screens: cracks, scratches, foreign matter, broken pieces, bubbles and dirt.
[0007] To address the limited accuracy of traditional image-based detection solutions, polarization and multi-spectral information are innovatively introduced in the OLED image acquisition stage to enhance defect characteristics and thereby improve the system's ability to detect subtle defects.
[0008] In view of the fact that OLED screen defects are diverse and complex in form, and that a single device cannot fully detect them, a dual-point detection system is used to detect different types of defects in stages based on the causes of defects and material properties. Taking advantage of the fact that polarization information is sensitive to the roughness and topographical features of the surface of the object to be tested, the first detection point based on polarization fusion is used to detect cracks, scratches, foreign matter, and broken pieces; taking advantage of the fact that bubbles and dirt are sensitive to spectral characteristics, the second device based on multi-spectral fusion is used to detect bubbles and dirt defects on the OLED screen.
[0009] The present invention is achieved through the following technical solutions:
[0010] The present invention discloses a multi-level multi-modal fusion OLED defect detection device, the device comprising:
[0011] Sample transmission control device: including a conveyor belt for automatically loading, placing and transmitting OLED samples;
[0012] Dual-point image acquisition device: polarization camera and multispectral camera, used for detection based on polarization fusion and multispectral fusion respectively;
[0013] Illumination device: a coaxial light source and a multi-spectral light source, the coaxial light source is used to emit coaxial light to the OLED sample and generate reflected light to the field of view of the polarization camera; the multi-spectral light source is used to emit multi-spectral annular light to the OLED sample and generate reflected light to the field of view of the multi-spectral camera;
[0014] Light source controller: used to control the switch and brightness of the coaxial light source and the multi-spectral light source, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source to be consistent with the acquisition frequency of the multi-spectral camera;
[0015] Industrial computer: used to synchronously control the acquisition speed of the image acquisition device, generate polarization fusion images and multi-spectral fusion images respectively, execute the defect detection algorithm, and post-fuse the two-stage detection results to determine whether the sample is qualified;
[0016] Polarizing film: attached to the emitting surface of the coaxial light source to obtain polarized light;
[0017] The light source controller, conveyor belt, polarization camera and multispectral camera are all connected to the industrial computer, the coaxial light source and the multispectral light source are all connected to the light source controller, the polarization camera, coaxial light source and polarization film constitute the first detection point through which the OLED sample passes, and the multispectral light source and multispectral camera constitute the second detection point through which the OLED sample passes.
[0018] As a further improvement, the detection device of the present invention further comprises a sample stage, which is placed on a conveyor belt and is used for placing OLED samples to be detected.
[0019] As a further improvement, the conveyor belt described in the present invention is used to stably transport samples through two detection points; the coaxial light source is used to provide uniform illumination; and the multi-spectral ring light is used to provide illumination with a wide spectrum.
[0020] As a further improvement, the detection point one described in the present invention is used to detect cracks, scratches, foreign matter and broken pieces on the surface of the OLED screen; the detection point is used to detect defects such as bubbles and dirt on the OLED screen.
[0021] As a further improvement, the light emitting area of the coaxial light source described in the present invention is greater than or equal to 1.5 times the size of the sample, ensuring that the light emitted by the coaxial light source can be evenly irradiated on the surface of the sample.
[0022] As a further improvement, the two detection points described in the present invention also include a fixing device for fixing the cameras and light sources in the two points on the main device, wherein the fixing device is connected to the polarization camera and coaxial light source in point one, and the fixing device is connected to the multispectral camera and multispectral ring light in point two, thereby ensuring the stability and accuracy of each point during the detection process.
[0023] The beneficial effects of the present invention are as follows:
[0024] 1. Acquisition of high-quality images in a large vertical field of view: The present invention uses a large-size coaxial light source to effectively remove the interference of camera reflections in the collected images, solves the problem of reduced measurement accuracy in the usual oblique shooting de-reflection strategy, and realizes the acquisition of high-quality images in a large vertical field of view;
[0025] 2. Acquisition of high-quality polarization images: The present invention solves the problem of more noise in the polarization degree and polarization image obtained in the conventional light source acquisition scenario with low polarization degree by adding a linear polarization film to the coaxial light source, thereby achieving the acquisition of high-quality polarization images;
[0026] 3. Improved detection capability of subtle defects: The present invention introduces polarization modal information into the image by designing a point-by-point one-segment polarization fusion device; and introduces spectral information into the image by designing a point-by-point two-segment multi-spectral fusion device. The introduction of two types of modal information enhances the features of subtle defects in the collected image, thereby improving the detection rate of subtle defects;
[0027] 4. Achieve efficient, comprehensive and reliable OLED defect detection: This device uses a multi-level multi-modal fusion device, which divides the detection process into a one-stage detection based on a polarization fusion device and a two-stage detection based on a multi-spectral fusion device, and uses an industrial computer to post-fuse the two-stage detection results to make a more comprehensive and accurate assessment of the defects of the OLED screen. This multi-level detection result fusion method effectively improves the recall rate of the defect detection system, reduces the false detection rate, and provides a more efficient, comprehensive and reliable OLED screen defect detection technical solution. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 is a schematic diagram of a multi-level multi-modal fusion detection device;
[0029] In the figure, 1 is a polarization camera, 2 is a coaxial light source, 3 is a polarization film, 4 is a sample stage, 5 is a conveyor belt, 7 is a multispectral light source, 8 is a multispectral camera, 9 is a light source controller, and 10 is an industrial computer. DETAILED DESCRIPTION
[0030] like Figure 1 The figure shows a multi-level multi-modal fusion detection device designed by the present invention, which mainly includes a sample transmission control device, a dual-point image acquisition device, an illumination device, a light source controller 9, an industrial computer 10 and a polarization film 3, etc.
[0031] Sample transmission control device: includes a conveyor belt 5 and a control motor, which adopts the Zhenyu Automation multi-station guide line group for automatic loading, placement and transmission of OLED samples; the conveyor belt 5 is used to stably transport the samples through the next two detection points.
[0032] Dual-point acquisition device: includes polarization camera 1 and multispectral camera 8. Polarization camera 1 uses Hikvision MV-CH050-10UP industrial camera and MVL-MF2528M-8MP industrial camera lens group to acquire four-directional polarization images; multispectral camera 8 uses Keyence CA-H500MX industrial camera and CA-LH25 industrial camera lens group to acquire multi-spectral images.
[0033] Lighting device: including Dongguan COAX-200X coaxial light source 2 and Keyence CA-DRM10X multi-spectral light source 7. The coaxial light source 2 is used to provide uniform lighting, emit coaxial light to the OLED sample and generate reflected light to the field of view of the polarization camera 1. The light emitting area of the coaxial light source 2 is greater than or equal to 1.5 times the size of the sample, ensuring that the light emitted by the coaxial light source 2 can be evenly irradiated on the sample surface.
[0034] The multi-spectral light source 7 provides illumination with a wide spectrum, emitting multi-spectral ring light onto the OLED sample and generating reflected light into the field of view of the multi-spectral camera 8 .
[0035] Light source controller 9: A KEYENCE CA-DRM10X light source controller 9 is used to control the switch and brightness of the coaxial light source 2 and the multi-spectral light source 7, and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source 7 to be consistent with the acquisition frequency of the multi-spectral camera 8;
[0036] Industrial computer 10: A Dell Inspiron 3020 office unit is used to synchronously control the acquisition speed of the image acquisition device, generate polarization fusion images and multi-spectral fusion images respectively, execute the defect detection algorithm, and post-fuse the two-stage detection results to determine whether the sample is qualified.
[0037] Polarizing film 3: A new material 200mm*200mm linear polarizing film 3 is grown, which matches the coaxial light emitting area and is attached to the coaxial light emitting surface to obtain polarized light.
[0038] The detection device further comprises a sample stage 4 , which is placed on a conveyor belt 5 and is used for placing the OLED sample to be detected.
[0039] The light source controller 9, the conveyor belt 5, the polarization camera 1 and the multispectral camera 8 are all connected to the industrial computer 10, the coaxial light source 2 and the multispectral light source 7 are all connected to the light source controller 9, the polarization camera 1, the coaxial light source 2 and the polarization film 3 constitute the first detection point through which the OLED sample passes, and the multispectral light source 7 and the multispectral camera 8 constitute the second detection point through which the OLED sample passes. The first detection point is used to detect cracks, scratches, foreign matter and broken pieces on the surface of the OLED screen; the detection point is used to detect defects such as bubbles and dirt on the OLED screen.
[0040] The two detection points also include a fixing device for fixing the cameras and light sources in the two points on the main device, wherein the fixing device is connected to the polarization camera 1 and the coaxial light source 2 in point one, and the fixing device is connected to the multispectral camera 8 and the multispectral ring light in point two, to ensure the stability and accuracy of each point during the detection process.
[0041] The testing process is as follows:
[0042] 1OLED samples are placed on conveyor belt 5 through the automatic loading system and transferred to point 1 for a first-stage test.
[0043] A coaxial light source 2 is provided at point 2, and a linear polarization film 3 is attached to the light-emitting surface. The emitted coaxial light is evenly reflected by the OLED sample. The four-directional polarization image of the OLED screen is captured by the polarization camera 1 and input into the industrial computer 10 for fusion processing to obtain a polarization fusion image, and the first-stage defect detection is performed through the back-end detection algorithm.
[0044] After the first stage of testing is completed, the OLED sample is further transferred to point 2 via conveyor belt 5 for the second stage of testing.
[0045] The multi-spectral light source 7 in the second point of the 4-point position emits light of different spectral bands in time-sharing under the control of the industrial computer 10. The multi-spectral camera 8 acquires multi-spectral images synchronously under the control of the light source controller 9 and the industrial computer 10. The multi-spectral images are fused in the industrial computer 10 to obtain a multi-spectral fused image, and the two-stage defect detection is performed through the back-end detection algorithm.
[0046] 5 In the industrial computer 10, the defect detection results of the first stage and the second stage are post-fused, and whether the OLED sample is qualified is determined based on the fusion result.
[0047] Specifically, it is divided into two stages of fusion. The first stage is polarization fusion based on four-way polarization images and multispectral fusion based on multi-spectral images. The second stage is polarization and multispectral image fusion based on multi-scale decomposition. The specific detection steps are as follows:
[0048] 1. Polarization image acquisition and processing:
[0049] (1) When detecting the first point, use polarization camera 1 to collect polarization images of the OLED screen at four directions of 0°, 45°, 90° and 135°, denoted as I 0 , I 45 , I 90 , I 135 ;
[0050] (2) According to the calculation formula of Stokes parameters, the Stokes parameters S are solved using the polarization images collected in four directions, where
[0051] (3) The degree of linear polarization (DoLP) and angle of polarization (AoP) are calculated from the Stokes parameters. The calculation formulas are:
[0052] (4) S 0 , DoLP and AoP are mapped to the HSV color space to obtain the polarization fused image.
[0053] 2. Multispectral image acquisition and processing:
[0054] (5) When detecting the second point, use the multi-spectral camera 8 to collect images of the OLED screen under seven spectral bands of 405 nm, 457 nm, 527 nm, 600 nm, 660 nm, 730 nm, and 860 nm;
[0055] (6) Perform weighted averaging on the collected multispectral images to generate a multispectral fusion image. The weighted average calculation formula is: where w i is the weight of each spectral image, P i (x, y) is the pixel value of each spectral image at position (x, y).
[0056] 3. Image registration: Due to the differences in acquisition equipment and shooting conditions, before the polarization fusion image and the multispectral fusion image are fused, it is necessary to ensure that the two images are aligned in terms of spatial position and resolution. Therefore, image registration based on the SIFT algorithm is required.
[0057] (7) Feature point detection: SIFT algorithm is applied to polarization fusion image and multispectral fusion image to detect key points (i.e., feature points). These key points are scale invariant and rotation invariant.
[0058] (8) Feature point description: For each detected key point, the gradient direction histogram of the surrounding area is calculated and a feature vector describing the key point is generated.
[0059] (9) Feature point matching: Use Euclidean distance as a measure of similarity between feature vectors to find matching feature point pairs between the polarization fusion image and the multispectral fusion image.
[0060] (10) Transformation model estimation: Based on the matched feature point pairs, the multispectral fusion image is mapped to the spatial position and resolution of the polarization fusion image using affine transformation.
[0061] (11) Image resampling and interpolation: The multispectral fusion image is resampled and interpolated to generate a new image that is aligned with the polarization fusion image in terms of space and resolution.
[0062] (12) Registration optimization and verification: Use the RANSAC (Random Sample Consensus) algorithm to eliminate incorrect matching point pairs and optimize the transformation model.
[0063] 4. Polarization and multispectral image fusion: Use wavelet transform to perform multi-scale decomposition of polarization fusion and multispectral fusion images to obtain sub-images of different scales, which are fused separately and finally inversely transformed to generate polarization multispectral fusion images.
[0064] (13) Image decomposition: Use two-dimensional discrete wavelet transform (2D-DWT) to perform multi-scale decomposition on the registered polarization fusion image and multispectral fusion image, respectively, and decompose them into low-frequency coefficient I, high-frequency coefficient I, low-frequency coefficient II, and high-frequency coefficient III;
[0065] (14) Fusion operation: For the approximate component (low-frequency information), since it contains the main structure and contour information of the image, weighted averaging can be used for fusion to obtain the fused low-frequency coefficients; for the detail component (high-frequency information), since it contains the texture and edge information of the image, local contrast-based fusion is used to obtain the fused high-frequency coefficients;
[0066] (15) Inverse transformation and reconstruction: The fused approximate component and detail component are reconstructed through inverse two-dimensional discrete wavelet transform (2D-IDWT) to generate the final polarization multispectral fusion image.
[0067] 5. Defect detection and qualification determination:
[0068] (16) Data enhancement: Based on the generated polarization multispectral fusion image, five data enhancement strategies such as translation, rotation, brightness adjustment, random noise addition, and random cropping are used to expand the collected OLED defect dataset by five times;
[0069] (17) Introduction of model attention mechanism: In order to improve the model's feature extraction and detection and classification capabilities in small sample scenarios, the original YoloV8 model was optimized by adding the CABM attention mechanism, and the output of CABM was fused with the original features to enhance the generalization ability of the model;
[0070] (18) Model pre-training: Using the transfer learning training strategy, the improved YoloV8 model is pre-trained on the NEU-DET public dataset to obtain the pre-train model, and then formally trained on the enhanced OLED dataset to obtain the trained detection model.
[0071] (19) Model prediction and result output: The OLED fusion image to be tested is input into the model for defect detection. Based on the test results, it is determined whether the OLED screen is OK or NG, and the corresponding judgment result is output.
[0072] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A multi-level multi-modal fusion OLED defect detection device, characterized in that: The device comprises: The sample transmission control device comprises a conveyor belt (5) for automatically loading, placing and transmitting OLED samples; A dual-point image acquisition device: a polarization camera (1) and a multispectral camera (8), which are used for detection based on polarization fusion and multispectral fusion respectively; Illumination device: a coaxial light source (2) and a multispectral light source (7), wherein the coaxial light source is used to emit coaxial light to the OLED sample and generate reflected light to the field of view of the polarization camera; and the multispectral light source is used to emit multispectral annular light to the OLED sample and generate reflected light to the field of view of the multispectral camera; A light source controller (9): used to control the switching and brightness of the coaxial light source (2) and the multi-spectral light source (7), and to control the emission frequency of the multi-spectral ring light emitted by the multi-spectral light source to be consistent with the acquisition frequency of the multi-spectral camera; Industrial computer (10): used to synchronously control the acquisition speed of the image acquisition device, generate polarization fusion images and multi-spectral fusion images respectively, execute defect detection algorithm, and post-fuse the two-stage detection results to determine whether the sample is qualified; Polarizing film (3): attached to the light-emitting surface of the coaxial light source (2) to obtain polarized light; The light source controller (9), the conveyor belt (5), the polarization camera and the multispectral camera are all connected to the industrial computer; the coaxial light source and the multispectral light source are all connected to the light source controller (9); the polarization camera, the coaxial light source and the polarization film constitute a first detection point through which the OLED sample passes; and the multispectral light source and the multispectral camera constitute a second detection point through which the OLED sample passes.
2. The OLED defect detection device according to claim 1, characterized in that: The detection device further comprises a sample stage, which is placed on a conveyor belt and is used for placing the OLED samples to be detected.
3. The OLED defect detection device according to claim 1 or 2, characterized in that: The conveyor belt is used to stably transport the sample through two detection points; the coaxial light source is used to provide uniform illumination; and the multi-spectral ring light (7) is used to provide illumination with a wide spectrum.
4. The OLED defect detection device according to claim 3, characterized in that: The detection point one is used to detect cracks, scratches, foreign matter and broken pieces on the surface of the OLED screen; the detection point is used to detect defects such as bubbles and dirt on the OLED screen.
5. The OLED defect detection device according to claim 1, 2 or 4, characterized in that: The light emitting area of the coaxial light source is greater than or equal to 1.5 times the size of the sample, ensuring that the light emitted by the coaxial light source can be evenly irradiated on the surface of the sample.
6. The OLED defect detection device according to claim 5, characterized in that: The two detection points also include fixing devices (11) and (12) for fixing the cameras and light sources in the two points on the main device, wherein the fixing device (11) is connected to the polarization camera (1) and the coaxial light source (2) in point one, and the fixing device (12) is connected to the multi-spectral camera (8) and the multi-spectral ring light (7) in point two, so as to ensure the stability and accuracy of each point during the detection process.
Citation Information
Patent Citations
Multispectral chromatic polarization detection method of foreign impurity in the textile raw material
CN101587079A
Method and system for detecting defect of patterned substrate
CN101819165A
Near-infrared polarization invisible anti-fake element identification system
CN104075998A
Visual observing system for growth process of microcrystalline silicon film and measurement method
CN104122209A
Defect detection device based on optical imaging
CN214374364U
Cited By
A method for detecting OLED screen defects based on optical-magnetic multimodal fusion
CN119780107A
A method for detecting OLED screen defects based on optical-magnetic multimodal fusion
CN119780107B
Defect detection method and device applied to glass substrate in display panel
CN119880802A
Defect Detection Method and Device for Inner Glass Substrate of Display Panel
CN119880802B
Visual inspection system and method based on aluminum plate size and appearance surface defects
CN119951764A
