Multi-stage multi-modal fusion-based OLED defect detection device

Abstract

A multi-stage multi-modal fusion-based OLED defect detection device. In or accuracy in conventional image-based detection solutions, the multi-stage multi-modal fusion-based OLED defect detection device introduces polarization and multi-spectral information during an OLED image acquisition stage to enhance defect features, thereby improving detection capability of a system for subtle defects. In order to solve the problem that OLED screen defects exhibit diverse types and complex and variable morphologies, and thus a single device cannot fully detect the OLED screen defects, on the basis of causes of defects and material characteristics, the multi-stage multi-modal fusion-based OLED defect detection device uses a dual-point detection system to perform staged detection for different types of defects. Utilizing the characteristic that polarization information is sensitive to roughness and morphological features of the surface of an object to be tested, a polarization fusion-based detection point I is used to detect crack, scratch, foreign matter, and broken fragment defects. Utilizing the characteristic that bubbles and stains are sensitive to spectral properties, a multi-spectral fusion-based device II is used to detect bubble and stain defects on the OLED screen.

Description

Multi-level multi-modal fusion OLED defect detection device TECHNICAL FIELD

[0001] The present application relates to the field of workpiece manufacturing and processing, in particular to a multi-level multi-modal fusion OLED defect detection device. BACKGROUND

[0002] Defect detection of OLED flexible screen is a crucial link in its manufacturing process, and is of great significance to product quality and manufacturing process analysis. Traditional visual inspection scheme has high labor cost, low efficiency and is easily affected by subjective factors. The widely used automatic optical detection technology (Automatic Optic Inspection, AOI) at present mostly adopts machine vision scheme based on intensity image. These detection schemes mostly have the following problems:

[0003] 1. Mirror reflection interferes with the measurement results: due to the smooth surface of OLED screen, it is a mirror reflection object, and there is often a camera reflection in the high-resolution image of vertical shooting, which further brings great interference to the detection results. If the camera angle is adjusted to shoot obliquely, the full field of view resolution will be sacrificed, and the measurement accuracy will be low.

[0004] 2. Limited detection accuracy: due to the material properties of OLED, subtle defects can cause the screen to oxidize and damage at a relatively fast speed. The subtle defects in the intensity image collected by ordinary industrial cameras often have low contrast and are difficult to distinguish, which cannot meet the needs of actual industrial detection.

[0005] 3. Incomplete detection types: the defect types of OLED screen are rich, and the shapes and characteristics are complex and changeable. A single detection device cannot detect all types of defects. SUMMARY

[0006] The present application improves the deficiencies of the prior art and designs a multi-level multi-modal fusion device based on polarization and multispectral for six types of defects of OLED screen: cracks, scratches, foreign matter, broken pieces, bubbles and dirt.

[0007] In view of the problem of limited accuracy of traditional image-based detection scheme, polarization and multispectral information are innovatively introduced in the OLED image acquisition stage to enhance defect features and improve the detection ability of the system for subtle defects.

[0008] Aiming at the problem that the defects of OLED screen body are rich in types, complex in form and cannot be completely detected by a single device, a two-point detection system is adopted to detect different types of defects in stages from the causes of defects and material properties. The roughness and topographic features of the surface of the object to be measured are more sensitive to the characteristics of polarization information, and the detection point based on polarization fusion is used to detect crack, scratch, foreign matter and fragment defects; bubbles and dirt are sensitive to the characteristics of the spectrum, and the device two based on multispectral fusion is used to detect bubble and dirt defects on the OLED screen body.

[0009] The present application is realized by the following technical solutions:

[0010] The present application discloses a kind of multi-level multi-modal fusion's OLED defect detection device, device includes:

[0011] Sample transmission control device: including conveyor belt, for automatic feeding, placing and transmitting OLED sample;

[0012] Double-point image acquisition device: polarization camera and multispectral camera, respectively for detecting based on polarization fusion and multispectral fusion;

[0013] Illumination device: coaxial light source and multispectral light source, coaxial light source is used to emit coaxial light to OLED sample and produce reflected light to the field of view range of polarization camera;Multispectral light source is used to emit multispectral ring light to OLED sample and produce reflected light to the field of view range of multispectral camera;

[0014] Light source controller: for controlling the switch and brightness of coaxial light source and multispectral light source, and controlling the emission frequency of multispectral ring light emitted by multispectral light source and the acquisition frequency of multispectral camera;

[0015] Industrial computer: for synchronously controlling the acquisition speed of image acquisition device, generating polarization fusion image and multispectral fusion image respectively, executing defect detection algorithm, and post-fusing two-stage detection results to determine whether sample is qualified;

[0016] Polarization film: attached to the light emitting surface of coaxial light source, for obtaining polarized light;

[0017] Light source controller, conveyor belt, polarization camera and multispectral camera are connected with industrial computer, coaxial light source and multispectral light source are connected with light source controller, polarization camera, coaxial light source and polarization film constitute the first detection point passed by OLED sample, and multispectral light source and multispectral camera constitute the second detection point passed by OLED sample.

[0018] As a further improvement, the detection device of the present application further comprises a sample stage, which is placed on the conveyor belt and used to place the OLED sample to be detected.

[0019] As a further improvement, the conveying belt is used to stably transport the sample through two detection points; the coaxial light source is used to provide uniform illumination; the multi-spectral ring light is used to provide illumination of a wide range of spectra.

[0020] As a further improvement, the detection point one is used to detect cracks, scratches, foreign matter and broken pieces on the surface of the OLED screen; the detection point two is used to detect bubbles and dirt on the OLED screen.

[0021] As a further improvement, the light-emitting area of the coaxial light source is greater than or equal to 1.5 times the size of the sample, ensuring that the light emitted by the coaxial light source can be uniformly illuminated on the surface of the sample.

[0022] As a further improvement, the two detection points further comprise a fixing device for fixing the camera and the light source in the two points on the main device, wherein the fixing device is connected with the polarized camera and the coaxial light source in the point one, and the fixing device is connected with the multi-spectral camera and the multi-spectral ring light in the point two, ensuring the stability and accuracy of each point during detection.

[0023] The beneficial effects of the present application are as follows:

[0024] 1. High-quality image acquisition under large vertical field of view: the present application effectively removes the interference of camera reflection in the collected image by using a large-size coaxial light source, solves the problem of reduced measurement accuracy in the conventional anti-reflection strategy of tilted shooting, and realizes high-quality image acquisition under large vertical field of view;

[0025] 2. High-quality polarization image acquisition: the present application solves the problem of noise in the acquired polarization degree and polarization perception image under the conventional light source with low polarization degree by adding a linear polarization film on the coaxial light source, and realizes high-quality polarization image acquisition;

[0026] 3. Improved detection ability for subtle defects: the present application introduces polarization modal information in the image by designing a polarization fusion device in the point one; by designing a multi-spectral fusion device in the point two, spectral information is introduced into the image, and the introduction of the two modal information enhances the characteristics of the subtle defects in the collected image, thereby improving the detection rate of the subtle defects;

[0027] 4. Achieve efficient, comprehensive and reliable OLED defect detection: The device adopts multi-level multi-modal fusion device, divides the detection process into one-stage detection based on polarization fusion device and two-stage detection based on multi-spectral fusion device, and uses industrial computer to perform post-fusion on the two-stage detection results, to more comprehensively and accurately evaluate the defects of OLED screen body. This multi-level detection result fusion method effectively improves the recall rate of the defect detection system, reduces the false detection rate, and provides a more efficient, comprehensive and reliable OLED screen body defect detection technical solution. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a schematic diagram of a multi-level multi-modal fusion detection device;

[0029] In the figure, 1 is a polarization camera, 2 is a coaxial light source, 3 is a polarization film, 4 is a sample stage, 5 is a conveyor belt, 7 is a multi-spectral light source, 8 is a multi-spectral camera, 9 is a light source controller, and 10 is an industrial computer. DETAILED DESCRIPTION

[0030] As shown in Figure 1, the multi-level multi-modal fusion detection device designed by the device mainly includes a sample transmission control device, a double-point image acquisition device, an illumination device, a light source controller 9, an industrial computer 10 and a polarization film 3, etc.

[0031] Sample transmission control device: including a conveyor belt 5 and a control motor, using a Zhen Yuan automation multi-station guide rail line group for automatic feeding, placing and transmitting OLED samples; the conveyor belt 5 is used to stably transport the samples through the next two detection points.

[0032] Double-point acquisition device: including a polarization camera 1 and a multi-spectral camera 8. The polarization camera 1 uses a Hikvision MV-CH050-10UP industrial camera and a MVL-MF2528M-8MP industrial camera lens group to acquire four-way polarization images; the multi-spectral camera 8 uses a Keyence CA-H500MX industrial camera and a CA-LH25 industrial camera lens group to acquire multi-spectral images.

[0033] Illumination device: including Dongguan COAX-200X coaxial light source 2 and Keyence CA-DRM10X multi-spectral light source 7. The coaxial light source 2 is used to provide uniform illumination, emit coaxial light to the OLED sample and produce reflected light to the field of view range of the polarization camera 1, and the light emitting area of the coaxial light source 2 is greater than or equal to 1.5 times the size of the sample, to ensure that the light emitted by the coaxial light source 2 can be uniformly irradiated on the sample surface.

[0034] The multi-spectral light source 7 provides wide range of spectrum illumination, emits multi-spectral ring light to the OLED sample and produces reflected light to the field of view range of the multi-spectral camera 8.

[0035] Light source controller 9: use the Keyence CA-DRM10X light source controller 9 to control the switching and brightness of the coaxial light source 2 and the multi-spectral light source 7, and control the multi-spectral ring light emission frequency of the multi-spectral light source 7 to be consistent with the acquisition frequency of the multi-spectral camera 8;

[0036] Industrial computer 10: use Dell Ling Yue 3020 office machine group to synchronously control the acquisition speed of the image acquisition device, respectively generate polarization fusion images and multi-spectral fusion images, execute defect detection algorithms, and perform post-fusion of the two-stage detection results to determine whether the sample is qualified.

[0037] Polarization film 3: long Xinxin 200mm*200mm linear polarization film 3 matches the coaxial light emitting area, and is attached to the coaxial light emitting surface to obtain polarized light.

[0038] The detection device further comprises a sample table 4 placed on the conveying belt 5 for placing the OLED sample to be detected.

[0039] The light source controller 9, the conveying belt 5, the polarization camera 1 and the multi-spectral camera 8 are connected with the industrial computer 10, the coaxial light source 2 and the multi-spectral light source 7 are connected with the light source controller 9, the polarization camera 1, the coaxial light source 2 and the polarization film 3 constitute a detection first point position through which the OLED sample passes, and the multi-spectral light source 7 and the multi-spectral camera 8 constitute a detection second point position through which the OLED sample passes. The detection point position one is used for detecting the crack, scratch, foreign matter and fragment defects on the surface of the OLED screen body; and the detection point position is used for detecting the bubble and dirt defects on the OLED screen body.

[0040] The two detection point positions further comprise a fixing device for fixing the cameras and light sources in the two point positions on the main device, wherein the fixing device is connected with the polarization camera 1 and the coaxial light source 2 in the point position one, and the fixing device is connected with the multi-spectral camera 8 and the multi-spectral ring light in the point position two, so as to ensure the stability and precision of each point position in the detection process.

[0041] The detection process is as follows:

[0042] 1. The OLED sample is placed on the conveying belt 5 by the automatic feeding system and is transmitted to the point position one for one-stage detection.

[0043] 2. The coaxial light source 2 is provided in the point position one, and the linear polarization film 3 is attached on the light emitting surface. The emitted coaxial light is uniformly reflected by the OLED sample, the four-way polarization images of the OLED screen are captured by the polarization camera 1, and the polarization fusion images are obtained by inputting the polarization fusion images into the industrial computer 10 for fusion processing, and one-stage defect detection is performed by the rear-end detection algorithm.

[0044] 3. After the one-stage detection is completed, the OLED sample is continuously transmitted to the point position two by the conveying belt 5 for two-stage detection.

[0045] 4The multispectral light source 7 in the second point position emits light of different spectral bands under the control of the industrial computer 10. The multispectral camera 8 synchronously collects multispectral images under the control of the light source controller 9 and the industrial computer 10. The multispectral images are fused in the industrial computer 10 to obtain a multispectral fusion image. The two-stage defect detection is performed by a back-end detection algorithm.

[0046] 5The first-stage and second-stage defect detection results are fused in the industrial computer 10, and the OLED sample is determined to be qualified or not according to the fusion result.

[0047] Specifically, the fusion is mainly divided into two stages. The first stage is polarization fusion based on four-direction polarization images and multispectral fusion based on multispectral images. The second stage is polarization and multispectral image fusion based on multiscale decomposition. The specific detection steps are as follows:

[0048] 1. Polarization image acquisition and processing:

[0049] (1) At the first point position, the polarization camera 1 is used to collect polarization images of the OLED screen at 0°, 45°, 90°, and 135°, respectively, denoted as I0, I 45 , I 90 , and I 135 .

[0050] (2) According to the calculation formula of Stokes parameters, the Stokes parameters S are solved using the collected polarization images of the four directions, where

[0051] (3) The degree of linear polarization (DoLP) and the angle of polarization (AoP) are calculated from the Stokes parameters, and the calculation formulas are

[0052] (4) S0, DoLP, and AoP are mapped to the HSV color space to obtain a polarization fusion image.

[0053] 2. Multispectral image acquisition and processing:

[0054] (5) At the second point position, the multispectral camera 8 is used to collect images of the OLED screen under seven spectral bands of 405 nm, 457 nm, 527 nm, 600 nm, 660 nm, 730 nm, and 860 nm;

[0055] (6) The collected multispectral images are processed by weighted average to generate a multispectral fusion image. The calculation formula of weighted average is where w i is the weight of each spectral band image, P i (x,y) is the pixel value of each spectral band image at position (x,y).

[0056] 3. Image registration: Before the fusion of the polarimetric and multispectral images, it is necessary to ensure that the two images are aligned in spatial position and resolution due to the differences in acquisition equipment and shooting conditions. Therefore, image registration based on the SIFT algorithm is needed.

[0057] (7) Feature point detection: Apply the SIFT algorithm on the polarimetric and multispectral images respectively to detect key points (i.e. feature points). These key points have scale invariance and rotation invariance.

[0058] (8) Feature point description: For each detected key point, calculate the gradient direction histogram of its surrounding area and generate a feature vector that describes the key point.

[0059] (9) Feature point matching: Use the Euclidean distance as a measure of similarity between feature vectors to find matching feature point pairs between the polarimetric and multispectral images.

[0060] (10) Transformation model estimation: Based on the matched feature point pairs, use affine transformation to map the multispectral image to the spatial position and resolution of the polarimetric image.

[0061] (11) Image resampling and interpolation: Resample and interpolate the multispectral image to generate a new image that is aligned with the polarimetric image in spatial and resolution.

[0062] (12) Registration optimization and verification: Use the RANSAC (Random Sample Consensus) algorithm to remove false matching point pairs and optimize the transformation model.

[0063] 4. Polarimetric and multispectral image fusion: Use wavelet transform to perform multiscale decomposition on the polarimetric and multispectral fusion images, obtaining sub-images of different scales, respectively, and perform fusion processing, and finally inverse transform to generate a polarimetric multispectral fusion image.

[0064] (13) Image decomposition: Use two-dimensional discrete wavelet transform (2D-DWT) to perform multiscale decomposition on the registered polarimetric and multispectral fusion images respectively, decomposing into low frequency coefficients I, high frequency coefficients I, low frequency coefficients II, high frequency coefficients III;

[0065] (14) Fusion operation: For the approximation component (low-frequency information), since it contains the main structure and contour information of the image, weighted average can be used for fusion to obtain the fused low-frequency coefficient; for the detail component (high-frequency information), it contains the texture and edge information of the image, so local contrast-based fusion is used to obtain the fused high-frequency coefficient;

[0066] (15) Inverse transform and reconstruction: the fused approximation component and detail component are reconstructed through inverse two-dimensional discrete wavelet transform (2D-IDWT) to generate the final polarized multispectral fusion image.

[0067] 5. Defect detection and qualification:

[0068] (16) Data augmentation: based on the generated polarized multispectral fusion image, five data augmentation strategies such as translation, rotation, brightness adjustment, random noise addition, and random cropping are used to expand the collected OLED defect dataset to five times;

[0069] (17) Model attention mechanism introduction: in order to improve the feature extraction and detection classification ability of the model in the small sample scene, the original YoloV8 model is optimized, the CABM attention mechanism is added, and the output of CABM is fused with the original feature to enhance the generalization ability of the model;

[0070] (18) Model pre-training: using the training strategy of transfer learning, the improved YoloV8 model is pre-trained on the NEU-DET public dataset to obtain the pre-train model, and then formally trained on the enhanced OLED dataset to obtain the trained detection model.

[0071] (19) Model prediction and result output: input the OLED fusion image to be detected into the model for defect detection, and judge whether the OLED screen is OK or NG according to the detection result, and output the corresponding judgment result.

[0072] The above is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-stage multi-modal fusion OLED defect detection device, characterized in that, The device comprises: Sample transmission control device: including a conveyor belt (5) for automatic feeding, placing and transmitting OLED samples; Dual-point image acquisition device: polarized camera (1) and multispectral camera (8) for detecting based on polarization fusion and multispectral fusion respectively; Illumination device: coaxial light source (2) and multispectral light source (7), the coaxial light source is used to emit coaxial light to the OLED sample and generate reflected light to the field of view range of the polarization camera; the multispectral light source is used to emit multispectral ring light to the OLED sample and generate reflected light to the field of view range of the multispectral camera; Light source controller (9): for controlling the switching and brightness of the coaxial light source (2) and the multispectral light source (7), and controlling the multispectral ring light emission frequency of the multispectral light source to be consistent with the multispectral camera acquisition frequency; Industrial computer (10): for synchronously controlling the acquisition speed of the image acquisition device, generating polarization fusion images and multispectral fusion images respectively, executing defect detection algorithm, and post-fusing the two-stage detection results to determine whether the sample is qualified; Polarization film (3): attached to the light-emitting surface of the coaxial light source (2) for obtaining polarized light; The light source controller (9), the conveyor belt (5), the polarization camera and the multispectral camera are connected with the industrial computer, the coaxial light source and the multispectral light source are connected with the light source controller (9), the polarization camera, the coaxial light source and the polarization film constitute the first detection point of the OLED sample, and the multispectral light source, the multispectral camera constitute the second detection point of the OLED sample.

2. The OLED defect detection apparatus of claim 1, wherein, The detection device further comprises a sample stage, which is placed on the conveyor belt and used for placing the OLED sample to be detected.

3. The OLED defect detection apparatus according to claim 1 or 2, characterized in that, The conveyor belt is used for stably conveying the sample through the two detection points; the coaxial light source is used for providing uniform illumination; and the multispectral ring light (7) is used for providing wide-range spectrum illumination.

4. The OLED defect detection apparatus of claim 3, wherein, The first detection point is used for detecting crack, scratch, foreign matter and fragment defects on the surface of the OLED screen body; and the second detection point is used for detecting bubble and dirt defects on the OLED screen body.

5. The OLED defect detection apparatus according to claim 1 or 2 or 4, characterized in that, The light-emitting area of the coaxial light source is greater than or equal to 1.5 times the size of the sample, so as to ensure that the light emitted by the coaxial light source can uniformly irradiate on the sample surface.

6. The OLED defect detection apparatus of claim 5, wherein, The two detection points further comprise fixing devices (11) and (12) for fixing the cameras and light sources in the two points on the main device, wherein the fixing device (11) is connected with the polarization camera (1) and the coaxial light source (2) in the first point, and the fixing device (12) is connected with the multispectral camera (8) and the multispectral ring light (7) in the second point, so as to ensure the stability and precision of each point during the detection process.

Images