ELEAP OLED gum bubble detection method, system, medium and equipment

Abstract

The invention relates to the technical field of OLED (Organic Light Emitting Diode) gum detection, in particular to an eLEAP OLED gum bubble detection method, system, medium and equipment, comprising the following steps: step S10, aligning an RGB (Red, Green, Blue) image with a depth image; step S20, collecting a data set; step S30, labeling the data set; identifying various objects, features or areas in the images, and distributing corresponding labels to the objects, the features or the areas; step S40, carrying out improved YOLOv8 training; step S50, collecting and detecting a real-time product detection image; and step S60, outputting a result. According to the invention, efficient and accurate gum bubble detection can be realized, and the production efficiency and the quality control level are improved; meanwhile, the method can also reduce the workload and error rate of manual detection, and improve the production efficiency and yield.

Description

Technical Field

[0001] The present invention relates to the technical field of OLED adhesive detection, and in particular to an eLEAP OLED adhesive bubble detection method, system, medium and equipment. Background Art

[0002] eLEAP OLED is a new generation of OLED panel technology developed by Japan Display Inc. (JDI), which is manufactured using "evaporation + photolithography technology". Unlike traditional OLED, eLEAP OLED does not require FFM for evaporation during production. It has the remarkable characteristics of high brightness, long life, low power consumption and free form. In the eLEAP OLED module, the back glue is an indispensable component, which plays a key role in protecting the eLEAP OLED panel, IC and flexible circuit board. The adhesive bonding effect directly affects the production quality of eLEAP OLED. At present, the adhesive bonding of the eLEAP OLED module mainly relies on manual bonding, and the bonding quality varies greatly, and there are bubbles that are difficult to detect by the human eye. Therefore, an efficient detection method is urgently needed to solve the current situation; at present, the existing automatic detection methods are usually based on traditional machine learning methods, which require special personnel to design features, but the detection effect is relatively poor. In addition, there are also deep learning detection methods with single image information input. Although the robustness and detection effect have been improved, there are still some cases where the adhesive bubble defective products cannot be accurately detected. This indicates that the current automatic detection method still has a lot of room for improvement in terms of improving efficiency and accuracy. Therefore, the present invention aims to propose an eLEAP OLED back glue bubble detection method to solve the problems existing in the existing method. Summary of the invention

[0003] In view of the problems of the prior art, the present invention provides an eLEAP OLED adhesive bubble detection method, system, medium, and equipment, which can achieve efficient and accurate adhesive bubble detection and improve production efficiency and quality control level; at the same time, the method can also reduce the workload and error rate of manual detection and improve production efficiency and yield rate.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] The present invention provides an eLEAP OLED back glue bubble detection method, which comprises the following steps:

[0006] Step S10, aligning the RGB image with the depth image;

[0007] Step S20, data set collection;

[0008] Step S30, data set annotation: identifying various objects, features or regions therein and assigning corresponding labels to them;

[0009] Step S40: Improved YOLOv8 training;

[0010] Step S50: real-time product inspection image acquisition and inspection;

[0011] Step S60: output the result.

[0012] The method for aligning the RGB image with the depth image in step S10 is as follows: Step S11, calibration of color camera and depth camera parameters: in the same scene, multiple sets of checkerboard calibrations with different distances and angles are collected relative to the depth image and the RGB image, and the parameters of the color camera and the depth camera are obtained using any camera parameter calibration method;

[0013] Step S12: Solve the transformation matrix from the depth camera to the color camera, and solve the formula: Among them, R RGB is the external parameter rotation matrix of the color camera, T RGB is the extrinsic translation matrix of the color camera, R ir is the extrinsic rotation matrix of the depth camera, T ir is the extrinsic translation matrix of the depth camera;

[0014] Step S13, aligning the RGB image with the depth image: using the conversion parameters, according to the position of a specific pixel in the depth image and the camera parameters, calculate the corresponding pixel position in the RGB image, so that the two images are accurately aligned in the same coordinate system.

[0015] The improved YOLOv8 training method in step S40 is:

[0016] Step S41, model initialization: The initialization of the deep learning model refers to the process of setting initial values ​​for the weight parameters and bias values ​​in the model before the training begins; referring to the actual application of YOLOv8, choose to use Gaussian to initialize the model weights;

[0017] Step S42, training strategy: the hyperparameter value of the hyperparameter optimizer is SGD; the hyperparameter value of the hyperparameter learning rate is 0.01; the hyperparameter value of the hyperparameter learning rate decay is 0.0005; the hyperparameter value of the hyperparameter momentum is 0.937; the hyperparameter value of the hyperparameter Batch size is 16; the hyperparameter value of the hyperparameter Epochs is 500.

[0018] Among them, the real-time product inspection image acquisition and detection method in step S50 is: collecting aligned RGB images and depth images of products one by one; after collection, inputting the RGB image and the depth image into the corresponding input ends of the improved YOLOv8 model respectively.

[0019] The present invention also provides an eLEAP OLED adhesive bubble detection system, which includes:

[0020] The acquisition unit is used to collect data sets after aligning the RGB image with the depth image;

[0021] The dataset annotation unit is used to identify various objects, features or regions in the dataset after annotation, and assign corresponding labels to them;

[0022] Model training unit, used to initialize the model and then use the improved YOLOv8 model for multiple training optimization and hyperparameter adjustment;

[0023] The detection device is used to align the RGB image and the depth image of the collected products one by one; after the collection, the RGB image and the depth image are respectively input into the corresponding input ends of the improved YOLOv8 model.

[0024] The present invention also provides a computer storage medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the eLEAP OLED adhesive bubble detection method.

[0025] The present invention also provides an electronic device, wherein the electronic device includes: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, enable the processor to execute the eLEAPOLED back glue bubble detection method.

[0026] Beneficial effects of the present invention:

[0027] The RGB image and depth image alignment of the present invention refers to aligning the pixel points of the RGB image and the depth image so that they describe the color and depth information of the same object in the same coordinate system; through camera calibration and the association of depth information with RGB information, a correspondence between the RGB image and the depth image can be established, so as to align them to the same coordinate system; the embodiment of the present application is based on the principle of deep learning detection method, and on the basis of a single RGB image input, a depth map is added as input, and the image information of the depth map is integrated, so that the adhesive can be quickly detected, while improving the detection accuracy, which helps to improve production efficiency and yield rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1The present invention is a flow chart of a method for detecting bubbles in the eLEAP OLED adhesive backing.

[0029] Figure 2 This is a schematic diagram of the structure of the traditional Yolov8 model.

[0030] Figure 3 It is a structural schematic diagram of the improved Yolov8 model of the present invention. DETAILED DESCRIPTION

[0031] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with the embodiments and drawings, and the contents mentioned in the embodiments are not intended to limit the present invention. The present invention is described in detail below in conjunction with the drawings.

[0032] Embodiment 1

[0033] like Figures 1 to 3 As shown, the first embodiment of the present application provides an eLEAP OLED adhesive bubble detection method, which includes the following steps: step S10, aligning the RGB image with the depth image; step S20, data set acquisition; step S30, data set annotation; identifying various objects, features or areas therein, and assigning corresponding labels to them; step S40, improved YOLOv8 training; step S50, real-time product detection image acquisition and detection; step S60, outputting results; further, the method of aligning the RGB image with the depth image in step S10 is:

[0034] Step S11, calibration of color camera and depth camera parameters: in the same scene, collect multiple sets of checkerboard calibrations with different distances and angles relative to the depth image and the RGB image, and use any camera parameter calibration method to obtain the parameters of the color camera and the depth camera;

[0035] Step S12, solve the transformation matrix from the depth camera to the color camera. The transformation from the depth camera to the color camera is essentially a rigid transformation of the two coordinate systems. It is only necessary to solve the rotation matrix R and the translation matrix T of the depth camera coordinate system to transform the color camera coordinate system; the solution formula is: Among them, R RGB is the external parameter rotation matrix of the color camera, T RGB is the extrinsic translation matrix of the color camera, R ir is the extrinsic rotation matrix of the depth camera, T ir is the extrinsic translation matrix of the depth camera;

[0036] Step S13, aligning the RGB image with the depth image: using the conversion parameters, according to the position of a specific pixel in the depth image and the camera parameters, calculate the corresponding pixel position in the RGB image, so that the two images are accurately aligned in the same coordinate system.

[0037] Specifically, the alignment of the RGB image and the depth image in the embodiment of the present application refers to aligning the pixels of the RGB image and the depth image so that they describe the color and depth information of the same object in the same coordinate system; since the RGB image and the depth image may be acquired through different cameras or sensors, their pixels are not completely aligned; through camera calibration and the association of depth information with RGB information, a correspondence between the RGB image and the depth image can be established, thereby aligning them to the same coordinate system; the embodiment of the present application is based on the principle of deep learning detection method, and on the basis of a single RGB image input, a depth map is added as input, and the image information of the depth map is integrated, so that the adhesive can be quickly detected, while improving the detection accuracy, which helps to improve production efficiency and yield rate.

[0038] In the embodiment of the present application, the above steps have met the basic conditions for YOLOv8 training. Next, the training is carried out from the improved YOLOv8. The specific steps are as follows: Step S41, model initialization: The initialization of the deep learning model refers to the process of setting the initial values ​​for the weight parameters and bias values ​​in the model before the training begins. The quality of the initialization can significantly affect the training efficiency and final performance of the model. Referring to the actual application of YOLOv8, choose to use Gaussian to initialize the model weights. Step S42, training strategy: The hyperparameter value of the hyperparameter optimizer is SGD; the hyperparameter value of the hyperparameter learning rate is 0.01; the hyperparameter value of the hyperparameter learning rate decay is 0.0005; the hyperparameter value of the hyperparameter momentum is 0.937; the hyperparameter value of the hyperparameter Batch size is 16; the hyperparameter value of the hyperparameter Epochs is 500; its training strategy table is as follows:

[0039] Hyperparameter Name Hyperparameter Values optimizer SGD Learning Rate 0.01 Learning rate decay 0.0005 momentum 0.937 Batch size 16 Epochs 500

[0040] In the embodiment of the present application, through the above steps, the training and preliminary preparation of the model have been successfully completed, and it can be applied to actual production inspection. The real-time product inspection image acquisition and detection method in step S50 is: collect the aligned RGB image and depth image of the product one by one; after acquisition, input the RGB image and the depth image into the corresponding input end of the improved YOLOv8 model respectively; step S60 outputs the result; the improved YOLOv8 model is the detection result of the input product image, including outputting each detection result and its confidence, and the detection result can be filtered by setting the threshold of the detection confidence to eliminate useless detection results.

[0041] The embodiments of the present application have the following technical effects:

[0042] In the context that conventional RGB image defect detection methods cannot effectively identify adhesive bubbles, and pure 3D vision technology has high hardware and time costs, we propose an innovative method; this method is as follows Figure 3 As shown in the figure, the YOLOv8 backbone network is used to extract the features of RGB images and depth images respectively as basic features; this solution not only combines the feature information of two-dimensional RGB images, but also incorporates the three-dimensional depth feature information, making full use of the advantages of multi-dimensional features without increasing hardware costs and time costs;

[0043] At the same time, the feature fusion method of feature splicing and convolution is used to efficiently fuse the feature information of two-dimensional RGB images and three-dimensional depth feature information with a slight increase in the amount of calculation, and the neck and Detect network of the original YOLOv8 are guaranteed to make its performance more stable.

[0044] The eLEAP OLED adhesive bubble detection method proposed in the embodiment of the present application can achieve efficient and accurate adhesive bubble detection, thereby improving production efficiency and quality control level; at the same time, the method can also reduce the workload and error rate of manual inspection, thereby improving production efficiency and yield rate.

[0045] Embodiment 2

[0046] Embodiment 2 of the present application provides an eLEAP OLED adhesive bubble detection system, which includes:

[0047] The acquisition unit is used to collect data sets after aligning the RGB image with the depth image;

[0048] The dataset annotation unit is used to identify various objects, features or regions in the dataset after annotation, and assign corresponding labels to them;

[0049] Model training unit, used to initialize the model and then use the improved YOLOv8 model for multiple training optimization and hyperparameter adjustment;

[0050] The detection device is used to align the RGB image and the depth image of the collected products one by one; after the collection, the RGB image and the depth image are respectively input into the corresponding input ends of the improved YOLOv8 model.

[0051] Embodiment 3

[0052] Embodiment 3 of the present application provides a computer storage medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the eLEAP OLED adhesive bubble detection method.

[0053] Embodiment 4

[0054] Embodiment 4 of the present application provides an electronic device, wherein the electronic device includes: a processor; and a memory arranged to store computer executable instructions, wherein the executable instructions, when executed, cause the processor to execute the eLEAP OLED adhesive bubble detection method.

[0055] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0059] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention is disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modifications to equivalent embodiments of equivalent changes by using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent changes and modifications made to the above embodiments according to the technology of the present invention are within the scope of the technical solution of the present invention without departing from the content of the technical solution of the present invention.

Claims

1. A method for detecting bubbles in the adhesive of an eLEAP OLED, characterized in that: The following steps are involved: Step S10, aligning the RGB image with the depth image; Step S20, data set collection; Step S30, data set annotation: identifying various objects, features or regions therein and assigning corresponding labels to them; Step S40: Improved YOLOv8 training; Step S50: real-time product inspection image acquisition and inspection; Step S60: output the result.

2. The eLEAP OLED adhesive bubble detection method according to claim 1, characterized in that: The method for aligning the RGB image with the depth image in step S10 is: Step S11, calibration of color camera and depth camera parameters: in the same scene, collect multiple sets of checkerboard calibrations with different distances and angles relative to the depth image and the RGB image, and use any camera parameter calibration method to obtain the parameters of the color camera and the depth camera; Step S12: Solve the transformation matrix from the depth camera to the color camera, and solve the formula: Among them, R RGB is the external parameter rotation matrix of the color camera, T RGB is the extrinsic translation matrix of the color camera, R ir is the extrinsic rotation matrix of the depth camera, T ir is the extrinsic translation matrix of the depth camera; Step S13, aligning the RGB image with the depth image: using the conversion parameters, according to the position of a specific pixel in the depth image and the camera parameters, calculate the corresponding pixel position in the RGB image, so that the two images are accurately aligned in the same coordinate system.

3. The eLEAP OLED adhesive bubble detection method according to claim 1, characterized in that: The improved YOLOv8 training method in step S40 is: Step S41, model initialization: The initialization of the deep learning model refers to the process of setting initial values ​​for the weight parameters and bias values ​​in the model before the training begins; referring to the actual application of YOLOv8, choose to use Gaussian to initialize the model weights; Step S42, training strategy: the hyperparameter value of the hyperparameter optimizer is SGD; the hyperparameter value of the hyperparameter learning rate is 0.01; the hyperparameter value of the hyperparameter learning rate decay is 0.0005; the hyperparameter value of the hyperparameter momentum is 0.937; the hyperparameter value of the hyperparameter Batch size is 16; the hyperparameter value of the hyperparameter Epochs is 500.

4. The eLEAP OLED adhesive bubble detection method according to claim 1, characterized in that: The real-time product inspection image acquisition and inspection method in step S50 is: acquiring aligned RGB images and depth images of products one by one; after acquisition, inputting the RGB images and depth images into corresponding input terminals of the improved YOLOv8 model respectively.

5. An eLEAP OLED adhesive bubble detection system, characterized in that: include: The acquisition unit is used to collect data sets after aligning the RGB image with the depth image; The dataset annotation unit is used to identify various objects, features or regions in the dataset after annotation, and assign corresponding labels to them; Model training unit, used to initialize the model and then use the improved YOLOv8 model for multiple training optimization and hyperparameter adjustment; A detection device for collecting aligned RGB images and depth images of products one by one; After acquisition, the RGB image and depth image are input into the corresponding input terminals of the improved YOLOv8 model respectively.

6. A computer storage medium storing computer instructions, wherein the computer instructions, when called, are used to execute the eLEAP OLED adhesive bubble detection method according to any one of claims 1 to 4.

7. An electronic device, wherein: The electronic device comprises: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the eLEAP OLED back glue bubble detection method according to any one of claims 1 to 4.

Images