NLED / QLED / OLED pixel defect detection method and system based on feature pyramid network FPN

Abstract

The present invention relates to a NLED / QLED / OLED pixel defect detection method and system based on feature pyramid network FPN, the method comprising: obtaining the original image of the printed pixel panel to be tested in blocks; preprocessing the block original image; splicing all the block images of the same panel; establishing a sample database for the spliced ​​images, using normal images as training images, different types of defective images and normal images as test images; selecting normal images, using the pre-trained ResNet‑18 model as the backbone of the training network, and modeling the feature distribution of normal images; using the network model with the same architecture as the pre-trained network as the test network, inputting the test image, and performing data testing through feature pyramid matching; obtaining the defect detection result through the abnormal score between the test image and the normal image, judging whether the test image has defects, and if so, judging it as a defective product. The method and system improve the efficiency and accuracy of detection, reduce the detection cost, and thus improve production efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of display panel defect detection, and specifically relates to a NLED / QLED / OLED pixel defect detection method and system based on a feature pyramid network FPN. Background Art

[0002] Nano crystal Light-Emitting Diode (NLED) is an electroluminescent or photoluminescent device made of nano-luminescent materials; Quantum dot Light-Emitting Diode (QLED) and OLED have similar structures and their manufacturing methods are also compatible with each other; Organic Light-Emitting Diode (OLED) is an electroluminescent device based on organic materials. When making NLED / QLED / OLED devices, the inkjet head of the inkjet printing equipment is used to print ink into the pixel pits of the glass substrate to form a thin film of material on the device. The differences in ink, substrate, and printing process in the inkjet printing pixel process make it inevitable that various defects will appear on the panel, which will directly affect the display efficiency, lifespan, and price of the display.

[0003] Describe the types of defects. With the continuous development of defect detection technology in display panels, experts, scholars and enterprises have conducted in-depth exploration in this area, and defect detection technology has gradually begun to move from theoretical academics to practical projects. Common pixel film integrity defects of NLED / QLED / OLED mainly include: satellite points on the barrier between pixel banks, incorrect ink colors and ink color mixing deposited in adjacent pixel banks, as well as ink voids, pixel banks that are not fully filled, pixel banks that are overfilled, etc. There are several methods for detecting pixel defects: (1) Using the average pixel values ​​of adjacent pixels of different colors and the specified color pixel threshold to determine whether the pixel to be detected is defective. This method has richer theoretical support and does not require a training process, but the parameters are too variable and the stability is not strong. (2) Detecting pixel defects based on periodic extraction. This method is simple and effective, and has high detection accuracy for scenes with highly controllable acquisition environments. When the scene or target changes, the effect will be very different. (3) Methods based on big data and neural networks automatically learn image features from large amounts of data and fit mathematical models to achieve pixel defect detection. This type of method has high accuracy in identifying defects and strong adaptability, and can also achieve good detection results for many defects that are difficult to identify. This method requires a lot of samples and a lot of manual annotation of defect data. When the data is insufficient, underfitting problems will occur, and it cannot meet the needs of small samples. Defect detection in printed pixel film formation is difficult to train with large amounts of data due to the lack of experimental samples.

[0004] Therefore, in the process of NLED / QLED / OLED printed pixel defect detection, there are often problems such as weak detection stability, weak versatility, and lack of training data, which greatly reduces the effectiveness of pixel defect detection and allows defective printed panels to flow into subsequent steps, resulting in reduced product yield and cost losses. Summary of the invention

[0005] The purpose of the present invention is to provide a NLED / QLED / OLED pixel defect detection method and system based on a feature pyramid network FPN, which improves the efficiency and accuracy of detection, reduces the detection cost, and thus improves production efficiency.

[0006] To achieve the above object, the technical solution adopted by the present invention is: a NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN, comprising the following steps:

[0007] Step S1: using an image acquisition device to acquire the original image of the printed pixel panel to be tested in blocks;

[0008] Step S2: preprocessing the block original image obtained in step S1;

[0009] Step S3: After receiving the preprocessed image, all the block images of the same panel are stitched together using the SIFT algorithm;

[0010] Step S4: establishing a sample database for the spliced ​​images, using defect-free pixel images, i.e. normal images, as training images, and images with different types of defects and some normal images as test images;

[0011] Step S5: Select a non-defective pixel image, use the ResNet-18 model pre-trained on the ImageNet dataset as the backbone of the training network, and model the feature distribution of the non-defective pixel image;

[0012] Step S6: Use the network model with the same architecture as the pre-trained network in step S5 as the test network, input the test image, and perform data testing through feature pyramid matching;

[0013] Step S7: Obtain the defect detection result through the abnormality score between the test image and the normal image to determine whether the test image has defects. If so, it is determined as a defective product, otherwise it proceeds to the subsequent process.

[0014] Furthermore, in step S1, the image acquisition device includes:

[0015] The inkjet printing device is used to input the luminescent material into the nozzle liquid tube, receive it using the bank of the display panel, and provide a defect detection object; the industrial CCD camera is located above the display panel and is used to continuously collect inkjet printed block pixel images; the movable monitoring platform is used to fix the printed display panel after inkjet printing and move the display panel according to a preset step length; the computer is connected to the detection platform and the camera to control the platform operation speed and analyze the block pixel images collected by the camera.

[0016] Furthermore, in step S2, the acquired block original image is subjected to histogram equalization to improve the image brightness and enhance the details; and an image enhancement algorithm is used to perform an enhancement transformation on the equalized image to achieve the expansion of the image sample.

[0017] Furthermore, in step S2, an image enhancement algorithm is used to enhance the equalized image, including uniformly cropping the pixel image into a picture of 256*256 pixels, and then using random factors to flip, rotate, translate, and transform the noise of the image to expand and enhance the image data.

[0018] Furthermore, in step S3, the collected block inkjet printed pixel images are spliced ​​by SIFT algorithm, and images with certain overlap and difference and images with extremely high similarity but marked are accurately spliced ​​to achieve fusion of complete matching images; the implementation method of SIFT algorithm is:

[0019] S31: Construct scale space; the scale space of an image is the representation of an image at different resolutions. An image can generate multiple sets of Octave images, and a set of images includes multiple layers of images. The method of constructing scale space is to construct a Gaussian pyramid, with the original image as the bottom layer, and then Gaussian blur the image and downsample it as the next layer of image, and repeat the cycle. Scale the image to meet the scale invariance of feature points, retain image contours and details, and use the DOG function for calculation:

[0020] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0021]

[0022] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ);

[0023] Where I(x,y) represents the input image, G(x,y,σ) represents the Gaussian function, σ is the scale space factor, * is the convolution symbol, and L(x,y,σ) is the output of the Gaussian transform. In order to effectively detect the extreme values ​​in the scale space, the Gaussian difference (DOG) function, i.e., D(x,y,σ), is convolved with the image I(x,y) to obtain D(x,y,σ), where k represents the scale factor. Multiplying σ by a scale factor k gives a new scale space factor kσ.

[0024] S32: Select key points; first, local extreme points. Feature points are composed of local extreme points in DOG space. Compare the eigenvalues ​​of each pixel point with all its neighboring points. If the eigenvalue of a pixel point is greater than the eigenvalues ​​of its surrounding pixels, it is determined to be a feature point of the region, i.e., a local extreme point. Then remove edge response points. Since the DOG function has a strong edge response at the edge of the image, the edge response is excluded. The peak point of the DOG function has a larger principal curvature in the edge direction and a smaller principal curvature in the direction perpendicular to the edge. The principal curvature is obtained by calculating the 2×2 Hessian matrix at the point position scale, and the derivative is estimated by the adjacent difference of the sampling points:

[0025]

[0026] Among them, D xxIt means that the image of a certain scale in the DOG pyramid is derived twice in the x direction. Similarly, D yy Derived twice in the y direction, D xy It means to take the derivative once in the x and y directions respectively;

[0027] S33: Direction assignment of key points; find the extreme points through scale invariance to make them invariant to scaling, and use the gradient direction distribution characteristics of the key point neighborhood pixels to assign direction parameters to each key point, so that the descriptor is invariant to image rotation, and assign directions to the extreme points by finding the gradient of each extreme point; the gradient of the pixel point is expressed as:

[0028]

[0029] Gradient Magnitude:

[0030]

[0031] Gradient direction:

[0032]

[0033] Where L is the scale of the key point;

[0034] S34: Generate a feature point descriptor; the descriptor is represented by a 2×2×8 dimensional vector, that is, a direction histogram of 2×2×8 directions, and then calculate the gradient direction histogram of 8 directions in a 4×4 window, draw the accumulation of each gradient direction to form a seed point, and a feature point is composed of information of 4 seed points;

[0035] S35: key point matching; after the SIFT feature vectors of the two images are generated, a key point descriptor set is established, and the target is identified by comparing the key point descriptors in the two point sets. The Euclidean distance is used to measure the similarity of the 128-dimensional key point descriptors. The shorter the Euclidean distance, the better the matching degree of the two feature points.

[0036] Furthermore, in step S4, a sample database is established, a certain number of normal images are extracted from the sample database as training images, and 20% of different defect images and normal images are each extracted as test images.

[0037] Furthermore, in step S5, the feature distribution of the normal image is modeled, specifically using ResNet-18 as the backbone, and using the three blocks at the bottom thereof, namely conv2_x, conv3_x, and conv4_x, as feature pyramid extraction to extract the feature distribution of the normal image;

[0038] Given a normal image training set X = [x1, x2, ..........., xn ] captures the normal data manifold by matching the features extracted by the L bottom layers with the corresponding features of the test network; for an input image x K ∈R h×w×c , where h, w, and c represent the height, width, and number of channels respectively. The feature maps of the training image and the test image output by the Lth bottom layer group are and where h l 、w l ,d l Represent the height, width and number of channels of the feature image respectively; the image is input through the training network, which densely extracts features of local image regions.

[0039] Furthermore, in step S6, the specific method of performing data testing through feature pyramid matching is: the test network has the same structure as the training network, but the parameters are randomized, so as to perform NLED / QLED / OLED pixel defect detection.

[0040] Furthermore, in step S7, in the test phase, the goal is to obtain an anomaly map Ω with a size of w×h about the test image, and the test image sample set is defined as: Y = [y1, y2,......., y n ], test image y K ∈R h×w×c ; Fraction Ω ij ∈[0,1] represents the degree of deviation of the pixel at position (i,j) from the training data manifold, using F t l (y) and F s l (y) represents the feature map generated by the Lth bottom layer of the training and test networks, respectively, and a size of w is calculated. l ×h l The abnormal graph Ω l (y), where is the loss score of position (i,j), the anomaly map Ω l (y) The image is upsampled to w×h size through bilinear interpolation. The final abnormal image is the product of L equal-sized upsampled abnormal image elements:

[0041]

[0042] The defect detection result is obtained by the anomaly score between the test image and the normal image features. If any area of ​​a test image is abnormal, the image is designated as a defective image, and the higher the anomaly score, the larger the defective area; therefore, the maximum anomaly score, max(Ω(y)), is selected as the anomaly score of the test image y.

[0043] The present invention also provides an NLED / QLED / OLED pixel defect detection system based on a feature pyramid network FPN, comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects: the present invention can intelligently detect inkjet-printed NLED / QLED / OLED pixel defects. For defect detection of large-size panels, an algorithm for high-precision splicing of multiple panel images collected in blocks is used, so that the inkjet printing effect of the entire printed panel can be detected, which improves the efficiency and accuracy of detection, reduces the detection cost, objectively reduces the labor intensity of personnel, and provides beneficial possibilities for the automation level of the industry. Through statistical comprehensive consideration of defect categories, sizes and their positional relationships, it can provide guidance for the improvement of production processes, reduce the defective rate of production, and reduce the workload of review, thereby optimizing the production efficiency of the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0046] Figure 2 is a schematic diagram of the implementation principle of the image acquisition device in an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of the working principle of inkjet printing in an embodiment of the present invention;

[0048] Figure 4 is a block diagram of an implementation of a SIFT algorithm for image stitching in an embodiment of the present invention;

[0049] Figure 5 It is an integrated block diagram of an algorithm for detecting an image by a computer in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0051] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0052] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0053] Step S1: using an image acquisition device to acquire the original image of the printed pixel panel to be tested in blocks.

[0054] Step S2: preprocessing the divided original image obtained in step S1.

[0055] Step S3: After receiving the preprocessed image, all the block images of the same panel are stitched together using the SIFT algorithm.

[0056] Step S4: Establish a sample database for the spliced ​​images, use sufficient defect-free pixel images, i.e., normal images, as training images, and different types of defect images and some normal images as test images.

[0057] Step S5: Select defect-free pixel images and use the ResNet-18 model pre-trained on the ImageNet dataset as the backbone of the training network to model the feature distribution of defect-free pixel images.

[0058] Step S6: Use the network model with the same architecture as the pre-trained network in step S5 as the test network, input the test image, and perform data testing through feature pyramid matching.

[0059] Step S7: Obtain the defect detection result through the abnormality score between the test image and the normal image to determine whether the test image has defects. If so, it is determined as a defective product, otherwise it proceeds to the subsequent process.

[0060] In this embodiment, in step S1, the image acquisition device includes:

[0061] The inkjet printing device is used to input the luminescent material into the nozzle liquid tube, receive it using the bank of the display panel, and provide a defect detection object; the industrial CCD camera is located above the display panel and is used to continuously collect inkjet printed block pixel images; the movable monitoring platform is used to fix the printed display panel after inkjet printing and can move the display panel according to a preset step length; the computer is connected to the detection platform and the camera to control the platform operation speed and analyze the block pixel images collected by the camera.

[0062] In this embodiment, in step S2, the acquired block original image is subjected to histogram equalization to increase the image brightness and enhance the details; and an image enhancement algorithm is used to perform appropriate enhancement transformation on the equalized image to achieve the expansion of the image sample.

[0063] In this embodiment, in step S2, an image enhancement algorithm is used to perform appropriate enhancement transformation on the equalized image, including first uniformly cropping the pixel image into a picture of 256*256 pixels, and then using random factors to flip, rotate, translate, and transform the noise of the image to expand and enhance the image data.

[0064] In this embodiment, in step S3, the collected block inkjet printed pixel images are spliced ​​by SIFT algorithm, and images with certain overlap and difference and images with extremely high similarity but marked are accurately spliced ​​to achieve fusion of complete matching images; the implementation method of SIFT algorithm is:

[0065] S31: Construct scale space; the scale space of an image is the representation of an image at different resolutions. An image can generate multiple sets of Octave images, and a set of images includes multiple layers of images. The method of constructing scale space is to construct a Gaussian pyramid, with the original image as the bottom layer, and then Gaussian blur the image and downsample it as the next layer of image, and repeat the cycle. Scale the image to meet the scale invariance of feature points, retain image contours and details, and use the DOG function for calculation:

[0066] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0067]

[0068] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ);

[0069] Where I(x,y) represents the input image, G(x,y,σ) represents the Gaussian function, σ is the scale space factor, * is the convolution symbol, and L(x,y,σ) is the output of the Gaussian transform. In order to effectively detect the scale space extreme values, the Gaussian difference (DOG) function, i.e. D(x,y,σ), is convolved with the image I(x,y) to obtain D(x,y,σ), where k represents the scale factor. Multiply σ by a scale factor k to obtain a new scale space factor kσ.

[0070] S32: Select key points; first, local extreme points. Feature points are composed of local extreme points in DOG space. Compare the eigenvalues ​​of each pixel point with all its neighboring points to see whether it is larger or smaller than the eigenvalues ​​of its neighboring points in the image domain. If the eigenvalue of the pixel point is larger than the eigenvalues ​​of the surrounding pixels, it can be determined as the feature point of the area, that is, the local extreme point; then remove the edge response points. Since the DOG function has a strong edge response at the edge of the image, the edge response is excluded. The peak point of the DOG function has a larger principal curvature in the edge direction and a smaller principal curvature in the direction perpendicular to the edge. The principal curvature can be obtained by calculating the 2×2 Hessian matrix at the point position scale, and the derivative is estimated by the adjacent difference of the sampling points:

[0071]

[0072] Where D xx It means that the image of a certain scale in the DOG pyramid is derived twice in the x direction. Similarly, D yy Derived twice in the y direction, D xy It means taking the derivative once in the x and y directions respectively.

[0073] S33: Direction assignment of key points; find the extreme points through scale invariance to make them invariant to scaling, and use the gradient direction distribution characteristics of the key point neighborhood pixels to assign direction parameters to each key point, so that the descriptor is invariant to image rotation, and assign directions to the extreme points by finding the gradient of each extreme point; the gradient of the pixel point is expressed as:

[0074]

[0075] Gradient Magnitude:

[0076]

[0077] Gradient direction:

[0078]

[0079] Where L is the scale of the keypoint.

[0080] S34: Generate a feature point descriptor; the descriptor is represented by a 2×2×8 dimensional vector, that is, a direction histogram of 2×2×8 directions, and then calculate the gradient direction histogram of 8 directions in a 4×4 window, and draw the accumulation of each gradient direction to form a seed point. A feature point is composed of the information of 4 seed points.

[0081] S35: key point matching; after the SIFT feature vectors of the two images are generated, a key point descriptor set is established, and the target is identified by comparing the key point descriptors in the two point sets. The Euclidean distance is used to measure the similarity of the 128-dimensional key point descriptors. The shorter the Euclidean distance, the better the matching degree of the two feature points.

[0082] In this embodiment, in step S4, a sample database is established, and any sufficient normal image samples are extracted from the sample database as training images, and 20% of different defect and normal image types are each extracted as test images.

[0083] In this embodiment, in step S5, the feature distribution of the normal image is modeled, specifically using ResNet-18 as the backbone, and using its bottom three blocks, namely conv2_x, conv3_x, and conv4_x, as feature pyramid extraction to extract the feature distribution of the normal image.

[0084] In this embodiment, in step S5, the purpose of feature extraction is to obtain a model that can imitate fixed output on a normal image. Given a training set of normal images X = [x1, x2, ..........., x n ] captures the normal data manifold by matching the features extracted by the L bottom layers with the corresponding features of the test network. For an input image x K ∈R h×w×c , where h, w, and c represent the height, width, and number of channels respectively. The feature maps of the training image and the test image output by the Lth bottom layer group are and where h l 、w l ,d l Represent the height, width and number of channels of the feature image respectively. The image is input through the training network, which densely extracts features of local image regions.

[0085] In this embodiment, in step S6, the specific method of performing data testing through feature pyramid matching is: the test network has the same structure as the training network, but the parameters are randomized, so as to perform NLED / QLED / OLED pixel defect detection.

[0086] In this embodiment, in step S7, in the test phase, the goal is to obtain an abnormal graph Ω with a test image size of w×h, and the test image sample set is defined as: Y = [y1, y2,......., y n ], test image y K ∈R h×w×c . Fraction Ω ij∈[0,1] represents the degree of deviation of the pixel at position (i,j) from the training data manifold, using and Represents the feature maps generated by the Lth bottom layer of the training and test networks, respectively, and a size of w can be calculated l ×h l The abnormal graph Ω l (y), where is the loss score of position (i,j), the anomaly map Ω l (y) The image is upsampled to w×h size through bilinear interpolation. The final abnormal image is the product of L equal-sized upsampled abnormal image elements:

[0087]

[0088] The defect detection result is obtained by the anomaly score between the test image and the normal image features. If any area of ​​a test image is abnormal, then the image is designated as a defective image, and the higher the anomaly score, the larger the defective area. Therefore, the maximum anomaly score, max(Ω(y)), is selected as the anomaly score of the test image y.

[0089] This embodiment also provides an NLED / QLED / OLED pixel defect detection system based on a feature pyramid network FPN, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0090] Please refer to Figure 2 , the image acquisition device includes:

[0091] Inkjet printing device 201 is used to print NLED / QLED / OLED pixels and provide detection objects. For specific implementation process, please refer to Figure 3 , the inkjet head injects ink droplets into the substrate, while the substrate moves in a two-dimensional space, and the nozzle is controlled by a digital pulse driver.

[0092] The industrial CCD camera 202 is disposed above the movable monitoring platform and is used to collect images to be tested.

[0093] The movable monitoring platform 203 is used to fix the printed display panel after inkjet printing, and can move the display panel according to a preset step length, and its operation speed is controlled by a computer.

[0094] The computer control device 204 is used to receive the image to be tested acquired by the camera, analyze and process the image acquired by the camera, and control the operation speed of the movable monitoring platform.

[0095] The computer controls the device to process the block-by-block acquired print pixel panel image and output the operation results. The specific implementation steps are as follows: Figure 1 As shown, it has been described in detail above and will not be repeated here.

[0096] Please refer to Figure 4 , is a block diagram of the SIFT algorithm used for high-precision image stitching in step S3 of this embodiment. The specific implementation process has been described in detail in the embodiment of step S3 and will not be repeated here.

[0097] Please refer to Figure 5 , is an integrated block diagram of the algorithm for image processing by the computer in this embodiment, and the specific algorithm flow is as follows:

[0098] After receiving the original segmented image in step S4, the image is passed through a pre-trained classification model network composed of a Unet network to detect pixel images with defects, and then passed through a positioning network module with resnet-18 as the backbone network to locate the position coordinates of the defective area and the mask binary mask image, and output a defect positioning image.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN, characterized in that: The following steps are involved: Step S1: using an image acquisition device to acquire the original image of the printed pixel panel to be tested in blocks; Step S2: preprocessing the block original image obtained in step S1; Step S3: After receiving the preprocessed image, all the block images of the same panel are stitched together using the SIFT algorithm; Step S4: establishing a sample database for the spliced ​​images, using defect-free pixel images, i.e. normal images, as training images, and images with different types of defects and some normal images as test images; Step S5: Select a non-defective pixel image, use the ResNet-18 model pre-trained on the ImageNet dataset as the backbone of the training network, and model the feature distribution of the non-defective pixel image; Step S6: Use the network model with the same architecture as the pre-trained network in step S5 as the test network, input the test image, and perform data testing through feature pyramid matching; Step S7: Obtain the defect detection result by the abnormal score between the test image and the normal image, and judge whether the test image has defects. If so, it is determined as a defective product, otherwise it enters the subsequent process; In step S3, the collected block inkjet printed pixel images are spliced ​​by SIFT algorithm, and images with certain overlap and difference and images with extremely high similarity but marked are accurately spliced ​​to achieve fusion of complete matching images; the implementation method of SIFT algorithm is: S31: Construct scale space; the scale space of an image is the representation of an image at different resolutions. An image can generate multiple sets of Octave images, and a set of images includes multiple layers of images. The method of constructing scale space is to construct a Gaussian pyramid, with the original image as the bottom layer, and then Gaussian blur the image and downsample it as the next layer of image, and repeat the cycle. Scale the image to meet the scale invariance of feature points, retain image contours and details, and use the DOG function for calculation: L(x,y,σ)=G(x,y,σ)*I(x,y) D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ); Where I(x,y) represents the input image, G(x,y,σ) represents the Gaussian function, σ is the scale space factor, * is the convolution symbol, and L(x,y,σ) is the output of the Gaussian transform. In order to effectively detect the extreme values ​​in the scale space, the Gaussian difference (DOG) function, i.e., D(x,y,σ), is convolved with the image I(x,y) to obtain D(x,y,σ), where k represents the scale factor. Multiplying σ by a scale factor k gives a new scale space factor kσ. S32: Select key points; first, local extreme points. Feature points are composed of local extreme points in DOG space. Compare the eigenvalues ​​of each pixel point with all its neighboring points. If the eigenvalue of a pixel point is greater than the eigenvalues ​​of its surrounding pixels, it is determined to be a feature point of the region, i.e., a local extreme point. Then remove edge response points. Since the DOG function has a strong edge response at the edge of the image, the edge response is excluded. The peak point of the DOG function has a larger principal curvature in the edge direction and a smaller principal curvature in the direction perpendicular to the edge. The principal curvature is obtained by calculating the 2×2 Hessian matrix at the point position scale, and the derivative is estimated by the adjacent difference of the sampling points: Among them, D xx It means that the image of a certain scale in the DOG pyramid is derived twice in the x direction. Similarly, D yy Derived twice in the y direction, D xy It means to take the derivative once in the x and y directions respectively; S33: Direction assignment of key points; find the extreme points through scale invariance to make them invariant to scaling, and use the gradient direction distribution characteristics of the key point neighborhood pixels to assign direction parameters to each key point, so that the descriptor is invariant to image rotation, and assign directions to the extreme points by finding the gradient of each extreme point; the gradient of the pixel point is expressed as: Gradient Magnitude: Gradient direction: Where L is the scale of the key point; S34: Generate a feature point descriptor; the descriptor is represented by a 2×2×8 dimensional vector, that is, a direction histogram of 2×2×8 directions, and then calculate the gradient direction histogram of 8 directions in a 4×4 window, draw the accumulation of each gradient direction to form a seed point, and a feature point is composed of information of 4 seed points; S35: key point matching; after the SIFT feature vectors of the two images are generated, a key point descriptor set is established, and the target is identified by comparing the key point descriptors in the two point sets. The Euclidean distance is used to measure the similarity of the 128-dimensional key point descriptors. The shorter the Euclidean distance, the better the matching degree of the two feature points.

2. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 1, characterized in that: In step S1, the image acquisition device includes: The inkjet printing device is used to input the luminescent material into the nozzle liquid tube, receive it using the bank of the display panel, and provide a defect detection object; the industrial CCD camera is located above the display panel and is used to continuously collect inkjet printed block pixel images; the movable monitoring platform is used to fix the printed display panel after inkjet printing and move the display panel according to a preset step length; the computer is connected to the detection platform and the camera to control the platform operation speed and analyze the block pixel images collected by the camera.

3. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 1, characterized in that: In the step S2, the acquired block original image is subjected to histogram equalization to improve the image brightness and enhance the details; and an image enhancement algorithm is used to perform an enhancement transformation on the equalized image to achieve the expansion of the image sample.

4. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 3, characterized in that: In step S2, an image enhancement algorithm is used to enhance the equalized image, including uniformly cropping the pixel image into a picture of 256*256 pixels, and then using random factors to flip, rotate, translate, and transform the noise of the image to expand and enhance the image data.

5. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 1, characterized in that: In step S4, a sample database is established, a certain number of normal images are extracted from the sample database as training images, and 20% of the images with different defects and normal images are each extracted as test images.

6. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 1, characterized in that: In the step S5, the feature distribution of the normal image is modeled, specifically using ResNet-18 as the backbone, and using the three blocks at the bottom thereof, namely conv2_x, conv3_x, and conv4_x, as feature pyramid extraction to extract the feature distribution of the normal image; Given a normal image training set X = [x1, x2, ..........., x n ], by matching the features extracted from the Q underlying layers with the features corresponding to the test network to capture the normal data manifold; for the input image x K ∈R h×w×c , where h, w, and c represent the height, width, and number of channels respectively. The feature maps of the training image and the test image output by the lth bottom layer are and where h l 、w l ,d l Represent the height, width and number of channels of the feature image respectively; the image is input through the training network, which densely extracts features of local image regions.

7. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 1, characterized in that: In step S6, the specific method of performing data testing through feature pyramid matching is: the test network has the same structure as the training network, but the parameters are randomized, so as to perform NLED / QLED / OLED pixel defect detection.

8. The NLED / QLED / OLED pixel defect detection method based on feature pyramid network FPN according to claim 1, characterized in that: In step S7, during the testing phase, the goal is to obtain an anomaly map Ω with a size of w×h for the test image, and the test image sample set is defined as: Y = [y1, y2, ......., y n ], test image y K ∈R h×w×c ; Fraction Ω ij ∈[0,1] represents the degree of deviation between the pixel at position (i,j) and the training data stream, using and Represents the feature maps generated by the lth bottom layer of the training and test networks, respectively, and calculates a size of w l ×h l The abnormal graph Ω l (y), where is the loss score of position (i,j), the anomaly map Ω l (y) The image is upsampled to w×h size through bilinear interpolation. The final abnormal image is the product of Q upsampled abnormal image elements of equal size: The defect detection result is obtained by the anomaly score between the test image and the normal image features. If any area of ​​a test image is abnormal, the image is designated as a defective image, and the higher the anomaly score, the larger the defective area; therefore, the maximum anomaly score, max(Ω(y)), is selected as the anomaly score of the test image y.

9. A NLED / QLED / OLED pixel defect detection system based on feature pyramid network FPN, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps as described in any one of claims 1 to 8 can be implemented.

Images