A lamp bead panel defect detection method and system based on a LineMod algorithm

Abstract

The application relates to the technical field and discloses a lamp bead panel defect detection method and system based on a LineMod algorithm, which comprises the following steps: S1, collecting a template image of a lamp bead panel and performing pyramid down-sampling on the template image; S2, performing gradient calculation on color channels of the down-sampled template image, so that gradient directions of the image are quantized into feature vectors in a plurality of directions; S3, traversing each layer of the pyramid of the template image to obtain feature point positions of feature points in each layer; S4, according to the feature point positions, calculating diffusion of the quantized feature vectors in each direction in the neighborhood of the feature vectors, and taking the diffusion of the feature vectors in each direction in the neighborhood of the feature vectors as template feature information; S5, preparing a pre-response table according to the template feature information; and S6, performing defect detection on a lamp bead panel image to be detected according to the pre-response table. The application solves the problem of low matching speed of the existing defect detection technology and has the characteristics of high precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of lamp bead panel intelligent detection, more particularly to a lamp bead panel defect detection method and system based on LineMod algorithm and normalized correlation coefficient matching method. BACKGROUND

[0002] MiniLED lamp bead panels are widely used in display fields such as televisions and displays, have the characteristics of high efficiency, low power consumption, high stability, mature technology, etc., and are an important choice for the next generation of mainstream display technology.

[0003] In the production process of MiniLED lamp bead panels, lamp bead missing and misalignment may inevitably occur. Artificial detection is affected by light, environment and physical factors, and the evaluation criteria for defects vary from person to person, and it is time-consuming and laborious. Using a computer for detection not only avoids human subjective influence and improves detection accuracy, but also frees the hands of workers.

[0004] LineMod algorithm is a method based on template matching; it extracts and saves the features of the gradient and normal vector direction of each viewpoint on the image and encodes them to help realize defect detection in industry.

[0005] The prior art has a cigarette package appearance detection method based on deep self-learning using RANSAC algorithm for matching and alignment, but this requires small image noise to accurately find the corresponding points for alignment, and the calculation amount is relatively large and the time consumption is relatively long.

[0006] The prior art also has a labeling positioning method based on LineMod matching and Hough transform, which uses LineMod algorithm for preliminary positioning and Hough line transform for fine positioning, mainly for positioning before labeling, while LineMod in the present method is used for image alignment to avoid false detection, and after alignment, template matching is performed to find the defect.

[0007] However, the existing defect detection technology still has the problem of low matching speed, and how to invent a lamp bead panel defect detection method with high matching speed is a technical problem that needs to be solved in this technical field. SUMMARY

[0008] The present application provides a lamp bead panel defect detection method and system based on LineMod algorithm to solve the problem of low matching speed of existing defect detection technology, which has the characteristics of high precision.

[0009] To achieve the above-mentioned purpose of the present application, the technical solutions adopted are as follows:

[0010] A lamp bead panel defect detection method based on LineMod algorithm, comprising the following steps:

[0011] S1, collecting a template image of a lamp bead panel, and performing pyramid down-sampling on the template image;

[0012] S2, calculating the gradient of the color channel of the down-sampled template image, so as to quantize the gradient direction of the image into a plurality of direction feature vectors;

[0013] S3, traversing each layer of the pyramid of the template image to obtain the feature point position of each layer of feature points;

[0014] S4, calculating the diffusion of each direction feature vector in its neighborhood according to the feature point position, and taking the diffusion of each direction feature vector in its neighborhood as the template feature information;

[0015] S5, preparing a pre-response table according to the template feature information;

[0016] S6, performing defect detection on the lamp bead panel image to be detected according to the pre-response table.

[0017] Preferably, in step S2, the gradient direction of the image is quantized into a plurality of direction feature vectors, specifically: the gradient with the maximum modulus in the three channels is represented by binary, and each bit in the binary represents a discrete direction, so that the gradient direction of the image is quantized into eight direction feature vectors.

[0018] Further, the pre-response table saves the maximum score of the discrete direction corresponding to each pixel value of each position of the template image.

[0019] Further, the pre-response table uses the SSE technology in the SIMD instruction set to accumulate the template image in parallel, so as to obtain the scores of all positions on the lamp bead panel image to be detected.

[0020] Further, in step S6, the lamp bead panel image to be detected is detected according to the pre-response table, and the specific steps are as follows:

[0021] S6.1, performing pyramid down-sampling on the lamp bead panel image to be detected and performing gradient calculation;

[0022] S6.2, calculating the feature points in the pre-response table that are most similar to the lamp bead panel image to be detected;

[0023] S6.3, performing template matching and obtaining the target defect area.

[0024] Further, in step S5.2, when calculating the feature point in the pre-response table that is most similar to the gradient of the image of the lamp bead panel to be detected, the similarity calculation method is:

[0025] ε({L m} m∈M , Γ, c) = ∑ (r，m)∈P |max t∈R(c+r) f m (O m (r), L m (t))|

[0026] Wherein, ε represents similarity, Γ represents template feature set collection formula, L m represents the image of the lamp bead panel to be detected, including all images M and a certain image m used for matching, c represents each position in an image, f m represents a function for calculating the similarity between the position r of the template image and the position of the lamp bead panel to be detected, that is, a similarity function for calculating the cosine value of the included angle or the normal vector direction included angle of the gradient direction of the template image and the lamp bead panel to be detected; R represents a window region, represents a region with a size of T centered at the position c+r of the input image L m t represents a position of the region R.

[0027] Further, in step S5.3, template matching is performed, and the specific steps are as follows:

[0028] S6.3.1, define a matching calculation kernel with a matrix size of b×b;

[0029] S6.3.2, divide the template image and the image to be matched with a matrix size of m×n into (m-b+1)×(n-b+1) template sub-images and image sub-images to be matched based on the matching calculation kernel;

[0030] S6.3.3, perform mean removal operation on the divided template sub-image and image sub-image to be matched, and vectorize the square matrix of the template sub-image and image sub-image to be matched;

[0031] S6.3.4, define a monitoring area; in the detection area, calculate the cosine value of the template sub-image and the image sub-image to be matched, and mark the area that does not meet the set threshold, so as to find the defect of the image sub-image to be matched.

[0032] Further, when detecting the lamp bead panel image to be detected, if the matching similarity ε is higher than the threshold τ, it is determined that the matching is successful, the primary template matching stage is completed, and the trained feature vector and feature point position are returned. Finally, the training information and position of the entire template are returned.

[0033] Further, the square matrix to be matched sub-image and template sub-image is vectorized, specifically:

[0034]

[0035]

[0036] Wherein, I(x,y) represents the gray value of the pixel point on the image to be matched, T(x,y) represents the gray value of the pixel point on the template image, w represents the width of the detection area, h represents the height of the detection area, x' represents the horizontal coordinate of the detection area, y' represents the vertical coordinate of the detection area, T'(x',y') is the template sub-image, I'(x',y') is the matching sub-image.

[0037] A lamp bead panel defect detection system based on LineMod algorithm, comprising an image initialization module, a gradient calculation module, a feature extraction module, a template feature module and a defect detection module.

[0038] The image initialization module is used to collect the template image of the lamp bead panel, and pyramid down-sampling is performed on the template image.

[0039] The gradient calculation module is used to calculate the gradient of the color channel of the down-sampled template image, so as to quantize the gradient direction of the image into a plurality of direction feature vectors.

[0040] The feature extraction module is used to traverse each layer of the pyramid of the template image to obtain the feature point position of each layer of feature points.

[0041] The template feature module is used to calculate the diffusion of each direction feature vector in its neighborhood according to the feature point position, and take the diffusion of each direction feature vector in its neighborhood as the template feature information.

[0042] The template feature module is used to make a pre-response table according to the template feature information.

[0043] The defect detection module is used to detect the lamp bead panel image to be detected according to the pre-response table.

[0044] The beneficial effects of the present application are as follows:

[0045] This invention discloses a defect detection method for LED chip panels based on the LineMod algorithm. The method utilizes the LineMod algorithm for image alignment, eliminating the need to find corresponding points between a template image and the image to be matched. Instead, it uses the template image, saves its feature information, and then matches it with the image to be matched. Furthermore, the LineMod algorithm is simple to apply and fast. Therefore, this invention solves the problem of low matching speed in existing defect detection technologies and offers high accuracy. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a defect detection method for LED bead panels based on the LineMod algorithm of the present invention.

[0047] Figure 2 This is a schematic diagram of the application process of the LED panel defect detection method based on the LineMod algorithm of the present invention in Example 2.

[0048] Figure 3 It is the template function of the LED panel defect detection method based on the LineMod algorithm of the present invention in Example 2.

[0049] Figure 4 This is a schematic diagram of gradient direction quantization for a defect detection method for LED bead panels based on the LineMod algorithm of the present invention.

[0050] Figure 5 This is a schematic diagram of the directional diffusion of a defect detection method for LED chip panels based on the LineMod algorithm of the present invention.

[0051] Figure 6 This is the defect image to be matched in the LED panel defect detection method based on the LineMod algorithm of the present invention in Embodiment 2.

[0052] Figure 7 This is a template diagram and a schematic diagram of the region of interest for a method for detecting defects in LED panels based on the LineMod algorithm according to the present invention in Embodiment 2.

[0053] Figure 8 This is a schematic diagram of the matching results of the LED panel defect detection method based on the LineMod algorithm of the present invention in Embodiment 2. Detailed Implementation

[0054] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] like Figure 1 As shown, a defect detection method for LED chip panels based on the LineMod algorithm includes the following steps:

[0057] S1, collect template images of the lamp bead panel, pyramid down-sample the template images;

[0058] S2, calculate gradients of color channels of the down-sampled template images, thereby quantizing gradient directions of the images into feature vectors of several directions;

[0059] S3, traverse each layer of the pyramid of the template images, and obtain feature point positions of feature points of each layer;

[0060] S4, according to the feature point positions, calculate diffusion of the quantized feature vectors of each direction in their neighborhoods, and take the diffusion of the feature vectors of each direction in their neighborhoods as template feature information;

[0061] S5, make a pre-response table according to the template feature information;

[0062] S6, perform defect detection on the lamp bead panel images to be detected according to the pre-response table.

[0063] Embodiment 2

[0064] A lamp bead panel defect detection method based on the LineMod algorithm, comprising the following steps:

[0065] S1, collect template images of the lamp bead panel, pyramid down-sample the template images;

[0066] S2, calculate gradients of color channels of the down-sampled template images, thereby quantizing gradient directions of the images into feature vectors of several directions;

[0067] S3, traverse each layer of the pyramid of the template images, and obtain feature point positions of feature points of each layer;

[0068] S4, according to the feature point positions, calculate diffusion of the quantized feature vectors of each direction in their neighborhoods, and take the diffusion of the feature vectors of each direction in their neighborhoods as template feature information;

[0069] S5, make a pre-response table according to the template feature information;

[0070] S6, perform defect detection on the lamp bead panel images to be detected according to the pre-response table.

[0071] In one specific embodiment, as shown in Figure 4 In step S2, the gradient directions of the images are quantized into feature vectors of several directions, specifically: the gradients with the largest modulus in the three channels are represented by binary, and each bit in the binary represents a discrete direction, thereby quantizing the gradient directions of the images into feature vectors of 8 directions.

[0072] In this embodiment, after obtaining the feature vector (gradient) of the feature point, each layer of the template image is traversed to obtain the coordinate position of each layer feature point. After the feature point coordinate correction is calculated by loop, the coordinate position corresponding to the feature point is obtained, and the feature point position and feature vector are saved;

[0073] In this embodiment, as shown in Figure 5 Each direction after quantization is diffused in its TxT neighborhood, and is stored as continuous feature information data in memory.

[0074] In one specific embodiment, the pre-response table saves the maximum score of the discrete direction corresponding to each pixel value of each position of the template image.

[0075] In one specific embodiment, the pre-response table uses the SSE technology in the SIMD instruction set to accumulate the template image in parallel, thereby obtaining the scores of all positions on the lamp bead panel image to be detected.

[0076] In one specific embodiment, in step S6, the lamp bead panel image to be detected is detected according to the pre-response table, and the specific steps are as follows:

[0077] S6.1, pyramid down-sampling the lamp bead panel image to be detected and performing gradient calculation;

[0078] S6.2, calculating the feature point in the pre-response table that is most similar to the lamp bead panel image to be detected;

[0079] S6.3, performing template matching and obtaining the target defect area.

[0080] In one specific embodiment, in step S5.2, when calculating the feature point in the pre-response table that is most similar to the gradient of the lamp bead panel image to be detected, the similarity calculation method is as follows:

[0081] ε({L m} m∈M , Γ, c) = ∑ (r，m)∈P |max t∈R(c+r) f m (O m (r), L m (t))|

[0082] Wherein, ε represents similarity, Γ represents template feature set collection formula, L mrepresents the image of the lamp bead panel to be detected, including all images M and a certain image m used for matching, c represents each position in an image, fm represents a function used for calculating the similarity between the position r of the template image and the position of the lamp bead panel to be detected, i.e. a similarity function for calculating the cosine value of the included angle or the normal vector direction included angle of the gradient direction of the template image and the position of the lamp bead panel to be detected, R represents a window region, represents the input image L m a region with a size of T centered at the position c+r, and t represents a position of the region R.

[0083] In the embodiment, the matching degree ε is compared with the set threshold τ, if the matching similarity ε is higher than the threshold τ, the matching is successful, the primary template matching stage is completed, and the trained feature vector and the feature point position are returned, finally, the training information and the position of the entire template are returned.

[0084] In a specific embodiment, in the step S5.3, the template matching is performed, and the specific steps are as follows:

[0085] S6.3.1, a matching calculation kernel with a matrix size of b×b is defined;

[0086] S6.3.2, the template image and the image to be matched with a matrix size of m×n are divided into (m-b+1)×(n-b+1) template sub-images and sub-images to be matched based on the matching calculation kernel;

[0087] S6.3.3, the divided template sub-images and sub-images to be matched are subjected to a de-meaning operation one by one, and the square matrix vectors of the template sub-images and the sub-images to be matched are obtained;

[0088] S6.3.4, a monitoring region is defined, in the detection region, the cosine value of the template sub-image and the sub-image to be matched is calculated, the region not meeting the set threshold is marked, and thus the defect of the sub-image to be matched is found.

[0089] In a specific embodiment, when the lamp bead panel image to be detected is subjected to defect detection, if the matching similarity ε is higher than the threshold τ, it is determined that the matching is successful, the primary template matching stage is completed, and the trained feature vector and the feature point position are returned, finally, the training information and the position of the entire template are returned.

[0090] In a specific embodiment, the square matrix of the template sub-image and the sub-image to be matched is vectorized, and the specific steps are as follows:

[0091]

[0092]

[0093] Wherein, I(x, y) represents the gray value of the pixel point on the image to be matched, T(x, y) represents the gray value of the pixel point on the template image, w represents the width of the detection area, h represents the height of the detection area, x' represents the horizontal coordinate of the detection area, y' represents the vertical coordinate of the detection area, T'(x', y') is a template sub-image, and I'(x', y') is a matching sub-image.

[0094] In this embodiment, as shown in the formula (1), when detecting defects of the Mini LED lamp bead panel, first, the template picture and the picture to be matched are read, and then the Mini LED lamp bead panel defect detection is matched through the template matching function based on the present application. Figure 2

[0095] As shown in the formula (1), the Mini LED lamp bead panel defect detection is matched through the template matching function based on the present application, and specifically: Figure 3

[0096] First, it is judged whether to enter the matching

[0097] A If mode!= test, enter the template training mode, and execute the following steps.

[0098] A1 pyramid down-sampling is performed on the template image;

[0099] A2 the template layer is rendered to prevent rotation out of the frame;

[0100] A3 the template image is rotated to obtain the template feature points;

[0101] A3.1 the gradient direction of the template image is calculated

[0102] A3.2 the gradient direction is quantized into 8 directions and saved, as shown in the formula (2) Figure 4

[0103] A3.3 each layer of the pyramid of the image is traversed to obtain the coordinates of each layer of feature points, and the coordinates are corrected by looping calculation to retain the feature point position and the feature vector;

[0104] A3.4 the quantized direction is diffused in the neighborhood, and saved in the memory, as shown in the formula (3) Figure 5

[0105] B If mode == test, enter the image matching mode, and execute the following steps.

[0106] B.1 pyramid down-sampling is performed on the image to be matched;

[0107] B.2 the gradient of the image to be matched is generated

[0108] B.3 the table is looked up to find the most similar feature points

[0109] ​​​​B.4 Compute the Region Of Interest (ROI) in the image to be matched, complete the alignment as in Figure 6 With Figure 7 the block (the block can be removed as needed)

[0110] B.4.1 Template matching in the ROI using the NC algorithm, and frame the target defect area, as in Figure 8 the rectangular frame in

[0111] The application discloses a lamp bead panel defect detection method based on a LineMod algorithm. In the application, the LineMod algorithm is used for image alignment, without the need to find corresponding points in a template image and an image to be matched, but by saving feature information of the template image, and then matching the template image with the image to be matched, and the LineMod algorithm is accelerated by using a SIMD instruction set and OpenMP, and is simple to apply and fast. During detection, a normalized correlation coefficient matching method is used for matching and finding defects, the algorithm subtracts the mean value of image matrix pixels, and the cosine value of two vectors is calculated, and only when complete matching is achieved, 1 is output, so that the sensitivity of the image to brightness change is almost reduced to 0. Therefore, the application solves the problem of low matching speed of the existing defect detection technology, and has the characteristics of high precision.

[0112] Embodiment 3

[0113] A lamp bead panel defect detection system based on a LineMod algorithm, comprising an image initialization module, a gradient calculation module, a feature extraction module, a template feature module and a defect detection module.

[0114] The image initialization module is used for collecting a template image of a lamp bead panel, and performing pyramid down-sampling on the template image.

[0115] The gradient calculation module is used for performing gradient calculation on color channels of the down-sampled template image, so as to quantize the gradient direction of the image into feature vectors in a plurality of directions.

[0116] The feature extraction module is used for traversing each pyramid layer of the template image to obtain feature point positions of feature points in each pyramid layer.

[0117] The template feature module is used for calculating diffusion of the quantized feature vectors in each direction in a neighborhood according to the feature point positions, and taking the diffusion of the feature vectors in each direction in the neighborhood as template feature information.

[0118] The template feature module is used for making a pre-response table according to the template feature information.

[0119] The defect detection module is used for detecting defects of the lamp bead panel image to be detected according to the pre-response table.

[0120] Obviously, the above embodiments of the application are only examples for clearly illustrating the application, but not limitation on the embodiments of the application. Any modification, equivalent replacement and improvement made in the spirit and principle of the application should be included in the protection scope of the claims of the application.

Claims

1. A method for detecting defects in LED chip panels based on the LineMod algorithm, characterized in that: Includes the following steps: S1. Collect template images of the LED bead panel and perform pyramid downsampling on the template images; S2. By performing gradient calculation on the color channels of the downsampled template image, the gradient direction of the image is quantized into feature vectors in several directions. S3. Traverse each layer of the pyramid in the template image to obtain the feature point positions of each layer of feature points; S4. Based on the location of the feature points, calculate the diffusion of the quantized feature vector in each direction in its neighborhood, and use the diffusion of the feature vector in each direction in its neighborhood as template feature information. S5. Create a pre-response table based on template feature information; the pre-response table stores the maximum score of each discrete direction corresponding to each pixel value at each position of the template image; the pre-response table uses the SSE technology in the SIMD instruction set to accumulate the template image in parallel, thereby obtaining the scores of all positions on the LED panel image to be detected. S6. Defect detection is performed on the LED panel image to be inspected according to the pre-response table. The specific steps are as follows: S6.1 Perform pyramid downsampling and gradient calculation on the LED panel image to be tested; S6.2 Calculate the feature points in the pre-response table that are most similar to the panel image of the LED bead to be detected; S6.3 Perform template matching to obtain the target defect area. The specific steps are as follows: S6.3.1 Define a matching calculation kernel with a matrix size of b×b; S6.3.

2. The image to be matched and the template image of matrix size m×n are divided into (m-b+1)×(n-b+1) sub-images to be matched and template sub-images based on the matching calculation kernel; S6.3.3 Perform mean removal operation on the segmented sub-images to be matched and the template sub-images one by one, and vectorize the square matrices of the sub-images to be matched and the template sub-images; S6.3.4 Custom monitoring area: Within the detection area, calculate the cosine value of the sub-image to be matched and the template sub-image, and mark the areas that do not meet the set threshold, thereby finding the defects of the sub-image to be matched.

2. The LED panel defect detection method based on the LineMod algorithm according to claim 1, characterized in that: In step S2, the gradient direction of the image is quantized into feature vectors in several directions. Specifically, the gradient is calculated for each of the RGB channels of the template image, and the gradient with the largest magnitude among the three channels is represented in binary. Each bit in the binary represents a discrete direction, thereby quantizing the gradient direction of the image into feature vectors in eight directions.

3. The LED panel defect detection method based on the LineMod algorithm according to claim 1, characterized in that: In step S6.2, when calculating the feature points in the pre-response table that are most similar to the gradient of the LED panel image to be detected, the similarity is calculated as follows: in, Indicates similarity, Formula for collecting template feature sets This represents the image of the LED panel to be detected, including all images M and a specific image m used for matching. Represents the positions in an image, This function represents the similarity between the position r of the template image and the position of the LED panel image to be detected. In other words, it is a similarity function that calculates the cosine value of the angle between the gradient direction of the position of the template image and the position of the LED panel image to be detected, or the angle between the normal vector direction of the two images. Represents a window area. , indicating that the input image Location A region of size T centered at point T. This represents a location within region R.

4. The LED panel defect detection method based on the LineMod algorithm according to claim 1, characterized in that: When performing defect detection on the LED panel image to be inspected, if the matching similarity ε is higher than the threshold τ, the matching is considered successful, the primary template matching stage is completed, and the trained feature vector and feature point positions are returned. Finally, the training information and position of the entire template are returned.

5. The LED panel defect detection method based on the LineMod algorithm according to claim 4, characterized in that: The matrix of the sub-image to be matched and the template sub-image is vectorized as follows: in, Represents the grayscale value of a pixel in the image to be matched. Represents the grayscale value of a pixel in the template image. Indicates the width of the detection area, Indicates the height of the detection area, Represents the x-coordinate of the detection area, Represents the vertical coordinate of the detection area, For template sub-images, To match sub-images.

6. A defect detection system for LED bead panels based on the LineMod algorithm, characterized in that: The method for implementing the method as described in any one of claims 1 to 5 includes an image initialization module, a gradient calculation module, a feature extraction module, a template feature module, and a defect detection module; The image initialization module is used to collect template images of the LED bead panel and perform pyramid downsampling on the template images; The gradient calculation module is used to perform gradient calculation on the color channels of the downsampled template image, thereby quantizing the gradient direction of the image into feature vectors in several directions. The feature extraction module is used to traverse each layer of the pyramid of the template image to obtain the feature point position of each layer of feature points. The template feature module is used to calculate the diffusion of the quantized feature vector in each direction in its neighborhood based on the feature point position, and to use the diffusion of the feature vector in each direction in its neighborhood as template feature information. The template feature module is used to create a pre-response table based on template feature information; The defect detection module is used to perform defect detection on the LED panel image to be inspected based on the pre-response table.

Images