Mini LED backlight panel particle visual positioning and matching method

Abstract

The application discloses a kind of MiniLED backlight panel granule visual positioning and matching method, comprising the following steps: S1: obtaining FOV image data stream;S2: image segmentation: for the image information obtained, each pixel on the image is marked according to gray value, and the target pixel region is extracted according to the mark;S3: contour screening: according to the contour formed by the target pixel region extracted in S2, the contour is screened, and the local image of fixed size is intercepted by the screened contour, and the reference coordinates of the upper left point of the local image on the FOV image are recorded;S4: obtain accurate coordinates: on the multiple local images obtained in S3, respectively using standard shape template is searched, and the sub-pixel level coordinates of the granule on the local image are obtained, and after superimposing the reference coordinates, the detection coordinates of the granule on the whole FOV image are obtained;S5: space matching: obtaining the theoretical coordinates of all granules in the FOV image, find the center point granule coordinates in the detection coordinates.

Description

Technical Field

[0001] This invention relates to the field of miniLED panel defect detection technology, specifically a method for visual positioning and matching of particles in a MiniLED backlight panel. Background Technology

[0002] To achieve better display effects, the number of independently illuminated zones on MiniLED backlight panels is rapidly increasing, and the number of particles on the backlight panel is also growing exponentially from 0.1k / piece to the order of 10k / piece. Manual inspection using microscopes alone is far from sufficient to meet the production capacity of MiniLED backlight panels. Therefore, high-speed, precise automated inspection machines based on machine vision are being widely used on MiniLED backlight panel production lines.

[0003] In the visual inspection of MiniLED backlight panels, over 80% of the inspection content is concentrated on LED and IC particles. However, the size of the LED / IC particles on a MiniLED backlight panel is much smaller than the particle spacing, so the effective detection area in the FOV image typically occupies less than 10% of the entire image. To improve inspection efficiency, the detection algorithms in each detection segment only need to detect the particles and their surrounding small areas; therefore, pre-visual localization of the LED and IC particles is essential.

[0004] The various visual inspection devices for MiniLED backlight panels use large-area area scan cameras or line scan cameras, with each FOV image containing several to hundreds of LED and IC chips. Due to manufacturing and calibration errors in the equipment, there is a discrepancy between the actual coordinates of the chips in the image and the calculated theoretical coordinates. Therefore, after visually locating the chips, it is necessary to match the location coordinates of each chip with its theoretical coordinates before proceeding to the next step of the inspection process.

[0005] The existing positioning and matching methods have the following drawbacks: using a large-format camera to scan and photograph the odd and even columns of LED beads on the MiniLED backlight panel, each scan can only process the image of one column of LED beads, which is inefficient. Summary of the Invention

[0006] The purpose of this invention is to provide a method for visual positioning and matching of MiniLED backlight panel particles to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for visual positioning and matching of MiniLED backlight panel particles, comprising the following steps:

[0009] S1: Obtain the FOV image data stream;

[0010] S2: Image segmentation: For the acquired image information, each pixel on the image is marked according to its grayscale value, and the target pixel region is extracted based on the marking.

[0011] S3: Contour Filtering: Based on the contours formed by the target pixel regions extracted in S2, the contours are filtered, and a local image of a fixed size is cropped through the filtered contours. The reference coordinates of the upper left point of the local image on the FOV image are recorded.

[0012] S4: Obtain precise coordinates: On the multiple local images obtained in S3, a standard shape template is used to search for sub-pixel level coordinates of the particles on the local images. After superimposing the reference coordinates, the detection coordinates of the particles on the entire FOV image are obtained.

[0013] S5: Spatial Matching: Obtain the theoretical coordinates of all particles in the FOV image, find the center point particle coordinates in the detection coordinates, determine the optimal offset between the theoretical coordinates and the center point particle coordinates, and achieve matching between the detection coordinates and the theoretical coordinates based on the optimal offset.

[0014] As a further aspect of the present invention: before step S1, image preprocessing is further included, wherein for each pixel in the original FOV image, the difference between its blue channel grayscale value and its red channel grayscale value is calculated, and the difference is assigned to the pixel at the same coordinates in the preprocessed image.

[0015] As a further aspect of the present invention: in step S2, each pixel on the image is marked according to its grayscale value, and the target pixel region is extracted based on the marking, the method further includes:

[0016] For each pixel in the preprocessed image, its gray value is denoted as G. The average gray value of the NxN pixels surrounding each pixel is calculated and denoted as Gavg. If G-Gavg>threshold 1, the pixel is marked as 255; otherwise, the pixel is marked as 0. The area marked as 255 is the target pixel area.

[0017] As a further aspect of the present invention: in step S3, the contour is filtered, and a local image of a fixed size is extracted from the filtered contour, which further includes:

[0018] Set upper and lower thresholds for the length and width of the contour to limit the range of the target contour to be filtered. Calculate the minimum bounding rectangle of each contour marked in S2. When the minimum bounding rectangle of the contour is within the range of the target contour, the contour is filtered out and its center point coordinates are calculated. Then, a local image of a fixed size is cropped on the FOV image with the center point as the center.

[0019] As a further aspect of the present invention: in step S5, finding the center point particle coordinates in the detection coordinates further includes calculating the average X and average Y values ​​of all detection coordinates to obtain the average center point of the detection coordinates; the center point particle coordinates are the particle coordinates of the detection coordinates that are closest to the average center point.

[0020] As a further aspect of the present invention: in step S5, determining the optimal offset between the theoretical coordinates and the center point particle coordinates further includes: finding multiple first theoretical coordinates in the theoretical coordinates that are less than a threshold 1 away from the center point particle coordinates; calculating a first offset between each first theoretical coordinate and the center point particle coordinates; superimposing each first offset onto the detection coordinates to obtain multiple sets of offset first detection coordinates; finding the point closest to the multiple sets of first detection coordinates in the theoretical coordinates and obtaining multiple second offsets, wherein the optimal offset is the first offset corresponding to the minimum value among the second offsets.

[0021] As a further aspect of the present invention: in step S5, matching the detected coordinates with the theoretical coordinates based on the optimal offset further includes: the first detected coordinates corresponding to the optimal offset are represented as the optimal point coordinates, and the point closest to the optimal point coordinates is found in the theoretical coordinates as the final matched theoretical point.

[0022] Compared with the prior art, the beneficial effects of the present invention are:

[0023] 1. The visual positioning and matching method for MiniLED backlight panel particles of the present invention extracts FOV image information based on grayscale values, filters the extracted contours, and extracts a fixed-size local image. The detection coordinates of the particle are obtained by accumulating the reference coordinates on the local image with the sub-pixel coordinates of the particle obtained through a standard shape model search. The optimal offset between the detection coordinates and the theoretical coordinates is then determined from the center point coordinates. Matching the theoretical coordinates is achieved through this optimal offset, thereby improving the efficiency of subsequent AOI inspection. Because particle positioning is performed in advance, the subsequent AOI inspection function only needs to detect the area around the particle. The detection area can be reduced by 50% to 90%, and the detection algorithm time can be reduced exponentially.

[0024] 2. Reduced processing load and saved disk space: Because particle positioning is performed in advance, subsequent AOI inspection equipment only needs to store images around the particles. Disk space usage can be reduced by 50% to 90%.

[0025] 3. Avoid the risk of particle serial number mismatch: Since this solution can match the positioning results with the particle serial numbers one by one, when the AOI inspection function detects NG particles and transmits the results to the rework station, the rework station can accurately locate the coordinates of the NG particles and perform repairs. This can reduce the time spent by the rework station searching for particles during repairs. Attached Figure Description

[0026] Figure 1 A flowchart for the visual positioning and matching method of MiniLED backlight panel particles;

[0027] Figure 2 This is the result after Step 1 preprocessing;

[0028] Figure 3 This is a schematic diagram of shape-based contour matching in Step 4;

[0029] Figure 4 Here is the flowchart for the Step 5 spatial matching algorithm;

[0030] Figure 5 The result of the Step5 spatial matching algorithm is shown in the figure. Detailed Implementation

[0031] To explain in detail the technical content, structural features, objectives, and effects of the technical solution, the following description is provided in conjunction with specific embodiments and accompanying drawings.

[0032] Please see Figure 1 In this embodiment, a method for visual positioning and matching of MiniLED backlight panel particles includes the following steps:

[0033] Step 0. Obtain the image data stream from the camera and temporarily store it in the computer's memory.

[0034] Step 1. Image Preprocessing:

[0035] See Figure 2 For each pixel in the original color FOV image, calculate the difference between its blue channel grayscale value and its red channel grayscale value, and assign this difference to the pixel at the same coordinates in the preprocessed image.

[0036] Step 2. Image segmentation:

[0037] For each pixel in the preprocessed image, its grayscale value is denoted as G. The average grayscale value of the NxN pixels surrounding each pixel is calculated and denoted as Gavg. If G - Gavg > threshold 1, the pixel is marked as 255; otherwise, it is marked as 0. The region marked as 255 is the particle region to be extracted. Through this step, pixels with large deviations (i.e., G - Gavg greater than threshold 1) in the preprocessed image are marked, thereby achieving the extraction of the target pixel region.

[0038] Step 3. Outline Filtering:

[0039] Set upper and lower thresholds for the length and width of the contours. These thresholds define the range of target contours to be filtered. Calculate the minimum bounding rectangle for each contour marked in Step 2. Record the length and width of this minimum bounding rectangle as the length and width of the corresponding contour. When the length and width of the minimum bounding rectangle are both within the upper and lower thresholds (i.e., the minimum bounding rectangle of the contour is within the target contour range), the contour is filtered out, and its center point coordinates are calculated. Then, a fixed-size local image is cropped from the FOV image centered on this center point. Simultaneously, record the reference coordinates (X_local, Y_local) of the upper left point of the local image on the FOV image. This step aims to minimize erroneous filtering caused by background interference.

[0040] Step 4. Obtain precise coordinates:

[0041] See Figure 3 In the multiple local images obtained in Step 3, a standard shape template is used to search for the subpixel coordinates (X_shape, Y_shape) of the particle on the local image. The reference coordinates (X_local, Y_local) of the top left corner of the local image in Step 3 are superimposed to obtain the detection coordinates of the particle on the entire FOV image, denoted as Pt_S = (X_shape + X_local, Y_shape + Y_local).

[0042] See Figure 4-5 Step 5. Spatial matching, including the following steps:

[0043] S51. The theoretical coordinates of all particles on the FOV image are denoted as Pt_T1, Pt_T2, ...;

[0044] S52. Obtain the detection coordinates of all particles on the FOV image according to Step 4, and denot them as Pt_S1, Pt_S2...;

[0045] S53. Calculate the mean X and mean Y values ​​of all detection coordinates Pt_S to obtain the average center point of detection coordinates Pt_S, denoted as Pt_S_C;

[0046] S54. Find the coordinates of the center point particle that is closest to the average center point Pt_S_C in the detection coordinates Pt_S, and denote it as Pt_S_X;

[0047] S55. In the theoretical coordinate system Pt_T, find multiple first theoretical coordinates Pt_T' that are less than a threshold 1 away from the center point particle coordinate Pt_S_X. The particles corresponding to these first theoretical coordinates Pt_T' are relatively close to the aforementioned center point particle, i.e., the theoretical coordinates corresponding to multiple particles located around the center point particle in the theoretical coordinate system. Each first theoretical coordinate Pt_T' has a first offset between it and the center point particle coordinate Pt_S_X, and calculate the corresponding multiple first offsets offset1, offset2...;

[0048] S56. Using the first offset obtained in S55, each offset is superimposed onto the detection coordinate Pt_S to obtain multiple sets of offset first detection coordinates Pt_S'. For each set of first detection coordinates Pt_S', find the point in the theoretical coordinates Pt_T that is closest to the first detection coordinate Pt_S'. There is a second offset between this closest point and the corresponding first detection coordinate Pt_S', denoted as disTotal. For each set of first detection coordinates Pt_S', a second offset disTotal is obtained. Select the first offset corresponding to the minimum value of the second offset disTotal as the optimal offset offset_best. This optimal offset is the overall offset from the true coordinates of the particles in the image to the theoretical coordinates.

[0049] S57. Take the detection coordinates Pt_S of all particles on the FOV image obtained in Step 4, and superimpose them with the optimal offset_best obtained in S56 to obtain the optimal point coordinates Pt_S_B. That is, the first detection coordinate Pt_S' corresponding to the optimal offset_best is represented as the optimal point coordinates Pt_S_B.

[0050] For each optimal point coordinate Pt_S_B, find the point closest to it in the theoretical coordinates Pt_T as the final theoretical point to match it, and complete the matching.

[0051] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." or "including..." does not exclude the presence of additional elements in the process, method, article, or terminal device that includes said element. Additionally, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the stated number; "above," "below," "within," etc., are understood to include the stated number.

[0052] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for visual positioning and matching of MiniLED backlight panel particles, characterized in that: Includes the following steps: Step S1: Acquire FOV image data stream; Step S2: Image segmentation: For the acquired image information, each pixel in the image is marked according to its grayscale value, and the target pixel region is extracted based on the marking. Step S3: Contour filtering: Based on the contours formed by the target pixel regions extracted in S2, the contours are filtered, and a local image of a fixed size is cropped through the filtered contours. The reference coordinates of the upper left point of the local image on the FOV image are recorded. Step S4: Obtain precise coordinates: On the multiple local images obtained in S3, the standard shape template is used to search for the sub-pixel coordinates of the particles on the local images. After superimposing the reference coordinates, the detection coordinates of the particles on the entire FOV image are obtained. Step S5: Spatial matching: Obtain the theoretical coordinates of all particles in the FOV image, find the center point particle coordinates in the detection coordinates, determine the optimal offset between the theoretical coordinates and the center point particle coordinates, and achieve matching between the detection coordinates and the theoretical coordinates based on the optimal offset. In step S5, determining the optimal offset between the theoretical coordinates and the center point particle coordinates further includes: finding multiple first theoretical coordinates in the theoretical coordinates that are less than a threshold 1 away from the center point particle coordinates, and calculating the first offset between each first theoretical coordinate and the center point particle coordinates. Each of the first offsets is superimposed on the detection coordinates to obtain multiple sets of offset first detection coordinates; the point closest to the multiple sets of first detection coordinates is found in the theoretical coordinates and multiple second offsets are obtained, the optimal offset being the first offset corresponding to the minimum value among the second offsets.

2. The visual positioning and matching method for MiniLED backlight panel particles according to claim 1, characterized in that: Before step S1, image preprocessing is also included, in which the difference between the blue channel gray value and the red channel gray value of each pixel in the original FOV image is calculated, and the difference is assigned to the pixel at the same coordinate in the preprocessed image.

3. The visual positioning and matching method for MiniLED backlight panel particles according to claim 1, characterized in that: In step S2, each pixel in the image is marked according to its grayscale value, and the target pixel region is extracted based on the markings. This step also includes: For each pixel in the preprocessed image, its gray value is denoted as G. The average gray value of the NxN pixels surrounding each pixel is calculated and denoted as Gavg. If G-Gavg>threshold 1, the pixel is marked as 255; otherwise, the pixel is marked as 0. The area marked as 255 is the target pixel area.

4. The visual positioning and matching method for MiniLED backlight panel particles according to claim 1, characterized in that: In step S3, the contours are filtered, and a fixed-size local image is extracted from the filtered contours. This also includes: Set upper and lower thresholds for the length and width of the contour to limit the range of the target contour to be filtered. Calculate the minimum bounding rectangle of each contour marked in S2. When the minimum bounding rectangle of the contour is within the range of the target contour, the contour is filtered out and its center point coordinates are calculated. Then, a local image of a fixed size is cropped on the FOV image with the center point as the center.

5. The visual positioning and matching method for MiniLED backlight panel particles according to claim 1, characterized in that: In step S5, finding the center point particle coordinates in the detection coordinates further includes calculating the average X and Y values ​​of all detection coordinates to obtain the average center point of the detection coordinates; the center point particle coordinates are the particle coordinates of the detection coordinates that are closest to the average center point.

6. The visual positioning and matching method for MiniLED backlight panel particles according to claim 1, characterized in that: In step S5, matching the detected coordinates with the theoretical coordinates based on the optimal offset further includes: the first detected coordinates corresponding to the optimal offset are represented as the optimal point coordinates, and the point closest to the optimal point coordinates in the theoretical coordinates is found as the final matched theoretical point.

Images