A micro led effective chip recognition method based on similar candidate region generation

Abstract

The application relates to the fields of chip detection and artificial intelligence, and relates to a Micro Led effective chip recognition method based on similar candidate area generation, which collects a wafer image containing a Micro Led chip as a to-be-searched image, uses a typical image of the Micro Led chip as a template image, uses the same feature extraction method to extract features of the to-be-searched image and the template image respectively, uses the features A of the to-be-searched image after convolution as the input of a depth separable convolution, uses the features B of the template image after convolution as the convolution kernel of the depth separable convolution, carries out depth separable convolution, and obtains features C containing similar feature information. The application can not only avoid data dependency of a neural network, so that the neural network can detect chips that have not been learned, but also can use the high efficiency of the neural network to quickly complete the recognition of effective chips.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of chip detection and artificial intelligence, and particularly relates to a Micro Led effective chip recognition method based on similar candidate region generation. BACKGROUND

[0002] Micro LED has smaller chip size and higher device performance, greatly improving the display performance of mobile terminal display, and also increasing the technical difficulty of chip manufacturing and detection.

[0003] In the production process of Micro LED, automatic optical detection and flying probe testing are needed to detect effective chips, but due to the small size and large number of individual chips, using traditional methods to detect effective chips not only consumes a lot of time, but also seriously reduces the precision, which seriously restricts the chip production capacity. At present, neural networks have surpassed traditional algorithms in many fields, so it is necessary to try to use neural networks to recognize Micro Led effective chips. SUMMARY

[0004] In order to solve the problems in the prior art, the present application provides a Micro Led effective chip recognition method based on similar candidate region generation, which can reduce the detection time of Micro Led effective chips, improve the detection precision, and at the same time get rid of the data dependency. It can also detect effective chips for chips that have not been learned.

[0005] In order to achieve the above-mentioned target, the present application adopts the following technical scheme: a Micro Led effective chip recognition method based on similar candidate region generation, comprising the following steps:

[0006] Step 1: use an industrial camera to shoot a 4K image containing multiple same Micro Led chips on the front of a wafer as a to-be-searched image, and use a 4K image of an effective Micro Led chip on the wafer as a template image;

[0007] Step 2: use the same feature extraction method to extract the features of the to-be-searched image and the template image respectively to obtain a to-be-searched feature image and a template feature image;

[0008] Step 3: perform feature fusion on the to-be-searched feature image and the template feature image through a feature fusion network to obtain a feature C;

[0009] Step 4: input the feature C into the class branch and the position branch of the candidate region generation network (Region Proposal Network, RPN) respectively to obtain a confidence feature and a candidate region feature;

[0010] Step five: set a threshold, screen out the prior box with confidence greater than the threshold, find the position offset of the prior box corresponding to the candidate region feature, the chip corresponding to the prior box with confidence greater than the threshold is the Micro Led effective chip, and the position information of the Micro Led effective chip on the wafer is determined according to the position offset of the prior box corresponding to the Micro Led effective chip.

[0011] The coaxial light source and the ring light source are used to ensure the consistency of light during shooting. The coaxial light source is a supplementary light source installed coaxially with the industrial camera directly above the wafer. The ring light source is installed on the side of the wafer to irradiate the wafer surface at an oblique angle, and the center of the ring is the industrial camera.

[0012] In step two, the feature extraction method is a traditional feature point extraction algorithm or a deep learning algorithm.

[0013] In step three, the feature fusion network performs feature fusion in the following manner: a template feature image is input into a convolution layer and an activation layer to obtain a feature A with a size of (N, Wa, Ha), where N is the number of channels of the feature A, Wa is the width of the feature A, Ha is the height of the feature A, and Wa and Ha are positive odd numbers less than 10; a feature image to be searched is input into a convolution layer and an activation layer to obtain a feature B with a size of (N, Wb, Hb), where the number of channels of the feature B is the same as that of the feature A, Wb is the width of the feature B, Hb is the height of the feature B, Wb > Wa, and Hb > Ha; the feature B is taken as the convolution input, and the feature A is taken as the convolution kernel to perform deep separable convolution, thereby obtaining a feature C with a size of (N, Wc, Hc), where Wc is the width of the feature C, Hc is the height of the feature C, Wc ≤ Wb, and Hc ≤ Hb. The feature C contains similar feature information between the image to be searched and the template image, and the greater Wc and Hc are, the higher the accuracy of the position similar features for subsequent processing is.

[0014] In step four, the class branch of the candidate region generation network is composed of a convolution layer, an activation layer, and a convolution layer stacked in sequence, and the position branch is also composed of a convolution layer, an activation layer, and a convolution layer stacked in sequence. The former convolution of each branch is responsible for adjusting the width and height of the feature map, and the latter convolution is responsible for adjusting the number of channels of the feature map. The class branch outputs a confidence feature with a size of (Na, Wo, Ho), where Na is the number of anchors, Wo and Ho represent the width and height of the output feature map, and the confidence of the target in the Na anchors contained in each position on the output feature map is represented. The position branch outputs a candidate region feature with a size of (4Na, Wo, Ho), which represents four position offsets corresponding to each prior box at each position on the output feature map.

[0015] In step five, the position information of the Micro Led effective chip on the wafer is determined according to the position offset corresponding to the prior box corresponding to the Micro Led effective chip, wherein the prior box Anchor is a predefined Na rectangular box, and each rectangular box is denoted as A i ={cx i ,cy i ,w i ,h i}, 0≤i≤Na, cx i is the x coordinate of the center point of the prior box, cy i is the y coordinate of the center point of the prior box, w i is the width of the prior box, h i is the height of the prior box, the candidate region feature outputs Na*Wo*Ho position offsets, for each output position offset O j ={offset_x j ,offset_y j ,offset_w j ,offset_h j}, 0≤j≤Na*Wo*Ho, offset_x j is the offset of the x coordinate of the center point of the prior box, offset_y j is the offset of the y coordinate of the center point of the prior box, offset_w j is the offset of the width of the prior box, offset_h j is the offset of the height of the prior box, and finally Na*Wo*Ho target recognition boxes are obtained, and each target recognition box is R k ={real_x k ,real_y k ,real_w k ,real_h k}, 0≤k≤Na*Wo*Ho, wherein:

[0016] real_x k =cx i +w i *offset_x j

[0017] real_y k =cx i +w i *offset_y j

[0018] real_w k =cx i +w j *offset_x j

[0019] real_h k = h i * offset_y j .

[0020] Meanwhile, the Micro Led effective chip recognition based on similar candidate region generation is carried out by using the template image and the image to be searched, which can not only avoid the data dependency of the neural network, but also enable the neural network to detect chips that have not been learned, and also enable the neural network to quickly complete the recognition of effective chips by the high efficiency of the neural network.

[0021] The 4K image containing a plurality of same Micro Led chips on the front of the wafer is shot by using an industrial camera vertically above the wafer, and the 4K image containing the Micro Led chip is shot by using an industrial camera vertically above the wafer.

[0022] The Micro Led effective chip recognition based on similar candidate region generation is carried out by using the template image and the image to be searched, which can not only avoid the data dependency of the neural network, but also enable the neural network to detect chips that have not been learned, and also enable the neural network to quickly complete the recognition of effective chips by the high efficiency of the neural network. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a schematic view of an image to be searched and a template image;

[0024] Figure 2 is a schematic view of a coaxial light source and a ring light source;

[0025] Figure 3 is a schematic view of similar feature calculation;

[0026] Figure 4 is a schematic view of candidate region generation calculation. DETAILED DESCRIPTION

[0027] The application will be further described below in conjunction with the drawings. The following examples are only used to more clearly illustrate the technical solutions of the application, and cannot be used to limit the protection scope of the application.

[0028] A Micro Led effective chip recognition method based on similar candidate region generation comprises the following steps:

[0029] Step 1: a 4K image containing a Micro Led chip is shot by using an industrial camera vertically above the wafer as an image to be searched, and a typical effective chip appearance image of the shot Micro Led chip is known, and the typical effective chip appearance image is used as a template image of the shot image to be searched, such as Figure 1As shown, the left side is the image to be searched, and the right side is the template image. When shooting, use the coaxial light source and ring light source to ensure consistent light, such as Figure 2 As shown, the left side is the coaxial light source, which is a fill-in light source installed coaxially above the wafer and the industrial camera, and the right side is the ring light source, which is installed on the side of the wafer to irradiate the wafer surface at an oblique angle. The center of the ring is the industrial camera.

[0030] Step two: using the same feature extraction method to extract the features of the image to be searched and the template image, respectively, to obtain the feature image to be searched and the template feature image, the feature extraction method can be a traditional feature point extraction algorithm or a deep learning algorithm, here the Alexnet neural network is used for feature extraction, the image to be searched and the template image are input into the same Alexnet neural network, and the size of the feature image to be searched is (256, 25, 25) and the size of the template feature image is (256, 6, 6).

[0031] Step three: as shown in Figure 3 , a feature A with a size of (256, 5, 5) is obtained after the template feature image passes through a convolution layer and an activation layer. A feature B with a size of (256, 20, 20) is obtained after the image to be searched passes through a convolution layer and an activation layer. Then, feature B is taken as the convolution input, and feature A is taken as the convolution kernel to perform deep separable convolution, and a feature C with a size of (256, 20, 20) is obtained, which contains the similar feature information between the image to be searched and the template image;

[0032] Step four: as shown in Figure 4 , feature C is input into the class branch and the position branch for candidate region generation, respectively. The class branch outputs a confidence feature with a size of (3, 15, 15); the position branch outputs a candidate region feature with a size of (12, 15, 15). The class branch is composed of a convolution layer, an activation layer and a convolution layer in sequence, and the position branch is also composed of a convolution layer, an activation layer and a convolution layer in sequence.

[0033] Step five: according to the confidence feature, the effective chips with a confidence greater than a threshold value are screened out, the corresponding position offset is found from the candidate region feature, and the position information of the effective chips is calculated according to the prior frame information and the position offset. The prior frame Anchor is a predefined three rectangular frames, each rectangular frame is denoted as A i ={cx i ,cy i ,w i ,h i}, 0≤i≤Na, cx i is the x coordinate of the center point of the prior frame, cy i is the y coordinate of the center point of the prior frame, wi is the width of the prior box, h i is the height of the prior box, the candidate region feature outputs 675 position offsets, for each position offset O j ={offset_x j ,offset_y j ,offset_w j ,offset_h j}, 0≤j≤675, offset_x j is the offset of the x-coordinate of the center point of the prior box, offset_y j is the offset of the y-coordinate of the center point of the prior box, offset_w j is the offset of the width of the prior box, offset_h j is the offset of the height of the prior box, and finally 675 target recognition boxes are obtained, each target recognition box is R k ={real_x k ,real_y k ,real_w k ,real_h k}, 0≤k≤Na*Wo*Ho, wherein:

[0034] real_x k =cx i +w i *offset_x j

[0035] real_y k =cy i +h i *offset_y j

[0036] real_w k =w i *offset_x j

[0037] real_h k =h i *offst_y j

[0038] Only when the corresponding confidence of the 675 target recognition boxes is greater than 0.7 can it be considered as an effective chip.

[0039] The above only describes the preferred embodiments of the present application, and it should be noted that for ordinary skilled persons in the art, without departing from the technical principles of the present application, several improvements and modifications can be made, and these improvements and modifications should be considered as the protection scope of the present application.

Claims

1. A Micro Led effective chip identification method based on similar candidate region generation, characterized in that, The method comprises the following steps: Step 1: using an industrial camera to shoot a 4K image containing a plurality of same Micro Led chips on the front of a wafer as a to-be-searched image, and using a 4K image of an effective Micro Led chip on the wafer as a template image; Step 2: using the same feature extraction method to extract features of the to-be-searched image and the template image respectively to obtain a to-be-searched feature image and a template feature image; Step 3: performing feature fusion on the to-be-searched feature image and the template feature image through a feature fusion network to obtain feature C; Step 4: inputting the feature C into a class branch and a position branch of a candidate region generation network respectively to obtain a confidence feature and a candidate region feature; Step 5: setting a threshold, screening out a prior box with a confidence greater than the threshold, finding a position offset corresponding to the prior box from the candidate region feature, and determining a Micro Led effective chip corresponding to the prior box with the confidence greater than the threshold, and determining position information of the Micro Led effective chip on the wafer according to the position offset corresponding to the prior box of the Micro Led effective chip. 2.The Micro Led effective chip identification method based on similar candidate regions is generated according to claim 1, characterized in that: When shooting, a coaxial light source and a ring light source are used to ensure that the light is consistent, the coaxial light source is a supplementary light source coaxial with the industrial camera and installed above the wafer, and the ring light source is installed on the side of the wafer to irradiate the wafer surface at an oblique angle, and the center of the ring is the industrial camera. 3.The Micro Led effective chip identification method based on similar candidate region generation according to claim 1, characterized in that: In step 2, the feature extraction method is a traditional feature point extraction algorithm or a deep learning algorithm. 4.The Micro Led effective chip identification method based on similar candidate regions is generated according to claim 1, characterized in that: In step 3, the feature fusion network performs feature fusion in the following manner: after the template feature image passes through a convolution layer and an activation layer, a feature A with a size of (N, Wa, Ha) is obtained, wherein N is the channel number of the feature A, Wa is the width of the feature A, and Ha is the height of the feature A, Wa and Ha are positive odd numbers less than 10; after the to-be-searched feature image passes through a convolution layer and an activation layer, a feature B with a size of (N, Wb, Hb) is obtained, the channel number of the feature B is the same as that of the feature A, Wb is the width of the feature B, Hb is the height of the feature B, Wb>Wa, and Hb>Ha; the feature B is taken as convolution input, and the feature A is taken as convolution kernel to perform deep separable convolution, so as to obtain a feature C with a size of (N, Wc, Hc), Wc is the width of the feature C, Hc is the height of the feature C, Wc≤Wb, and Hc≤Hb; the feature C contains similar feature information between the to-be-searched image and the template image, the greater Wc and Hc are, the higher the accuracy of the position similar feature used for subsequent processing is.

5. The Micro Led effective chip identification method based on similar candidate region generation according to claim 4, characterized in that: In step 4, the class branch in the candidate region generation network is composed of a convolution layer, an activation layer and a convolution layer in sequence, and the position branch is also composed of a convolution layer, an activation layer and a convolution layer in sequence, the former convolution of each branch is responsible for adjusting the width and height of the feature map, and the latter convolution is responsible for adjusting the channel number of the feature map; the class branch outputs a confidence feature with a size of (Na, Wo, Ho), wherein Na is the number of prior boxes, Wo and Ho represent the width and height of the output feature map, and represent the confidence of the target in the Na prior boxes contained by each position on the output feature map. The position branch outputs a candidate region feature with a size of (4Na, Wo, Ho), which represents that each prior box on each position of the output feature map corresponds to four position offsets. 6.The Micro Led effective chip identification method based on similar candidate regions is generated according to claim 5, characterized in that: In step five, the position information of the Micro Led effective chip on the wafer is determined according to the position offset corresponding to the prior box corresponding to the Micro Led effective chip, wherein the prior box Anchor is a predefined Na rectangular box, and each rectangular box is denoted as A i ={cx i , cy i , w i , h i},0≤i≤Na,cx i is the x-coordinate of the center point of the prior box, cy i is the y-coordinate of the center point of the prior box, w i is the width of the prior box, h i is the height of the prior box, the candidate region feature outputs Na*Wo*Ho position offsets, for each output position offset O j ={offset_x j , offset_y j , offset_w j , offset_h j},0≤j≤Na*Wo*Ho,offset_x j is the offset of the x-coordinate of the center point of the prior box, offset_y j is the offset of the y-coordinate of the center point of the prior box, offset_w j is the offset of the width of the prior box, offset_h j is the offset of the height of the prior box, and finally Na*Wo*Ho target recognition boxes are obtained, and each target recognition box is R k ={real_x k , real_y k , real_w k , real_h k},0≤k≤Na*Wo*Ho, wherein: .

7. The Micro Led effective chip recognition method based on similar candidate region generation according to claim 1, characterized in that: Meanwhile, the Micro Led effective chip identification based on the similar candidate region generation is performed by using the template image and the image to be searched, which can avoid the data dependency of the neural network, enable the neural network to detect chips that have not been learned, and enable the neural network to quickly complete the identification of the effective chip by means of the high efficiency of the neural network. 8.The Micro Led effective chip identification method based on similar candidate region generation of claim 1, wherein: The 4K image containing a plurality of same Micro Led chips on the front side of the wafer is shot by using an industrial camera vertically above the wafer. The 4K image containing the Micro Led chip is shot by using an industrial camera vertically above the wafer.

Images