Semiconductor bare chip high-speed sorting and positioning method based on CCD image

Abstract

The invention discloses a semiconductor bare chip high-speed sorting and positioning method based on a CCD image. The semiconductor bare chip high-speed sorting and positioning method comprises the steps that S1, a template image is input; s2, performing Gaussian filtering on the template image, traversing and searching an area similar to the template image in the target image by using pyramid processing, performing template matching, and generating a template image pyramid; s3, after the template image pyramid is generated, extracting edge information of each layer of the template image pyramid; step S4, storing data information used for matching as a model file, wherein the data information comprises template width w, template height h, pyramid layer number t, models of each layer of the pyramid and sub-pixel feature points of each layer of the pyramid; and S5, after a model file is generated according to the template image, matching is performed in the target image according to the model data to obtain a matching result. The semiconductor bare chip sorting and positioning precision is higher, the processing speed is higher, the semiconductor bare chip sorting and positioning device is not easily influenced by noise and partial shielding, and the matching stability is better.

Description

Technical Field

[0001] The invention relates to the technical field of computer software, and in particular to a high-speed sorting and positioning method for semiconductor bare chips based on CCD images. Background Art

[0002] In the semiconductor manufacturing process, die sorting is a crucial link, which involves taking each die out of the substrate and classifying and boxing them by type. Dies are usually unpackaged semiconductor chips with small size, complex surface, and may have various types of defects. Efficient and accurate sorting and positioning are the basis for ensuring subsequent packaging, testing and integration.

[0003] Traditional die sorting methods usually rely on microscopes and artificial vision, which are not only inefficient, but also easily limited by human factors and difficult to meet the requirements of high-speed and high-precision automation. In addition, traditional sorting methods are difficult to handle the high efficiency requirements in large-scale production, especially when facing small-sized and complex-surface die, the limitations of manual methods are particularly obvious.

[0004] With the advancement of computer vision and image processing technology, the sorting and positioning method based on CCD (semiconductor device) images has become an effective alternative. CCD image technology can obtain detailed information of bare chips through high-resolution images, and combine computer vision algorithms for image analysis to achieve automatic detection, positioning and sorting of bare chips.

[0005] The sorting and positioning method based on CCD images has many advantages, but in practical applications, it still faces some challenges: Complex background: The surface of the bare chip may have uneven lighting and complex background, which will affect the accuracy and stability of image processing. Small size and high resolution requirements: As the chip size decreases, CCD cameras are required to provide higher resolution, and the processing speed also needs to match it. High-speed real-time processing: As the production pace accelerates, the sorting and positioning system needs to be able to process image data in real time on high-speed equipment to ensure that the sorting process is not affected.

[0006] In response to these problems, existing detection technologies, such as grayscale-based image matching, shape context-based matching methods, and histogram-based matching, can meet the needs of naked object sorting and positioning to a certain extent, but they have limitations in practical applications. For example, although grayscale-based image matching can meet the positioning accuracy requirements, it is often slow because it needs to calculate all pixel positions in the template. In addition, since the colors of the same model of bare chips in different batches are different, they need to be debugged for settings, which increases the complexity of the application. For most context shape matching methods, changes in the internal objects of the image will cause inaccurate matching for actual images. They are sensitive to the sampling method, resulting in partial missing of the shape and low robustness to occlusion. In addition, most of them need to debug the parameters of the sampling, context calculation, cost matrix and other processes, making it difficult to quickly apply them to different models.

[0007] Aiming at the shortcomings of the existing technology such as low detection speed, insufficient generalization and adaptability, and large amount of parameter debugging, a high-speed sorting and positioning method for semiconductor bare chips based on CCD images is proposed. Summary of the invention

[0008] The purpose of the present invention is to overcome the existing defects and provide a high-speed sorting and positioning method for semiconductor bare chips based on CCD images, so that the semiconductor bare chip sorting and positioning has higher accuracy, faster processing speed, is not easily affected by noise and partial occlusion, and has better matching stability.

[0009] The technical solution to achieve the above purpose is: A high-speed sorting and positioning method for semiconductor bare chips based on CCD images, comprising: Step S1, input template image; Step S2, performing Gaussian filtering on the template image, and using pyramid processing to traverse and search for areas similar to the template image in the target image, perform template matching, and generate a template image pyramid; Step S3, after generating the template image pyramid, extracting edge information of each layer of the template image pyramid; Step S4, save the data information used for matching as a model file, the data information includes the template width , template height , the number of pyramid layers , models of each layer of the pyramid, sub-pixel feature points of each layer of the pyramid; Step S5, after generating the model file according to the template image, matching is performed in the target image according to the model data to obtain a matching result.

[0010] Preferably, in step S2, a Gaussian pyramid of the template image is generated, and the steps of constructing the Gaussian pyramid are: Performing Gaussian filtering on the template image to obtain a filtered image; Downsampling is performed by deleting the even rows and even columns of the filtered image to obtain the next layer of pyramid image; Continue to perform Gaussian filtering and downsampling on the obtained pyramid image until the number of pyramid layers reaches the set number of layers ; Among them, the number of pyramid layers The calculation is as follows: Get the template area according to the size of the template image ; Among them, the template area Obtained from template width and template height; Then execute the loop ,until , get the number of pyramid layers that need to be calculated .

[0011] Preferably, in step S3, the edge information includes pixel coordinates and direction information of edge points of each layer of the template image pyramid and sub-pixel coordinates of edge points of the bottom layer template; The process of extracting edge information of each layer of the template image pyramid is as follows: Calculate the Sobel (edge ​​detection algorithm) gradient matrix in the horizontal and vertical directions of the image , , the gradient matrix is ​​obtained by convolving the corresponding convolution kernel with the image. The convolution kernels in the horizontal and vertical directions are: ; ; Depend on , Calculate the gradient image of the image : ; In the formula, Represents the position coordinates of a point in the image; According to the gradient image Calculate the threshold for Canny edge detection: Generate a cumulative histogram from the data of the gradient image G, where the horizontal axis of the histogram is the gradient size and the vertical axis is the cumulative value of the number of pixels corresponding to the gradient size; Then, the gradient sizes corresponding to the accumulated values ​​of 0.9 times and 0.3 times the total number of pixels are used as the high threshold and low threshold of Canny edge detection respectively. After obtaining the thresholds, the Canny (edge ​​detection algorithm) detection algorithm is used to obtain the edge image of the template image. The image pixel coordinates of all edge points can be obtained from the edge image , convert pixel coordinates to relative coordinates relative to the center point of the template: ; ; In the formula, and are the width and height of the current layer template image respectively, and the direction of the edge point is determined by the direction gradient of the edge point image pixel coordinates , The values ​​at the corresponding positions are: ; ; Through the above, the edge information data of each layer of the template pyramid is obtained, and then the template edge point map is obtained, where the coordinates are the relative coordinates of the edge points relative to the center point of the template image , the Sobel gradient values ​​in the horizontal and vertical directions ; After obtaining the edge information data of each layer of the template pyramid, set the mask image to filter the unnecessary areas: Performing pyramid operation on the mask image to obtain a mask image pyramid image corresponding to each template layer; Then the mask pyramid image is converted into a binary image of 0 and 1, and the obtained template edge point image is multiplied with the mask image to obtain the edge image after being "filtered" by the mask image; The edge points of the edge image are further processed to extract the direction data from the edge information in the image: Loop through each pixel in the entire image. If the point belongs to an edge, perform a depth-first search to find and connect all edge points connected to the current point. Then filter the edge segment of the current layer. The filtering method is as follows: By input parameters and calculate, , by comparing each edge segment with Compare and filter shorter edges, where Indicates the shortest edge segment threshold of the current layer, Indicates the current pyramid layer index; Further calculations show that ,in, represents the filtering step size of the corresponding layer, Indicates the size, determined by the input, Indicates the total number of pyramid layers; Calculate the distance between each edge segment and the center point and Axis offset and : By calculating the center point of the template graph: , the center point of the current layer ,in, and denote rows and columns respectively, and Obtained from the template graph, and Each point and The difference is obtained; Then, calculate , , and Respectively expressed in and The gradient component in the direction, Indicates the point position, according to and Gradient component in the direction, calculate the modulus of the gradient , that is, the gradient strength of each point; Finally , , , and The orientation information is saved to the model; Then calculate the sub-pixel edge of the bottom layer of the template pyramid, which is the original image: After obtaining the pixel edge of the bottom layer template, a sub-pixel accurate edge extraction algorithm based on the facet (multi-dimensional model) model is used. Assume It is the coordinate of a pixel point on the edge of the pixel. The facet model converts the template gradient image Fit a three-dimensional surface at this coordinate: ; In the formula, is the image coordinate, is the pixel gradient, , , , and Are coefficients, obtained by convolving the kernel at the pixel position: ; ; ; ; ; Through the above 5 kernels and gradient images China-Israel The dot product of the 3*3 area centered at the pixel position gets the coefficient. After getting the fitted surface function, we get the gradient direction of the pixel: ; ; The sub-pixel precise position coordinates of the pixel position are and It can be calculated by the following formula: ; ; in: ; The calculated coordinates are considered valid sub-pixel coordinates when the following conditions are met: ; ; All pixels at the bottom layer of the template pyramid are subjected to the above calculation and the effective sub-pixel coordinates are extracted to obtain the sub-pixel edge feature points.

[0012] Preferably, in step S5, the matching result is obtained by matching the model data in the target image, and the matching process is as follows: The first step is Gaussian filtering: Use Gaussian filtering to remove noise from the target image; Step 2: Pyramid processing: Perform pyramid processing on the target image. The number of pyramid layers is the same as the number of edge information pyramid layers in the model. same; The third step is the initial matching of the top layer of the pyramid: After obtaining the target image pyramid, the top image of the target pyramid is processed with the top image of the edge information pyramid to obtain the initial matching result, namely: For the edge point data at the top layer of the edge information pyramid, it is rotated and scaled to obtain a set of edge information for matching. The transformed edge information is: ; ; ; ; In the formula, , , and are the information of the original edge points, and are the position and direction information of the transformed points, respectively, where: ; ; ; In the formula, represents the scaling factor, Indicates the rotation angle; Perform the above transformation on all edge points to obtain edge information after rotation and scaling; The edge information of all combinations of rotation angles and scaling is matched with the top layer of the target image pyramid. The matching process is: Get all the combinations of rotation angles and scaling factors according to the input parameters, where the rotation angle is determined by the initial rotation angle, rotation range, and angle step. The initial rotation angle and rotation range are input parameters, and the angle step is determined by the size of the template image: ; In the formula, represents the angle step, and are the width and height of the template image, respectively. is the number of pyramid layers; The initial and final values ​​of the scaling factor are determined by the input parameters, and the scaling step length The calculation is obtained by calculating the template image size of different chips: ; In the formula, , , and Respectively represent the width and height of the target image, and the width and height of the template image; Calculate the top step of the pyramid : ; Calculate the number of scale dimensions : ,generate The discrete scale of the quantity is obtained by the same calculation method. The number of discrete angles is obtained, and then all the combinations of rotation angles and scaling factors of the top layer are obtained, where represents the initial value of the scaling factor, Indicates the end value of the scaling factor; Calculate the baseline scores for each layer using linear layer decay : ; In the formula, is the input parameter; Then calculate the minimum score requirements for each layer ,in, is an input parameter, indicating the greediness, and adjusting the score attenuation of each layer; A matching score map is established to store the matching scores of the corresponding image coordinates, which is equal to the top image size of the target image pyramid; In the image coordinates The matching score at position is calculated as: ; In the formula, represents the number of edge points, , , and Respectively represent The relative coordinate information and direction information of the edge points, Indicates the position of the target image orientation in horizontal and vertical directions; Traverse to obtain the matching scores of all positions, and compare them with the score threshold. The score greater than the score threshold is taken as the initial matching result. Find all matching results under all combinations of rotation angles and scaling factors to obtain a set of initial matching results. The matching result data includes the rotation angle, scaling factor and image coordinate position. Step 4: Track the matching results to the bottom layer: After obtaining a set of initial matching results, track each matching result to the next layer until the matching score is less than the threshold or reaches the bottom layer. The tracking process is as follows: Based on the matching results of the previous layer, determine the matching condition range of the next layer. Assume that the matching results of the previous layer are: matching position , matching angle , scaling factor , then the matching condition at the next level is: Matching position: The corresponding position in the following layer is In the 3*3 area at the center; Matching angle: 3 angle conditions , is the angle step of the current layer; Scaling factor: unchanged; The matching score is calculated based on the matching conditions and positions of the next layer, and the maximum matching score is compared with the threshold. If it is greater than the threshold, the position is used as the matching result to continue tracking to the next layer until the bottom layer. If it is less than the threshold, the tracking is stopped and the matching result is deleted. Step 5: Suppression of overlapping targets: Sort the matching results by score and then calculate the matching results Matches all other The intersection area , according to the input parameters ,if then discard; Step 6: Refine the matching parameters: After tracking all the initial matching results, all the matching results at the bottom level are obtained, and the most suitable one, that is, the one with the largest matching score, is selected from all the matching results belonging to the same target; Refine the matching parameters of the pixel-level matching position of the matching result to obtain the matching pose result: Calculate refinement angle : ; In the formula, Indicates the angle of the current matching result. Indicates the range The independent variable that increases by 1 each time. represents the number of refinements, express When the value is , the angle refinement produces Angles; Will Each angle in is applied to the template model, new template points and directions of each point are calculated, new template information is generated, and then re-matched matching angles and scores; The polynomial fitting based on the least squares method is used to optimize the matching size+1 angles to obtain a more accurate angle; Sub-pixelization, matching the results to match the points Calculate the similarity within a 3x3 range and get the point and ; Based on the prediction of the maximum position of the quadratic surface fitting, the maximum position of the surface is found. The calculation process is as follows: ; In the formula, and are the input coordinates and values, namely and , obtained by using singular value decomposition , , , , and Coefficient, and then the maximum value position on the fitting surface, that is, the sub-pixel position, is obtained by the following formula : ; ; After the above processing, a more refined and accurate pixel position is obtained; Step 7: Filter results: Filter the results according to the input parameter matching score, angle, and target number as needed and output the final result.

[0013] Preferably, in the third step of step S5, when there are multiple angles and zooms, these situations are evenly distributed to multiple threads for processing, and the distribution method is as follows: First, get the number of hardware CPU cores , obtained by combining all rotation angles and scaling factors of the top layer , calculate the number of calculations each thread needs to perform: ; In the formula, is the remainder, right Take the remainder; Search strategy: For the target image, the number of potential matching result positions is very small compared to the total number of pixels. The amount of calculation required to calculate the matching scores of all positions based on the matching formula is too large. The calculation is terminated early based on the conditions to reduce the amount of calculation. That is, the condition for the initial matching result is that the matching score of the position is greater than the score threshold. : ; In all cumulative items, the value of each item is in Between, assuming For already accumulated Scores for each item: ; In the formula, For all points; If the values ​​of the remaining items are all 1, the sum of the remaining items It is expressed as: ; At this time, if: ; Then, the matching score of the position will not be able to reach the score threshold, and the above conditions can be used to terminate the calculation of most positions in advance, reduce the amount of calculation, and optimize the strategy.

[0014] The beneficial effects of the present invention are as follows: the present invention utilizes the edge information of the template for matching, so that the template matching is not easily affected by illumination, noise and partial occlusion, and uses a mask image to filter unstable areas, so that the stability of sorting and positioning is higher; by sparsely defining the feature points and using processing methods such as pyramids and automatic thread allocation, the processing speed is made faster to meet the high-speed performance requirements of the sorting and positioning equipment; the matching results are used to refine and adjust the posture, so that the accuracy of positioning is higher, thereby being able to better meet the position accuracy requirements of the selected position during sorting; and the needs of efficient sorting equipment for simultaneously sorting and selecting multiple chips of the same type and multiple different chips can be met. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of a semiconductor die high-speed sorting and positioning method based on CCD images of the present invention; Figure 2 It is a detailed flow chart of the semiconductor die high-speed sorting and positioning method based on CCD image in the present invention; Figure 3 It is a schematic diagram of the template image pyramid processing in the present invention; Figure 4 It is a schematic diagram of feature extraction of each layer of template image in the present invention; Figure 5 Schematic diagram of the top-level matching result of the target image pyramid in the present invention; Figure 6 It is the bottom layer matching result of the target image and the enlarged image of some areas in the present invention. DETAILED DESCRIPTION

[0016] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0017] The present invention will be further described below in conjunction with the accompanying drawings.

[0018] like Figure 1 , 2 As shown, a semiconductor die high-speed sorting and positioning method based on CCD images includes: Step S1, the input template image is used to extract matching data information, and the data information mainly includes: edge point information pyramid, template sub-pixel edge points and pyramid layer number.

[0019] Step S2-Step S4 Figure 2 On the left side, step S5 is as follows Figure 2 Right part.

[0020] Step S2, Gaussian filtering is performed on the template image, and pyramid processing is used to traverse and search for areas similar to the template image in the target image, perform template matching, and generate a template image pyramid.

[0021] In the embodiment, template matching is to search for areas similar to the template image in the target image. When the target image is large, searching in the original image will cause too much calculation, resulting in too long matching time. Using pyramid processing is an effective method to speed up the matching speed. Pyramid processing can reduce the calculation amount of matching without reducing the accuracy of matching. After the initial matching estimate is obtained at the top of the pyramid, further matching is performed on the image with higher resolution through a more accurate algorithm. As the number of layers of the pyramid increases, the resolution of the image decreases, and the calculation amount is correspondingly reduced, thereby improving the efficiency of matching and ensuring the accuracy of matching.

[0022] In the embodiment, a Gaussian pyramid of a template image is generated, such as Figure 3 As shown, the steps for constructing the Gaussian pyramid are: Performing Gaussian filtering on the template image to obtain a filtered image; Downsampling is performed by deleting the even rows and even columns of the filtered image to obtain the next layer of pyramid image; Continue to perform Gaussian filtering and downsampling on the obtained pyramid image until the number of pyramid layers reaches the set number of layers ; Among them, the number of pyramid layers The calculation is as follows: Get the template area according to the size of the template image ; Among them, the template area Obtained from template width and template height; Then execute the loop ,until , get the number of pyramid layers that need to be calculated .

[0023] Step S3, after the template image pyramid is generated, edge information of each layer of the template image pyramid is extracted.

[0024] In the embodiment, the edge information includes pixel coordinates and direction information of edge points of each layer of the template image pyramid and sub-pixel coordinates of edge points of the bottom layer template; The process of extracting edge information of each layer of the template image pyramid is as follows: Calculate the Sobel gradient matrix of the image in the horizontal and vertical directions , , the gradient matrix is ​​obtained by convolving the corresponding convolution kernel with the image. The convolution kernels in the horizontal and vertical directions are: ; ; Depend on , Calculate the gradient image of the image : ; In the formula, Represents the position coordinates of a point in the image; According to the gradient image Calculate the threshold for Canny edge detection: Generate a cumulative histogram from the data of the gradient image G, where the horizontal axis of the histogram is the gradient size and the vertical axis is the cumulative value of the number of pixels corresponding to the gradient size; Then, the gradient sizes corresponding to the accumulated values ​​of 0.9 times and 0.3 times the total number of pixels are used as the high threshold and low threshold of Canny edge detection respectively. After the thresholds are obtained, the Canny detection algorithm is used to obtain the edge image of the template image. The image pixel coordinates of all edge points can be obtained from the edge image , convert pixel coordinates to relative coordinates relative to the center point of the template: ; ; In the formula, and are the width and height of the current layer template image respectively, and the direction of the edge point is determined by the direction gradient of the edge point image pixel coordinates , The values ​​at the corresponding positions are: ; ; Through the above, the edge information data of each layer of the template pyramid is obtained, and then the template edge point map is obtained, where the coordinates are the relative coordinates of the edge points relative to the center point of the template image , the Sobel gradient values ​​in the horizontal and vertical directions ; Processing mask images: In actual production, the internal circuits of the chip are complex, and a large number of edge points will be extracted. At the same time, after the chip goes through the cutting process, the outermost edge is usually unstable and inconsistent. In order to speed up the subsequent matching and increase the robustness of the method, after obtaining the edge information data of each layer of the template pyramid, set the mask image to filter the unnecessary areas: Performing pyramid operation on the mask image to obtain a mask image pyramid image corresponding to each template layer; Then the mask pyramid image is converted into a binary image of 0 and 1, and the obtained template edge point image is multiplied with the mask image to obtain the edge image after being "filtered" by the mask image; The edge points of the edge image are further processed to extract the direction data from the edge information in the image: Loop through each pixel in the entire image. If the point belongs to the edge (non-zero value), perform a depth-first search to find and connect all edge points connected to the current point, exploring four directions (up, down, left, right, and right) and eight directions (diagonal directions) respectively, ensuring that all connected edge points are marked and added to the current edge segment; Then filter the edge segment of the current layer. The filtering method is as follows: By input parameters and calculate, , by comparing each edge segment with Compare and filter shorter edges, where Indicates the shortest edge segment threshold of the current layer, Indicates the current pyramid layer index; Further calculations show that ,in, represents the filtering step size of the corresponding layer, Indicates the size, determined by the input, Indicates the total number of pyramid layers; Calculate the distance between each edge segment and the center point and Axis offset and : By calculating the center point of the template graph: , the center point of the current layer ,in, and denote rows and columns respectively, and Obtained from the template graph, and Each point and The difference is obtained; Then, calculate , , and Respectively expressed in and The gradient component in the direction, Indicates the point position, according to and Gradient component in the direction, calculate the modulus of the gradient , that is, the gradient strength of each point; Finally , , , and The orientation information is saved to the model; Then calculate the sub-pixel edge of the bottom layer of the template pyramid, which is the original image: After obtaining the pixel edge of the bottom layer template, a sub-pixel accurate edge extraction algorithm based on the facet model is used. Assume It is the coordinate of a pixel point on the edge of the pixel. The facet model converts the template gradient image Fit a three-dimensional surface at this coordinate: ; In the formula, is the image coordinate, is the pixel gradient, , , , and Are coefficients, obtained by convolving the kernel at the pixel position: ; ; ; ; ; Through the above 5 kernels and gradient images China-Israel The dot product of the 3*3 area centered at the pixel position gets the coefficient. After getting the fitted surface function, we get the gradient direction of the pixel: ; ; The sub-pixel precise position coordinates of the pixel position are and It can be calculated by the following formula: ; ; in: ; The calculated coordinates are considered valid sub-pixel coordinates when the following conditions are met: ; ; All pixels at the bottom of the template pyramid are subjected to the above calculation and the effective sub-pixel coordinates are extracted to obtain the sub-pixel edge feature points, such as Figure 4 As shown, the sub-pixel feature points of each layer of template image are displayed.

[0025] Step S4, save the data information used for matching as a model file, the data information includes the template width , template height , the number of pyramid layers , models of each layer of the pyramid, sub-pixel feature points of each layer of the pyramid.

[0026] Step S5, after generating the model file according to the template image, matching is performed in the target image according to the model data to obtain a matching result.

[0027] In the embodiment, the matching result is obtained by matching the model data in the target image, and the matching process is as follows: The first step is Gaussian filtering: Use Gaussian filtering to remove noise from the target image; Step 2: Pyramid processing: Perform pyramid processing on the target image. The number of pyramid layers is the same as the number of edge information pyramid layers in the model. same; The third step is the initial matching of the top layer of the pyramid: After obtaining the target image pyramid, the top image of the target pyramid is processed with the top image of the edge information pyramid to obtain the initial matching result, namely: For the edge point data at the top layer of the edge information pyramid, it is rotated and scaled to obtain a set of edge information for matching. The transformed edge information is: ; ; ; ; In the formula, , , and are the information of the original edge points, and are the position and direction information of the transformed points, respectively, where: ; ; ; In the formula, represents the scaling factor, Indicates the rotation angle; Perform the above transformation on all edge points to obtain edge information after rotation and scaling; The edge information of all combinations of rotation angles and scaling is matched with the top layer of the target image pyramid. The matching process is: Get all the combinations of rotation angles and scaling factors according to the input parameters, where the rotation angle is determined by the initial rotation angle, rotation range, and angle step. The initial rotation angle and rotation range are input parameters, and the angle step is determined by the size of the template image: ; In the formula, represents the angle step, and are the width and height of the template image, respectively. is the number of pyramid layers; The initial and final values ​​of the scaling factor are determined by the input parameters, and the scaling step length The calculation is obtained by calculating the template image size of different chips: ; In the formula, , , and Respectively represent the width and height of the target image, and the width and height of the template image; Calculate the top step of the pyramid : ; Calculate the number of scale dimensions : ,generate The discrete scale of the quantity is obtained by the same calculation method. The number of discrete angles is obtained, and then all the combinations of rotation angles and scaling factors of the top layer are obtained, where represents the initial value of the scaling factor, Indicates the end value of the scaling factor; Calculate the baseline scores for each layer using linear layer decay : ; In the formula, is the input parameter; Then calculate the minimum score requirements for each layer ,in, is an input parameter, indicating the greediness, and adjusting the score attenuation of each layer; A matching score map is established to store the matching scores of the corresponding image coordinates, which is equal to the top image size of the target image pyramid; In the image coordinates The matching score at position is calculated as: ; In the formula, represents the number of edge points, , , and Respectively represent The relative coordinate information and direction information of the edge points, Indicates the position of the target image orientation in horizontal and vertical directions; If a feature is missing in the model or image, the noise will lead to random direction vectors, which on average will not contribute anything to the sum, which is why it is robust to partial occlusion and noise clutter. Even if the light source is controlled in the actual implementation, different batches of products will cause changes in ambient light, but due to the normalization of the direction vector, it will be more robust to lighting changes; Traverse to get the matching scores of all positions, and compare them with the score threshold. The score greater than the score threshold is taken as the initial matching result. Find all matching results under all combinations of rotation angles and scaling factors to get a set of initial matching results. The matching result data includes the rotation angle, scaling factor and image coordinate position, such as Figure 5 The figure shows the matching result of the top layer of the pyramid. The dot in the figure indicates that the matching score of the position is greater than the score threshold, which is a candidate matching result. In the third step, when there are multiple angles and zooms, these situations are evenly distributed to multiple threads for processing. The distribution method is as follows: First, get the number of hardware CPU cores , obtained by combining all rotation angles and scaling factors of the top layer , calculate the number of calculations each thread needs to perform: ; In the formula, is the remainder, right Take the remainder; Search strategy: For the target image, the number of potential matching result positions is very small compared to the total number of pixels. The amount of calculation required to calculate the matching scores of all positions based on the matching formula is too large. The calculation is terminated early based on the conditions to reduce the amount of calculation. That is, the condition for the initial matching result is that the matching score of the position is greater than the score threshold. : ; In all cumulative items, the value of each item is in Between, assuming For already accumulated Scores for each item: ; In the formula, For all points; If the values ​​of the remaining items are all 1, the sum of the remaining items It is expressed as: ; At this time, if: ; Then, the matching score of the position will not reach the score threshold, and the above conditions can be used to terminate the calculation of most positions in advance, reduce the amount of calculation, and optimize the strategy; Step 4: Track the matching results to the bottom layer: After obtaining a set of initial matching results, track each matching result to the next layer until the matching score is less than the threshold or reaches the bottom layer. The tracking process is as follows: Based on the matching results of the previous layer, determine the matching condition range of the next layer. Assume that the matching results of the previous layer are: matching position , matching angle , scaling factor , then the matching condition at the next level is: Matching position: The corresponding position in the following layer is In the 3*3 area at the center; Matching angle: 3 angle conditions , is the angle step of the current layer; Scaling factor: unchanged; The matching score is calculated based on the matching conditions and positions of the next layer, and the maximum matching score is compared with the threshold. If it is greater than the threshold, the position is used as the matching result to continue tracking to the next layer until the bottom layer. If it is less than the threshold, the tracking is stopped and the matching result is deleted. Step 5: Suppression of overlapping targets: Sort the matching results by score and then calculate the matching results Matches all other The intersection area , according to the input parameters ,if then discard; Step 6: Refine the matching parameters: After tracking all the initial matching results, all the matching results at the bottom level are obtained, and the most suitable one, that is, the one with the largest matching score, is selected from all the matching results belonging to the same target; Refine the matching parameters of the pixel-level matching position of the matching result to obtain the matching pose result: Calculate refinement angle : ; In the formula, Indicates the angle of the current matching result. Indicates the range The independent variable that increases by 1 each time. represents the number of refinements, express When the value is , the angle refinement produces Angles; Will Each angle in is applied to the template model, new template points and directions of each point are calculated, new template information is generated, and then re-matched matching angles and scores; The polynomial fitting based on the least squares method is used to optimize the matching size+1 angles to obtain a more accurate angle; Sub-pixelization, matching the results to match the points Calculate the similarity within a 3x3 range and get the point and ; Based on the prediction of the maximum position of the quadratic surface fitting, the maximum position of the surface is found. The calculation process is as follows: ; In the formula, and are the input coordinates and values, namely and , obtained by using singular value decomposition , , , , and Coefficient, and then the maximum value position on the fitting surface, that is, the sub-pixel position, is obtained by the following formula : ; ; After the above processing, a more refined and accurate pixel position is obtained; Step 7: Filter results: Filter the results according to the input parameter matching score, angle, and target number as needed and output the final results, such as Figure 6 shown.

[0028] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-speed sorting and positioning method for semiconductor bare chips based on CCD images, characterized in that: include: Step S1, input template image; Step S2, performing Gaussian filtering on the template image, and using pyramid processing to traverse and search for areas similar to the template image in the target image, perform template matching, and generate a template image pyramid; Step S3, after generating the template image pyramid, extracting edge information of each layer of the template image pyramid; Step S4, save the data information used for matching as a model file, the data information includes the template width , template height , the number of pyramid layers , models of each layer of the pyramid, sub-pixel feature points of each layer of the pyramid; Step S5, after generating the model file according to the template image, matching is performed in the target image according to the model data to obtain a matching result.

2. The method for high-speed sorting and positioning of semiconductor bare chips based on CCD images according to claim 1, characterized in that: In step S2, a Gaussian pyramid of the template image is generated, and the steps of constructing the Gaussian pyramid are: Performing Gaussian filtering on the template image to obtain a filtered image; Downsampling is performed by deleting the even rows and even columns of the filtered image to obtain the next layer of pyramid image; Continue to perform Gaussian filtering and downsampling on the obtained pyramid image until the number of pyramid layers reaches the set number of layers ; Among them, the number of pyramid layers The calculation is as follows: Get the template area according to the size of the template image ; Among them, the template area Obtained from template width and template height; Then execute the loop ,until , get the number of pyramid layers that need to be calculated .

3. The method for high-speed sorting and positioning of semiconductor bare chips based on CCD images according to claim 1, characterized in that: In step S3, the edge information includes the pixel coordinates and direction information of the edge points of each layer of the template image pyramid and the sub-pixel coordinates of the edge points of the bottom layer template; The process of extracting edge information of each layer of the template image pyramid is as follows: Calculate the Sobel gradient matrix of the image in the horizontal and vertical directions , , the gradient matrix is ​​obtained by convolving the corresponding convolution kernel with the image. The convolution kernels in the horizontal and vertical directions are: ； ； Depend on , Calculate the gradient image of the image : ； In the formula, Represents the position coordinates of a point in the image; According to the gradient image Calculate the threshold for Canny edge detection: Generate a cumulative histogram from the data of the gradient image G, where the horizontal axis of the histogram is the gradient size and the vertical axis is the cumulative value of the number of pixels corresponding to the gradient size; Then, the gradient sizes corresponding to the cumulative values ​​of 0.9 times and 0.3 times the total number of pixels are used as the high threshold and low threshold of Canny edge detection respectively. After the thresholds are obtained, the Canny detection algorithm is used to obtain the edge image of the template image; The image pixel coordinates of all edge points can be obtained from the edge image , convert pixel coordinates to relative coordinates relative to the center point of the template: ； ； In the formula, and are the width and height of the current layer template image respectively, and the direction of the edge point is determined by the direction gradient of the edge point image pixel coordinates , The values ​​at the corresponding positions are: ； ； Through the above, the edge information data of each layer of the template pyramid is obtained, and then the template edge point map is obtained, where the coordinates are the relative coordinates of the edge points relative to the center point of the template image , the Sobel gradient values ​​in the horizontal and vertical directions ; After obtaining the edge information data of each layer of the template pyramid, set the mask image to filter the unnecessary areas: Performing pyramid operation on the mask image to obtain a mask image pyramid image corresponding to each template layer; Then the mask pyramid image is converted into a binary image of 0 and 1, and the obtained template edge point image is multiplied with the mask image to obtain the edge image after being "filtered" by the mask image; The edge points of the edge image are further processed to extract the direction data from the edge information in the image: Loop through each pixel in the entire image. If the point belongs to an edge, perform a depth-first search to find and connect all edge points connected to the current point. Then filter the edge segment of the current layer. The filtering method is as follows: By input parameters and calculate, , by comparing each edge segment with Compare and filter shorter edges, where Indicates the shortest edge segment threshold of the current layer, Indicates the current pyramid layer index; Further calculations, ,in, represents the filtering step size of the corresponding layer, Indicates the size, determined by the input, Indicates the total number of pyramid layers; Calculate the distance between each edge segment and the center point. and Axis offset and : By calculating the center point of the template graph: , the center point of the current layer ,in, and denote rows and columns respectively, and Obtained from the template graph, and Each point and The difference is obtained; Then, calculate , , and Respectively expressed in and The gradient component in the direction, Indicates the point position, according to and Gradient component in the direction, calculate the modulus of the gradient , that is, the gradient strength of each point; Finally , , , and The orientation information is saved to the model; Then calculate the sub-pixel edge of the bottom layer of the template pyramid, which is the original image: After obtaining the pixel edge of the bottom layer template, a sub-pixel accurate edge extraction algorithm based on the facet model is used. Assume It is the coordinate of a pixel point on the edge of the pixel. The facet model converts the template gradient image Fit a three-dimensional surface at this coordinate: ； In the formula, is the image coordinate, is the pixel gradient, , , , and Are coefficients, obtained by convolving the kernel at the pixel position: ； ； ； ； ； Through the above 5 kernels and gradient images China-Israel The dot product of the 3*3 area centered at the pixel position gets the coefficient. After getting the fitted surface function, we get the gradient direction of the pixel: ； ； The sub-pixel precise position coordinates of the pixel position are and It can be calculated by the following formula: ； ； in: ； The calculated coordinates are considered valid sub-pixel coordinates when the following conditions are met: ； ； All pixels at the bottom layer of the template pyramid are subjected to the above calculation and the effective sub-pixel coordinates are extracted to obtain the sub-pixel edge feature points.

4. The method for high-speed sorting and positioning of semiconductor bare chips based on CCD images according to claim 1, characterized in that: In step S5, the matching result is obtained by matching the model data in the target image. The matching process is as follows: The first step is Gaussian filtering: Use Gaussian filtering to remove noise from the target image; Step 2: Pyramid processing: Perform pyramid processing on the target image. The number of pyramid layers is the same as the number of edge information pyramid layers in the model. same; The third step is the initial matching of the top layer of the pyramid: After obtaining the target image pyramid, the top image of the target pyramid is processed with the top image of the edge information pyramid to obtain the initial matching result, namely: For the edge point data at the top layer of the edge information pyramid, it is rotated and scaled to obtain a set of edge information for matching. The transformed edge information is: ； ； ； ； In the formula, , , and are the information of the original edge points, and are the position and direction information of the transformed points, respectively, where: ； ； ； In the formula, represents the scaling factor, Indicates the rotation angle; Perform the above transformation on all edge points to obtain edge information after rotation and scaling; The edge information of all combinations of rotation angles and scaling is matched with the top layer of the target image pyramid. The matching process is: Get all the combinations of rotation angles and scaling factors according to the input parameters, where the rotation angle is determined by the initial rotation angle, rotation range, and angle step. The initial rotation angle and rotation range are input parameters, and the angle step is determined by the size of the template image: ； In the formula, represents the angle step, and are the width and height of the template image, respectively. is the number of pyramid layers; The initial and final values ​​of the scaling factor are determined by the input parameters, and the scaling step length The calculation is obtained by calculating the template image size of different chips: ； In the formula, , , and Respectively represent the width and height of the target image, and the width and height of the template image; Calculate the top step of the pyramid : ; Calculate the number of scale dimensions : ,generate The discrete scale of the quantity is obtained by the same calculation method. The number of discrete angles is obtained, and then the combination of all rotation angles and scaling factors of the top layer is obtained, where represents the initial value of the scaling factor, Indicates the end value of the scaling factor; Calculate the baseline scores for each layer using linear layer decay : ； In the formula, is the input parameter; Then calculate the minimum score requirements for each layer ,in, is an input parameter, indicating the greediness, which adjusts the score attenuation of each layer; A matching score map is established to store the matching scores of the corresponding image coordinates, which is equal to the top image size of the target image pyramid; In the image coordinates The matching score at position is calculated as: ； In the formula, represents the number of edge points, , , and Respectively represent The relative coordinate information and direction information of the edge points, Indicates the position of the target image orientation in horizontal and vertical directions; Traverse to obtain the matching scores of all positions, and compare them with the score threshold. The score greater than the score threshold is taken as the initial matching result. Find all matching results under all combinations of rotation angles and scaling factors to obtain a set of initial matching results. The matching result data includes the rotation angle, scaling factor and image coordinate position. Step 4: Track the matching results to the bottom layer: After obtaining a set of initial matching results, track each matching result to the next layer until the matching score is less than the threshold or reaches the bottom layer. The tracking process is as follows: Based on the matching results of the previous layer, determine the matching condition range in the next layer. Assume that the matching results of the previous layer are: matching position , matching angle , scaling factor , then the matching condition at the next level is: Matching position: The corresponding position in the following layer is In the 3*3 area at the center; Matching angle: 3 angle conditions , is the angle step of the current layer; Scaling factor: unchanged; The matching score is calculated based on the matching conditions and positions of the next layer, and the maximum matching score is compared with the threshold. If it is greater than the threshold, the position is used as the matching result to continue tracking to the next layer until the bottom layer. If it is less than the threshold, the tracking is stopped and the matching result is deleted. Step 5: Suppression of overlapping targets: Sort the matching results by score and then calculate the matching results Matches all other The intersection area , according to the input parameters ,if then discard; Step 6: Refine the matching parameters: After tracking all the initial matching results, all the matching results at the bottom level are obtained, and the most suitable one, that is, the one with the largest matching score, is selected from all the matching results belonging to the same target; Refine the matching parameters of the pixel-level matching position of the matching result to obtain the matching pose result: Calculate refinement angle : ； In the formula, Indicates the angle of the current matching result. Indicates the range The independent variable that increases by 1 each time, represents the number of refinements, express When the value is , the angle refinement produces Angles; Will Each angle in is applied to the template model, new template points and directions of each point are calculated, new template information is generated, and then re-matched matching angles and scores; The polynomial fitting based on the least squares method is used to optimize the matching size+1 angles to obtain a more accurate angle; Sub-pixelization, matching the results to match the points Calculate the similarity within a 3x3 range and get the point and ; Based on the prediction of the maximum position of the quadratic surface fitting, the maximum position of the surface is found. The calculation process is as follows: ； In the formula, and are the input coordinates and values, namely and , obtained by using singular value decomposition , , , , and Coefficient, and then the maximum position on the fitting surface, that is, the sub-pixel position, is obtained by the following formula : ； ； After the above processing, a more refined and accurate pixel position is obtained; Step 7: Filter results: Filter the results according to the input parameter matching score, angle, and target number as needed and output the final result.

5. The method for high-speed sorting and positioning of semiconductor bare chips based on CCD images according to claim 4, characterized in that: In the third step of step S5, when there are multiple angles and zooms, these situations are evenly distributed to multiple threads for processing, and the distribution method is as follows: First, get the number of hardware CPU cores , obtained by combining all rotation angles and scaling factors of the top layer , calculate the number of calculations each thread needs to perform: ； In the formula, is the remainder, right Take the remainder; Search strategy: For the target image, the number of potential matching result positions is very small compared to the total number of pixels. The amount of calculation required to calculate the matching scores of all positions based on the matching formula is too large. The calculation is terminated early based on the conditions to reduce the amount of calculation. That is, the condition for the initial matching result is that the matching score of the position is greater than the score threshold. : ； In all cumulative items, the value of each item is in Between, assuming For already accumulated Scores for each item: ； In the formula, For all points; If the values ​​of the remaining items are all 1, the sum of the remaining items It is expressed as: ； At this time, if: ； Then, the matching score of the position will not be able to reach the score threshold, and the above conditions can be used to terminate the calculation of most positions in advance, reduce the amount of calculation, and optimize the strategy.

Images