A high-precision offset and rotation angle measurement method and system based on chip surface identification

Abstract

This invention discloses a high-precision method and system for measuring the offset and rotation angle of chip surface markings. The method includes: acquiring a template preprocessed image and a preprocessed image to be tested; extracting a template ROI image from the template preprocessed image based on the selected ROI; simultaneously, using a chip marking region extraction algorithm that integrates local adaptive thresholding and multi-scale region growing, extracting the printed marking region in the preprocessed image to be tested, generating the extracted marking image to be tested, a binary mask image, and corresponding coordinates; using an adaptive multi-scale pyramid feature fusion template matching algorithm to extract multi-scale fusion features and constructing a Gaussian pyramid to obtain a pixel-level matching result set; based on the matching result set, using an iterative nearest-point method for sub-pixel optimization, finally outputting sub-pixel-level offset and rotation angle. This invention achieves high-precision and robust measurement of the position and rotation angle of chip printed markings.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and in particular to a high-precision method and system for measuring offset and rotation angles based on chip surface markings. Background Technology

[0002] In the semiconductor packaging and electronics manufacturing industry, the positional and angular accuracy of printed markings on chip surfaces is a key indicator of quality control. Marking misalignment or rotation exceeding tolerance not only affects product appearance but may also interfere with subsequent automated identification, traceability, and assembly processes. Currently, most common marking detection methods are based on traditional image processing techniques, such as template matching and edge detection. While these methods are applicable in scenarios with uniform lighting and simple backgrounds, in real-world industrial environments, chip surfaces often suffer from issues like glare, texture interference, low local contrast, and tiny character sizes, leading to shortcomings in accuracy, robustness, and adaptability of traditional methods.

[0003] In recent years, deep learning-based detection methods have been increasingly applied to industrial vision. However, these methods typically require large amounts of labeled data and have high computational complexity, making them difficult to deploy in real-time on high-speed production lines. Therefore, achieving high-precision and robust detection of sub-pixel-level position and angle of printed markings on chips under complex lighting and texture backgrounds remains a pressing technical challenge.

[0004] In the field of chip surface printed marking detection, existing technical solutions still have several shortcomings in complex scenarios: most existing template matching and positioning technologies are geared towards general targets, corner detection or wafer macro positioning, and have not been specifically optimized for tiny printed markings on chip surfaces. They lack marking extraction and ROI pre-positioning steps, are not robust enough to reflection and texture interference, and are difficult to simultaneously meet the sub-pixel accuracy and high-speed detection requirements of chip marking detection. Summary of the Invention

[0005] Technical Objective: To address the shortcomings of existing technologies, this invention discloses a high-precision method and system for measuring the offset and rotation angle of chip surface markings. By integrating a chip marking extraction algorithm that combines local adaptive thresholding with multi-scale region growing, and combining adaptive multi-scale pyramid feature fusion template matching and sub-pixel ICP optimization, high-precision and robust measurement of the position and rotation angle of printed markings on chips is achieved.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution.

[0007] A high-precision offset and rotation angle measurement method based on chip surface markings, the method comprising:

[0008] (1) Acquire chip image and original template image, and perform unified preprocessing on the two images to obtain preprocessed image to be tested and template preprocessed image; in addition, select the template ROI by drawing a box in the original template image; then, extract the template ROI image from the template preprocessed image according to the selected template ROI.

[0009] (2) Using the template ROI as the initial position, the chip identifier region extraction algorithm that combines local adaptive thresholding and multi-scale region growth is adopted to extract the printed identifier region in the preprocessed image to be tested, and output the extracted identifier image and binary mask image to be tested.

[0010] (3) For the extracted template ROI image and the extracted target image, the adaptive multi-scale pyramid feature fusion template matching algorithm is used to extract multi-scale fusion features, and a Gaussian pyramid is constructed for coarse and fine matching, thereby obtaining a pixel-level target matching result set;

[0011] (4) Based on the identifier matching result set, the iterative nearest point method is used for sub-pixel optimization, and the final output is the sub-pixel level offset. With rotation angle .

[0012] This invention also discloses a high-precision offset and rotation angle measurement system based on chip surface markings, comprising:

[0013] The image acquisition and preprocessing module is used to acquire chip images and original template images, and perform consistent preprocessing on the two images to obtain the preprocessed image to be tested and the preprocessed template image; the module also performs bounding selection on the original template image to obtain the selected template ROI.

[0014] The identifier extraction module is used to extract the template ROI image from the template preprocessing image based on the selected template ROI; with the template ROI as the initial position, the chip identifier region extraction algorithm that integrates local adaptive thresholding and multi-scale region growing is used to extract the printed identifier region in the preprocessing image to be tested, and output the extracted identifier image to be tested and the binary mask image.

[0015] The feature extraction and pyramid construction module is used to extract multi-scale fusion features from the extracted template ROI image and the extracted target image using an adaptive multi-scale pyramid feature fusion template matching algorithm, and construct a Gaussian pyramid for coarse and fine matching, thereby obtaining a pixel-level target matching result set.

[0016] The sub-pixel optimization module performs sub-pixel optimization based on the identifier matching result set using an iterative nearest-point method, ultimately outputting sub-pixel-level offsets. With rotation angle .

[0017] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0018] This invention effectively addresses issues of uneven illumination, reflection, and texture interference by integrating a marker extraction algorithm that combines local adaptive thresholding with multi-scale region growing, achieving robust marker extraction. Furthermore, by combining multi-scale pyramid feature fusion matching and sub-pixel ICP optimization, it achieves sub-pixel-level high-precision measurement of marker position and rotation angle. This method significantly improves the robustness, accuracy, and adaptability of detection while maintaining low computational complexity, making it suitable for high-speed industrial vision inspection scenarios. Compared to existing technologies, this invention is specifically designed for chip surface printed markers. It eliminates background interference through precise extraction, improves efficiency with a coarse-to-fine matching architecture, enhances anti-interference capabilities by integrating multiple features, and achieves integrated high-precision measurement of sub-pixel-level offset and rotation angle through point-to-surface ICP iteration. The overall solution is more aligned with the actual working conditions and accuracy requirements of chip inspection. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0020] Figure 2 The flowchart shows the chip identifier region extraction algorithm that integrates local adaptive thresholding and multi-scale region growing.

[0021] Figure 3 The flowchart is for the adaptive multi-scale pyramid feature fusion template matching algorithm.

[0022] Figure 4 This is a schematic diagram of the original template image.

[0023] Figure 5 This is a schematic diagram of a chip image.

[0024] Figure 6 This is a schematic diagram of the template preprocessing image.

[0025] Figure 7 This is a schematic diagram of the preprocessed image to be tested.

[0026] Figure 8 This is a schematic diagram of the template ROI image.

[0027] Figure 9 This is a schematic diagram of the extraction result after the image to be tested has been processed by the region extraction algorithm. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0029] As attached Figure 1 As shown, this embodiment of a high-precision offset and rotation angle measurement method based on chip surface markings includes the following steps:

[0030] (1) Acquire chip images and original template images, and perform unified preprocessing on the two images to obtain the preprocessed image to be tested and the template preprocessed image; in addition, select the template ROI by drawing a box on the original template image. The template ROI is the location of the preset region of interest, such as the marker position; then, extract the template ROI image from the template preprocessed image according to the selected template ROI.

[0031] During image acquisition, an industrial camera is used to capture images of the chip surface under stable lighting conditions. A defect-free chip image is selected as the original template image. Alternatively, the original template image can be generated from a manually verified defect-free chip sample and stored in a template library for automatic retrieval during subsequent testing of chips of the same model. A new template is created when the product model or labeling changes. The original template image and chip image are attached as follows. Figure 4 and attached Figure 5 As shown. The chip image and the original template image undergo consistent preprocessing, including grayscale conversion, Gaussian filtering, histogram equalization, and normalization, to reduce the impact of noise and illumination variations. The preprocessed template image and the preprocessed image to be tested are shown in the attached figures. Figure 6 and attached Figure 7 As shown in the figure. Subsequently, based on the template ROI selected on the original template image, the template ROI image is extracted from the template preprocessing image. The extracted template ROI image is shown in the attached figure. Figure 8 As shown.

[0032] (2) Using the template ROI as the initial position, the chip identifier region extraction algorithm that combines local adaptive thresholding and multi-scale region growth is adopted to extract the printed identifier region in the preprocessed image to be tested, and output the extracted identifier image and binary mask image to be tested.

[0033] The image of the target image extracted in this step has two uses: first, it is sent to step (3) as an effective input region for adaptive multi-scale pyramid feature fusion template matching; second, it is sent to step (4) to extract the corresponding edges of the image of the target image.

[0034] A chip identifier region extraction algorithm integrating local adaptive thresholding and multi-scale region growing is proposed. This algorithm utilizes adaptive thresholding with local standard deviation, multi-scale stable seed point selection, and dynamic threshold region growing. Combined with region merging and morphological post-processing to output ROI information, it can extract chip identifier regions more completely and accurately under conditions of reflection and texture interference compared to existing methods using fixed thresholds, random seeds, or single-region growing. The specific process for extracting the printed identifier region from the preprocessed image includes:

[0035] (2.1) Input the preprocessed image to be tested and the selected ROI template. Calculate the local adaptive threshold for each pixel in the preprocessed image to be tested, and generate a local adaptive threshold map; pixels Corresponding local adaptive threshold The calculation formula is:

[0036] ,

[0037] in, For pixels The local mean, For local standard deviation, These are adjustable weighting coefficients; such as Figure 2 As shown, the identifier extraction process first performs local adaptive threshold extraction on the preprocessed image to be tested. For each pixel in the image... Take it as the center The neighborhood window (k=5 in this embodiment) is used to calculate the average gray value of the pixels within the window, which is the local mean. and local standard deviation Then, the local adaptive threshold is calculated; the local threshold This serves as the basis for the region growth acceptance determination in steps (2.3) and (2.4), and is used to generate a global local threshold map, providing a basis for pixel admission determination for subsequent region growth.

[0038] (2.2) Based on Canny edge detection and multi-scale stability verification, growth seed points are selected, and multi-scale trial growth is performed on the candidate seeds to determine whether the gray-scale characteristics of the region are stable. If they are unstable, the candidate seeds are discarded and reselected. Each stable final growth seed point independently initializes a growth region and a growth queue to obtain several marker contours.

[0039] In this step, Canny edge detection is performed on the preprocessed image to obtain the marker contours. Uniform grid sampling is performed within the contours at 3-pixel intervals to obtain a set of candidate seed points. For each candidate seed point, trial growth is performed using initial growth regions of three scales: 3×3, 5×5, and 7×7, to verify the stability of the region's grayscale characteristics. If the grayscale characteristics of the connected regions obtained from multi-scale growth are inconsistent, the current candidate seed point is discarded, and a new set of candidate seeds is selected. Only pixels with stable multi-scale trial growth results and consistent grayscale statistical characteristics are retained as final growth seed points. The consistency of the multi-scale trial growth results determines the stability of the connected regions and grayscale consistency, thus filtering out effective seed points. Each final growth seed point independently initializes a growth region and a growth queue, resulting in several marker contours.

[0040] (2.3) Take growth points from the growth queue in a loop and calculate the dynamic growth threshold; if the gray level of the neighborhood meets the dynamic growth threshold, it is included in the region and the region statistics are updated, and the neighborhood points are enqueued, and the loop is repeated; if it does not meet the threshold, it is determined whether the queue is empty. If the queue is not empty, the loop continues; for each identifier contour, the region is grown based on the dynamic growth threshold, and the region statistics are updated periodically during the growth process.

[0041] The current growth point p is retrieved from the growth queue in a loop, and the dynamic growth threshold is calculated by combining the local adaptive threshold obtained in step (2.1). It is determined whether the grayscale of the eight neighboring pixels of the current growth point meets the dynamic growth threshold condition. Only neighboring pixels that meet the condition are included in the current growth region, the grayscale statistics of the region are updated, and they are added to the growth queue. If the neighboring pixels do not meet the threshold condition, it is directly determined whether the growth queue is empty. If the queue is not empty, the point is continuously retrieved and growth is performed in a loop. When the queue is empty, the complete region growth of a single identifier contour is completed. The growth queue is a first-in-first-out set of candidate boundary points, which is used to drive the region growth iteration. The initial growth point is the final growth seed point selected in this step. Pixels added to the queue and to be verified during the iteration process are defined as the current growth point.

[0042] The pixels to be expanded are taken from the growth queue and initially used as the final growth seed point. During the iteration process, the current growth point is defined as... coordinates in pixels Its grayscale value The dynamic growth threshold is calculated using a local adaptive threshold; the dynamic growth threshold The calculation formula is:

[0043] ,

[0044] in, Current growth point Local adaptive threshold, Current growth point grayscale value; This represents the average gray level of the currently grown area. It is a small adjustment factor, typically ranging from 0.1 to 0.2.

[0045] (2.4) Perform region merging and morphological post-processing on the growth results; determine whether the marker is detected in the preprocessed image to be tested. If the marker region is successfully detected, output the extracted marker image to be tested, the binary mask image and the corresponding coordinates; otherwise, output that no marker region is detected in the preprocessed image to be tested, and directly exit the extraction process.

[0046] After completing the region growing operation for all the marker outlines, region merging and morphological post-processing are performed sequentially. Then, it is determined whether the marker has been detected. In this example, the presence of a valid marker region in the preprocessed image is determined based on a connected component area filtering rule: if a connected component exists with a pixel area falling within a preset range, the marker region is successfully detected, and the extracted marker image and binary mask image (character binary mask) are output. ) and the ROI set R={ }.in To create a binary mask for the connected components after filtering, pixels in the connected components are set to 1, and the background is set to 0; R={ The identifier is composed of the geometric parameters (center coordinates, width, height, and initial rotation angle) of the minimum bounding rectangle of each connected component. These parameters serve as the pixel mask and search range constraint input for subsequent matching and ICP optimization formulas. If no valid connected component satisfying the area range is found, the preprocessed image is determined to have no detected identifier region, and the identifier extraction process is terminated.

[0047] Check the current growth point Eight neighboring pixels If the following conditions are met:

[0048] ,

[0049] Then Add to the growth region and growth queue. Among them, For pixels The grayscale value is recalculated every 100 pixels grown. .

[0050] Growing regions that are spatially adjacent and have similar gray-level statistical characteristics are merged. The merging criteria are:

[0051] ,

[0052] in, , These are the average gray values ​​of adjacent growth regions R1 and R2, respectively. In this embodiment, as a preset threshold, .

[0053] After merging, morphological closing operations (3×3 cores) are performed to fill the small holes, and then opening operations (3×3 cores) are performed to remove isolated noise points.

[0054] The area range of the marking is determined based on prior knowledge. The process involves filtering connected components that meet certain criteria. The area filtering targets connected components after merging and morphological processing. For each identified connected component, its minimum bounding rectangle is calculated to obtain its center coordinates. ,width ,high Rotation angle The smallest bounding rectangle corresponds to the marked area, and its center coordinates are... ,width ,high Rotation angle As a feature for identifying the region, the single identifier parameter is encapsulated as... =( All identifiers corresponding to Summarize them to form the ROI set R = { If no valid labeled connected components are found in the preprocessed test image after screening, the extraction process is terminated directly; if a valid labeled region is detected, the extracted test labeled image is obtained by cropping based on the corresponding ROI and output to the subsequent template matching step. In this embodiment, chip labels are visible marks such as characters, numbers, batch numbers, directional marks, and logos formed on the chip surface by printing, laser etching, or inkjet printing.

[0055] In this step, the image of the marker to be tested is attached. Figure 9 As shown.

[0056] (3) For the extracted template ROI image and the extracted target image, the adaptive multi-scale pyramid feature fusion template matching algorithm is used to extract multi-scale fusion features, and a Gaussian pyramid is constructed for coarse and fine matching, thereby obtaining a pixel-level target matching result set;

[0057] In this step, a saliency map is generated based on the extracted template ROI image and binary mask image. An adaptive multi-scale pyramid feature fusion template matching algorithm is used to construct a Gaussian pyramid from the extracted template ROI image and the extracted target image, and multi-scale fusion features are extracted. Weighted similarity calculations are performed layer by layer to obtain pixel-level matching results. The adaptive multi-scale pyramid feature fusion template matching algorithm extracts edge gradient feature maps, local mean feature maps, and local standard deviation feature maps from the extracted template ROI image and the target image to construct a Gaussian pyramid. Fusion weights are adaptively allocated according to the contrast and texture entropy of each layer. Weighted matching is completed based on the template saliency map. This approach differs from existing pyramid construction methods using single gradient features, fixed weights, or no saliency guidance, significantly improving feature robustness and matching effectiveness in complex backgrounds.

[0058] The implementation process of the adaptive multi-scale pyramid feature fusion template matching algorithm is as follows:

[0059] Step 3.1: Input the extracted template ROI images respectively. Extracted image of the identifier to be tested Extract the edge gradient feature maps separately. The edge gradient feature map on the template side is denoted as... The edge gradient feature map of the side to be tested is denoted as ; Edge gradient feature map , Extract the local mean feature map and the local standard deviation feature map respectively, and denoted as the template. , The side to be tested is recorded as , ; Construct 3-5 layer Gaussian pyramids independently for the template and the three types of feature maps to be tested; Template branch synchronization is based on saliency maps. Generate hierarchical saliency weights that correspond one-to-one with the pyramid levels; for each level of the pyramid, use the same adaptive weighting formula to fuse them to obtain the template pyramid level. Layer fusion feature map The pyramid to be tested Layer fusion feature map Below, since the processing logic of the template image and the image under test is the same, only the template side will be described in detail.

[0060] like Figure 3 As shown, the feature extraction and matching stage first extracts the edge gradient feature map. Local mean feature map and local standard deviation feature map And construct a three-tiered Gaussian pyramid.

[0061] Sub-pixel edge points are extracted using the Facet-Steger algorithm. For each edge point, a quadratic surface is fitted to its 3×3 neighborhood:

[0062] ,

[0063] in, The gray-level quadratic surface fitted within a 3×3 neighborhood; For constant terms; , for , First-order coefficients of direction; , , These are the coefficients of the second-order terms; , These are the local coordinates centered on the current pixel;

[0064] The sub-pixel position is obtained by solving for the point where the first derivative of the gradient direction is zero. And the gradient magnitude, forming a gradient feature map .

[0065] For each pixel, calculate the average gray value within its 5×5 neighborhood. and standard deviation .

[0066] right , , Construct three layers of a Gaussian pyramid. The construction process is as follows: ,

[0067] in, For Gaussian kernel, This represents the number of layers in the pyramid.

[0068] In each layer Perform feature fusion to generate the pyramid number 1 Layer fusion feature map The calculation formula is:

[0069] ,

[0070] Among them, weight Adaptive calculations are performed based on the contrast and texture entropy of the image layer. , , The pyramids are respectively Layer edge gradient features, local mean features, and local standard deviation features. Weights The calculation formula is:

[0071] ,

[0072] in, For the first pyramid Layer fusion feature map Contrast For the first pyramid Layer fusion feature map Texture entropy.

[0073] Similarly, the same feature extraction, pyramid construction, and adaptive fusion processes are synchronously executed on the extracted target image to generate a hierarchical fusion feature map for the target side. Simultaneously, a template saliency map is generated only from the extracted template ROI image obtained from the input. This is used for the saliency-weighted similarity calculation in steps 3.2 and 3.3.

[0074] Step 3.2: Construct Gaussian pyramids independently for the three types of feature maps on the template side and the test side respectively; perform adaptive feature fusion layer by layer for the template pyramid and the test pyramid; pre-calculate the integral image, square integral image, and weighted integral image for each layer of the template pyramid; calculate only the integral image and square integral image for each layer of the test pyramid in real time; perform sliding window and multi-angle traversal coarse matching at the top of the pyramid, and fuse multi-feature similarity calculation and hierarchical saliency weighting.

[0075] Step 3.3: Perform coarse matching by sliding window traversal and rotation angle traversal at the top of the pyramid, and integrate multi-feature similarity calculation and saliency weighting; filter multiple candidate groups with similarity scores greater than the threshold τ; pass the candidate positions down layer by layer for each group, perform local fine search at each layer, reduce the rotation angle step size layer by layer, use integral image to accelerate similarity calculation, and update the optimal matching position; repeat the process until the bottom layer of the pyramid is determined.

[0076] Starting from the top of the pyramid, that is, from the lowest resolution, a sliding window traversal is used for coarse matching. The sliding window and rotation angle are traversed, and the multi-feature similarity score between the test window and the template window is calculated window by window. The template window is the fixed region of the template ROI extracted in step (2), and the test window is a sliding sub-window with the same size as the template window within the range of the image to be tested.

[0077] This embodiment adopts a coarse-to-fine hierarchical matching strategy: a large-scale coarse search at the top level and a local fine search at the lower level.

[0078] At the top of the pyramid (level 3), the center of the marker to be tested. Starting from a certain position, a sliding window is used for traversal within a range of ±20 pixels. For each candidate position... and rotation angle Calculate the multi-feature similarity score between the test window and the template window, and use it as the saliency-weighted similarity.

[0079] Retain similarity exceeding the threshold The candidate matches, each candidate containing the location and angle All high-scoring candidates are independently entered into a layer-by-layer refined matching process.

[0080] Multi-feature similarity score The calculation formula is:

[0081] ,

[0082] in, Candidate matching positions, To represent the pixel coordinates inside the template window, The significance weights are derived from the template significance map generated in step 3.1. The corresponding pixel grayscale value is generated using the template binary mask output in step (2). Limit the effective calculation area. and These are the saliency weights of the template and the current child window, respectively. The weighted average.

[0083] Each group of high-scoring candidates independently propagates their positions downwards layer by layer, performing a localized refined search at each layer. Integral graphs are used to accelerate similarity calculations, and the optimal matching position is updated. The top-level candidate position is then mapped to the next layer (layer 2), using the following mapping formula: Within a local small-scale traversal range of ±2 pixels at the mapped position and ±0.5° at the angle, the traversal step size of the angle is reduced layer by layer simultaneously. The integral image can transform the sum of features and the sum of squares of the window from pixel-by-pixel accumulation to a four-point lookup table for fast solution, which can quickly calculate the window mean, variance and similarity denominator, greatly reducing the computational cost. After completing the search of the current layer, it is determined whether the bottom layer of the pyramid has been reached. If not, the matching process continues to iterate downwards.

[0084] Integral graph Defined as:

[0085] ,

[0086] in, The pyramid hierarchical fusion feature map obtained by adaptive weight fusion in step 3.1. , In pixels Eigenvalues ​​at; , For summation indexing, the sum of pixels in any rectangular region can be quickly calculated using an integral image.

[0087] Step 3.4: Obtain multiple sets of pixel-level matching results at the bottom of the pyramid. ,in, Let be the pixel-level coordinates and initial rotation angles corresponding to each group of candidates.

[0088] The adaptive multi-scale pyramid feature fusion template matching algorithm uses the ROI information extracted in step (2) as the initial search benchmark for the image to be tested. It fully executes the top-level large-scale coarse matching, the layer-by-layer coordinate mapping local fine search, the integral image to accelerate weighted similarity calculation, and the layer-by-layer shrinking angle traversal range from coarse to fine matching logic, and finally obtains multiple sets of pixel-level matching positions and rotation angles. Compared with the existing template matching scheme with single-scale full image traversal, no significant weight constraints, and only a single gradient feature, this hierarchical matching architecture can significantly reduce the sliding window calculation and improve the overall detection speed while ensuring positioning accuracy.

[0089] (4) Based on the matching result set, the Iterative Closest Point (ICP) method is used for sub-pixel optimization, and the sub-pixel level offset is finally output. With rotation angle All initial pixel values ​​obtained in step 3.4 are fed into the ICP subpixel optimization process. Based on ICP iteration and minimizing the point-to-surface error, subpixel pose optimization is completed, and the final matching result is output. .

[0090] The implementation process of the Iterative Closest Point (ICP) method includes:

[0091] Step 4.1: Using each set of pixel-level matching results as the initial transformation matrix, map the template edge points to the coordinate system of the image to be tested;

[0092] Match initial values ​​at the pixel level Construct the initial similarity transformation matrix respectively The template edge point set extracted by the template binary mask Transform to the coordinate system of the image under test to obtain the transformed point set. .in, , For the template edge point set, For the first Template edge points and their normal vectors For the Transformed template edge point set;

[0093] Step 4.2: Locate the corresponding point along the normal direction in the gradient map of the image to be tested;

[0094] Gradient map of the image under test In, for each Points in In the direction of the normal Find the point corresponding to the largest gradient magnitude. The search range is ±1 pixel.

[0095] Step 4.3: Construct the point-to-surface distance error function, and obtain the sub-pixel level offset and rotation angle through Gauss-Newton method iterative optimization.

[0096] The formula for calculating the point-to-surface distance error function is:

[0097] ,

[0098] in, Incremental parameters, Indicates a point Apply an incremental transformation, where N is the number of edge points or corresponding points involved in the optimization.

[0099] First-order approximation of the rotation: , Linearize the nonlinear optimization problem and construct the Jacobian matrix. Solve the normal equation:

[0100] ,

[0101] in, This is the error vector.

[0102] The Gauss-Newton method is used for iterative solution, and the transformation parameters are updated. This continues until convergence. The final output is a sub-pixel offset. With rotation angle .

[0103] The solution is obtained iteratively using the Gauss-Newton method, and the update rule is as follows:

[0104] ,

[0105] The maximum number of iterations is set to 20, and the convergence threshold is... After the iteration is complete, the final sub-pixel offset is output. and rotation angle .

[0106] This step uses ICP subpixel optimization based on pixel-level matching results to calculate the point-to-surface distance error. It finds the optimal corresponding point along the normal direction and solves it through Gauss-Newton iteration, while outputting the offset and rotation angle. Compared with traditional point-to-point ICP or conventional least squares optimization, it achieves higher accuracy in pose estimation and effectively reduces the false detection rate.

[0107] This invention, by integrating local adaptive threshold extraction, multi-scale feature fusion matching, and sub-pixel optimization, effectively achieves sub-pixel-level high-precision and robust measurement of the position and rotation angle of printed markings on chips under complex scenarios such as uneven illumination, reflection, and texture interference. This invention significantly improves detection accuracy and stability, reduces false detection and false negative rates, while maintaining low computational complexity, making it suitable for the real-time detection needs of high-speed industrial production lines.

[0108] This invention also discloses a high-precision offset and rotation detection system based on printed markings on the chip surface, comprising:

[0109] The image acquisition and preprocessing module is used to acquire chip images and original template images, and perform consistent preprocessing on the two images to obtain the preprocessed image to be tested and the preprocessed template image; the module also performs bounding selection on the original template image to obtain the selected template ROI.

[0110] The identifier extraction module is used to extract the template ROI image from the template preprocessing image based on the selected template ROI; with the template ROI as the initial position, the chip identifier region extraction algorithm that integrates local adaptive thresholding and multi-scale region growing is used to extract the printed identifier region in the preprocessing image to be tested, and output the extracted identifier image to be tested and the binary mask image.

[0111] The feature extraction and pyramid construction module is used to extract multi-scale fusion features from the extracted template ROI image and the extracted target image using an adaptive multi-scale pyramid feature fusion template matching algorithm, and construct a Gaussian pyramid for coarse and fine matching, thereby obtaining a pixel-level target matching result set.

[0112] The sub-pixel optimization module performs sub-pixel optimization based on the identifier matching result set using an iterative nearest-point method, ultimately outputting sub-pixel-level offsets. With rotation angle .

[0113] The core improvement of this invention lies in the adoption of a multi-algorithm fusion and complementarity approach, which organically integrates local adaptive threshold extraction, multi-scale region growth, adaptive multi-scale pyramid feature matching, and point-to-surface ICP sub-pixel optimization. The preceding algorithm provides better input for the following algorithm, and the following algorithm makes up for the shortcomings of the preceding algorithm. This overcomes the problems of low accuracy and poor robustness of single traditional algorithms under uneven lighting and reflective interference, and realizes high-precision offset and rotation angle measurement of chip surface markings.

[0114] The present invention also discloses an electronic device, the device comprising: a memory for storing a computer program; and a processor for executing the computer program to cause the device to perform the aforementioned method.

[0115] The present invention also provides a computer storage medium on which a computer program is stored, wherein when the computer program is run, a device running the computer program implements the aforementioned method.

[0116] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium. The memory can be various types of memory, such as random access memory, read-only memory, flash memory, etc., such as read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which can be a personal computer, server, or network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0117] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A high-precision method for measuring offset and rotation angles based on chip surface markings, characterized in that, The methods include: (1) Acquire chip images and original template images, and perform unified preprocessing on the two images to obtain the preprocessed image to be tested and the preprocessed template image; Additionally, the selected template ROI is obtained by drawing a bounding box around the original template image; Then, the template ROI image is extracted from the template preprocessing image based on the selected template ROI. (2) Using the template ROI as the initial position, the chip identifier region extraction algorithm that combines local adaptive thresholding and multi-scale region growth is adopted to extract the printed identifier region in the preprocessed image to be tested, and output the extracted identifier image and binary mask image to be tested. (3) For the extracted template ROI image and the extracted target image, the adaptive multi-scale pyramid feature fusion template matching algorithm is used to extract multi-scale fusion features, and a Gaussian pyramid is constructed for coarse and fine matching, thereby obtaining a pixel-level target matching result set; (4) Based on the identifier matching result set, the iterative nearest point method is used for sub-pixel optimization, and the final output is the sub-pixel level offset. With rotation angle .

2. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 1, characterized in that: The chip identifier region extraction algorithm that integrates local adaptive thresholding and multi-scale region growing includes the following steps: (2.1) Input the preprocessed image to be tested and the selected template ROI. Calculate the local adaptive threshold for each pixel in the preprocessed image to be tested and generate a local adaptive threshold map. (2.2) Based on Canny edge detection and multi-scale stability verification, growth seed points are selected, and multi-scale trial growth is performed on the candidate seeds to determine whether the gray-scale characteristics of the region are stable. If they are unstable, the candidate seeds are discarded and reselected. Each stable final growth seed point independently initializes a growth region and a growth queue to obtain several marker contours. (2.3) Take growth points from the growth queue in a loop and calculate the dynamic growth threshold; if the gray level of the neighborhood meets the dynamic growth threshold, it is included in the region and the region statistics are updated, and the neighborhood points are enqueued, and the loop is repeated; if it does not meet the threshold, it is determined whether the queue is empty. If the queue is not empty, the loop continues; for each identifier contour, the region is grown based on the dynamic growth threshold, and the region statistics are updated periodically during the growth process. (2.4) Perform region merging and morphological post-processing on the growth results; determine whether the marker is detected in the preprocessed image to be tested. If the marker region is successfully detected, output the extracted marker image to be tested, the binary mask image and the corresponding coordinates; otherwise, output that no marker region is detected in the preprocessed image to be tested, and directly exit the extraction process.

3. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 2, characterized in that: The formula for calculating the dynamic growth threshold is: ， in, Current growth point Local adaptive threshold, Current growth point grayscale value; This represents the average gray level of the currently grown area. It is a regulating factor.

4. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 1, characterized in that: The implementation process of the adaptive multi-scale pyramid feature fusion template matching algorithm includes: Step 3.1: Extract three types of feature maps—edge gradient, local gray-level mean, and local standard deviation—from the extracted template ROI image and the extracted target image, respectively. The template side is denoted as... The side to be tested is denoted as , Additionally, a saliency map is generated based on the input binary mask image and the extracted template ROI image. ; Step 3.2: Construct Gaussian pyramids independently for the three types of feature maps on the template side and the test side respectively; perform adaptive feature fusion layer by layer on the template pyramid and the test pyramid; calculate the integral image, square integral image, and weighted integral image for each layer of the template pyramid, and calculate the integral image and square integral image for each layer of the test pyramid. Step 3.3: Perform coarse matching by sliding window traversal and rotation angle traversal at the top of the pyramid, and integrate multi-feature similarity calculation and saliency weighting; filter multiple candidate groups with similarity scores greater than the threshold τ; pass the candidate positions down layer by layer for each group, perform local fine search at each layer, reduce the rotation angle step size layer by layer, use integral image to accelerate similarity calculation, and update the optimal matching position; repeat the process until the bottom layer of the pyramid is determined. Step 3.4: Obtain multiple sets of pixel-level matching results at the bottom of the pyramid. ,in, The initial values ​​of pixel-level coordinates and rotation angles for each group of candidates are given.

5. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 4, characterized in that: For the extracted template ROI image, the Gaussian pyramid... Layer fusion feature map The calculation formula is: ， Among them, weight Adaptive calculations are performed based on the contrast and texture entropy of the image layer. , , The pyramids are respectively Layer edge gradient characteristics, local mean characteristics, and local standard deviation characteristics.

6. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 5, characterized in that: Weight The calculation formula is: ， in, For the first pyramid Layer fusion feature map Contrast For the first pyramid Layer fusion feature map Texture entropy.

7. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 4, characterized in that: Multi-feature similarity score The calculation formula is: ， in, For significance weight, and These are the saliency weights of the template and the current child window, respectively. The weighted average under the following conditions For the first Gaussian pyramid Layer fusion feature map.

8. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 1, characterized in that: The implementation process of the iterative nearest point method includes: Step 4.1: Construct an initial transformation matrix for each group of pixel-level matching results, and map the template edge points to the coordinate system of the image to be tested; Step 4.2: Locate the corresponding point along the normal direction in the gradient map of the image to be tested; Step 4.3: Construct the point-to-surface distance error function, and obtain the sub-pixel level offset and rotation angle through Gauss-Newton method iterative optimization.

9. The high-precision offset and rotation angle measurement method based on chip surface markings according to claim 8, characterized in that: The formula for calculating the point-to-surface distance error function is: ， in, For incremental parameters, Indicates a point Apply incremental transformation, For the transformed point set In the diagram, N represents the number of edge points or corresponding points participating in the optimization. The direction of the normal.

10. A high-precision offset and rotation angle measurement system based on chip surface markings, characterized in that, include: The image acquisition and preprocessing module is used to acquire chip images and original template images, and to perform consistent preprocessing on the two images to obtain the preprocessed image to be tested and the preprocessed template image. The selected ROI is obtained by drawing a bounding box around the original template image; The identifier extraction module is used to extract the template ROI image from the template preprocessing image based on the selected template ROI. Using the template ROI as the initial position, a chip identifier region extraction algorithm that integrates local adaptive thresholding and multi-scale region growing is adopted to extract the printed identifier region in the preprocessed image to be tested, and output the extracted identifier image and binary mask image to be tested. The feature extraction and pyramid construction module is used to extract multi-scale fusion features from the extracted template ROI image and the extracted target image using an adaptive multi-scale pyramid feature fusion template matching algorithm, and construct a Gaussian pyramid for coarse and fine matching, thereby obtaining a pixel-level target matching result set. The sub-pixel optimization module performs sub-pixel optimization based on the identifier matching result set using an iterative nearest-point method, ultimately outputting sub-pixel-level offsets. With rotation angle .

Images